Tuesday, April 26, 2016

Caused by: java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java

Problem with Custom value / key class when implementing Writable interface. Error Caused by: java.io.EOFException at java.io.DataInputStream.readInt

Solution: If having any java primitive types in Custom writable class, then while reading or writing
we should use the overloaded method for that type.
for e.g
- writeChars(String s) for String
 - writeInt(int i) for int
- writeLong(long l) for long
- writeBoolean(boolean b) for boolean

Stack-Trace:
java.lang.Exception: java.io.EOFException
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:400)
Caused by: java.io.EOFException
at java.io.DataInputStream.readInt(DataInputStream.java:392)
at com.apple.Comments.MinMaxCountTuple.readFields(MinMaxCountTuple.java:48)
at org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:73)
at org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:44)
at org.apache.hadoop.mapreduce.task.ReduceContextImpl.nextKeyValue(ReduceContextImpl.java:145)

at org.apache.hadoop.mapreduce.task.ReduceContextImpl.nextKey(ReduceContextImpl.java:121)


Custom Class: I am getting this exception with below custom Class

public class MinMaxCountTuple implements Writable {
private int min;
private int max;
private int count;

public void write(DataOutput out) throws IOException {
out.write(this.min);
out.write(this.max);
out.write(this.count);
}
public void readFields(DataInput in) throws IOException {
this.min = in.readInt();
this.max = in.readInt();
this.count = in.readInt();
}

// Changed the write method to below and it fixed.
public void write(DataOutput out) throws IOException {
out.writeInt(this.min);
out.writeInt(this.max);
out.writeInt(this.count);
}

2 comments:

  1. Big data is a term that describes the large volume of data – both structured and unstructured – that inundates a business on a day-to-day basis. big data projects for students But it’s not the amount of data that’s important. Project Center in Chennai It’s what organizations do with the data that matters. Big data can be analyzed for insights that lead to better decisions and strategic business moves.

    Spring Framework has already made serious inroads as an integrated technology stack for building user-facing applications. Corporate TRaining Spring Framework the authors explore the idea of using Java in Big Data platforms.
    Specifically, Spring Framework provides various tasks are geared around preparing data for further analysis and visualization. Spring Training in Chennai


    The Angular Training covers a wide range of topics including Components, Angular Directives, Angular Services, Pipes, security fundamentals, Routing, and Angular programmability. The new Angular TRaining will lay the foundation you need to specialise in Single Page Application developer. Angular Training

    ReplyDelete