So my question is that do hadoop 0.20 and 1.0.3 differ in their support of writing or reading sequencefiles? same code works fine with hadoop 0.20 but problem occurs when run it under hadoop 1.0.3.
On Sun, May 27, 2012 at 6:15 PM, waqas latif <waqas...@gmail.com> wrote: > But the thing is, it works with hadoop 0.20. even with 100 x100(and even > bigger matrices) but when it comes to hadoop 1.0.3 then even there is a > problem with 3x3 matrix. > > > On Sun, May 27, 2012 at 12:00 PM, Prashant Kommireddi <prash1...@gmail.com > > wrote: > >> I have seen this issue with large file writes using SequenceFile writer. >> Not found the same issue when testing with writing fairly small files ( < >> 1GB). >> >> On Fri, May 25, 2012 at 10:33 PM, Kasi Subrahmanyam >> <kasisubbu...@gmail.com>wrote: >> >> > Hi, >> > If you are using a custom writable object while passing data from the >> > mapper to the reducer make sure that the read fields and the write has >> the >> > same number of variables. It might be possible that you wrote datavtova >> > file using custom writable but later modified the custom writable (like >> > adding new attribute to the writable) which the old data doesn't have. >> > >> > It might be a possibility is please check once >> > >> > On Friday, May 25, 2012, waqas latif wrote: >> > >> > > Hi Experts, >> > > >> > > I am fairly new to hadoop MapR and I was trying to run a matrix >> > > multiplication example presented by Mr. Norstadt under following link >> > > http://www.norstad.org/matrix-multiply/index.html. I can run it >> > > successfully with hadoop 0.20.2 but I tried to run it with hadoop >> 1.0.3 >> > but >> > > I am getting following error. Is it the problem with my hadoop >> > > configuration or it is compatibility problem in the code which was >> > written >> > > in hadoop 0.20 by author.Also please guide me that how can I fix this >> > error >> > > in either case. Here is the error I am getting. >> > > >> > > in thread "main" java.io.EOFException >> > > at java.io.DataInputStream.readFully(DataInputStream.java:180) >> > > at java.io.DataInputStream.readFully(DataInputStream.java:152) >> > > at >> > > org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1508) >> > > at >> > > >> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1486) >> > > at >> > > >> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1475) >> > > at >> > > >> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1470) >> > > at TestMatrixMultiply.fillMatrix(TestMatrixMultiply.java:60) >> > > at TestMatrixMultiply.readMatrix(TestMatrixMultiply.java:87) >> > > at TestMatrixMultiply.checkAnswer(TestMatrixMultiply.java:112) >> > > at TestMatrixMultiply.runOneTest(TestMatrixMultiply.java:150) >> > > at TestMatrixMultiply.testRandom(TestMatrixMultiply.java:278) >> > > at TestMatrixMultiply.main(TestMatrixMultiply.java:308) >> > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> > > at >> > > >> > > >> > >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) >> > > at >> > > >> > > >> > >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >> > > at java.lang.reflect.Method.invoke(Method.java:597) >> > > at org.apache.hadoop.util.RunJar.main(RunJar.java:156) >> > > >> > > Thanks in advance >> > > >> > > Regards, >> > > waqas >> > > >> > >> > >
