In the context of that test (PCollectionGetSizeIT), the EOFException is expected-- there isn't any data to read b/c the file is empty. We could change up the impl to throw a CrunchRuntimeException instead.
For the cogroup test, that sounds suspiciously like a deep copy issue. Gabriel is the most knowledgable person to ask, he always fixes my deep copy errors. :) Josh On Tue, Apr 2, 2013 at 10:33 AM, Matthias Friedrich <[email protected]> wrote: > Hi, > > I'm a bit worried about the IO exceptions we get in our integration > test (see [1], grep for EOFException). Since we don't validate the > results, there may be data loss, at least in local mode. > > I have stumbled upon a test case [2] where Crunch seems to reliably > lose some data when using Writables, while the same code works fine > with Avros. This time, I was able to reproduce this on two machines > (with slightly different error messages, ie. [3]). > > Has anyone seen this before? Is anyone able to reproduce the error > or is it just me? :) > > Thanks, > Matthias > > [1] > https://builds.apache.org/job/Crunch-master-integration/244/consoleFull > [2] http://users.mafr.de/~matthias/crunch/CogroupIT.patch > [3] http://users.mafr.de/~matthias/crunch/CogroupIT-error.txt >
