The issue mentioned above (The NPE when the last item was already written) has been fixed in the version 464 in the public svn. https://code.google.com/p/appengine-mapreduce/source/detail?r=464
On Mon, May 20, 2013 at 6:32 PM, Tom Kaitchuck <[email protected]>wrote: > This is a bug I am working on. It occurs when the last record to be > written encounters a keyOrderingException (Meaning it was already written > but an ACK was not received so it was retried.) So it should be rare and > the retry of the shuffle that was added should cause it to be harmless. > > If you are seeing any broader problems send me a message off-list with > your appId. > > I'll post an update here when a patch has been pushed out. > > > > On Mon, May 20, 2013 at 4:28 PM, Tim Jones <[email protected]> wrote: > >> After upgrading, I'm getting the following NullPointerException in the >> InMemoryShuffler. Is this a known issue? >> >> Caused by: java.lang.NullPointerException >> at >> com.google.appengine.tools.mapreduce.impl.InMemoryShuffleJob$1.run(InMemoryShuffleJob.java:234) >> at >> com.google.appengine.tools.mapreduce.impl.InMemoryShuffleJob$1.run(InMemoryShuffleJob.java:231) >> at >> com.google.appengine.tools.mapreduce.impl.util.RetryHelper.doRetry(RetryHelper.java:62) >> at >> com.google.appengine.tools.mapreduce.impl.util.RetryHelper.runWithRetries(RetryHelper.java:101) >> at >> com.google.appengine.tools.mapreduce.impl.InMemoryShuffleJob.closeFinally(InMemoryShuffleJob.java:231) >> at >> com.google.appengine.tools.mapreduce.impl.InMemoryShuffleJob.writeOutput(InMemoryShuffleJob.java:227) >> at >> com.google.appengine.tools.mapreduce.impl.InMemoryShuffleJob.writeOutputs(InMemoryShuffleJob.java:243) >> at >> com.google.appengine.tools.mapreduce.impl.InMemoryShuffleJob.run(InMemoryShuffleJob.java:253) >> at >> com.google.appengine.tools.mapreduce.impl.InMemoryShuffleJob.run(InMemoryShuffleJob.java:42) >> ... 47 more >> >> On Wednesday, May 1, 2013 1:32:51 PM UTC-7, Tom Kaitchuck wrote: >>> >>> If you are using the experimental Java MapReduce library for App Engine, >>> you are strongly encouraged to update to the latest version of the library >>> in the public svn: https://code.google.com/p/** >>> appengine-mapreduce/source/**checkout<https://code.google.com/p/appengine-mapreduce/source/checkout> >>> >>> >>> Background: >>> >>> We are rolling out a fix to a long standing interaction bug between the >>> experimental MapReduce library and the experimental Files API that, in >>> certain circumstances, results in dropped data. Specifically this bug can >>> cause some records emitted by the Map to be excluded from the input to >>> Reduce. >>> >>> The bugfix involves patches to both the Files API and Java MapReduce. >>> Unfortunately older versions of the Java MapReduce library running against >>> the patched Files API will drop Map output under more common circumstances. >>> The Files API fix will roll out on its own (no action required by you), but >>> in order to avoid dropped data you must update to the latest version of >>> the Java MapReduce >>> library<https://code.google.com/p/appengine-mapreduce/source/checkout> >>> . >>> >>> <https://code.google.com/p/appengine-mapreduce/source/checkout> >>> >>> We apologize for the trouble. Rest assured we are working aggressively >>> to move MapReduce into a fully supported state. >>> >>> >>> Tom Kaitchuck on behalf of the Google App Engine Team >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine Pipeline API" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> >> For more options, visit https://groups.google.com/groups/opt_out. >> >> >> > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
