: ) I tried the same thing last night, but wasn't ready to declare victory because I saw new errors. Happily, it turns out those were from old tasks (with retry counts nearing 1000) whose states were not compatible with the new MR code. I've purged the queue and cleaned out the associated blobs and everything's humming along now.
When I first started using MR, I had to make a lot of modifications to get it working with py2.7 and NDB, using two different versions of the MR lib that I found. I was hesitant to touch that code since it had been working perfectly for so long, until sometime in February when I started experiencing these errors. I'm still not sure why that happened. But, happily, all I had to do to the svn version of MR this time around was change mapreduce/main.py to mapreduce.main.APP in include.yaml. Nice! On Wed, Mar 6, 2013 at 5:13 AM, bmurr <[email protected]> wrote: > Well, I think I have it solved. > > I was using an older version of the mapreduce library, which uses > mapreduce.lib.files to interact with the blobstore. > The newer version of mapreduce uses google.appengine.api.files instead, > which doesn't cause this problem. > > These two libraries seem pretty similar -- so I'm not sure what precisely > was causing the issue. > > > > On Saturday, March 2, 2013 4:25:17 AM UTC, Jamie Niemasik wrote: > >> I've been receiving intermittent errors from MapReduce jobs. I'm running >> Python 2.7. >> >> The specific error is "BadValueError: name must be under 500 bytes" which >> is raised when calling datastore.Key.from_path() within >> blobstore.get_blob_key(); the filename being provided is way too long to >> make a key from. >> >> This all occurs within the code in the mapreduce package… nothing in my >> code seems to affect it. >> >> Some of the filenames are 288 bytes long, while some are 992. The M/R >> spec name and id in each case is nearly the same and is very short; I don't >> see where this variance comes from. >> >> The sequence of events is this: >> mapreduce.output_writers.init_**job() creates a reasonable, short >> filename and passes it to files.blobstore.create() >> create() calls files.file._create('blobstore'**, …, filename) >> _create() sets up an rpc with that filename and calls >> _make_call('Create', ...) >> >> And that call sometimes returns a filename that's 288 bytes, sometimes >> 992. I have no idea why or how to work around this — any help would be >> appreciated. >> >> Thanks, >> Jamie >> > -- > You received this message because you are subscribed to a topic in the > Google Groups "Google App Engine" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/google-appengine/I0pXHW1poWU/unsubscribe?hl=en > . > To unsubscribe from this group and all its topics, send an email to > [email protected]. > To post to this group, send email to [email protected]. > Visit this group at http://groups.google.com/group/google-appengine?hl=en. > For more options, visit https://groups.google.com/groups/opt_out. > > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
