There is a somewhat new service to export/import datastore entities: https://cloud.google.com/datastore/docs/export-import-entities#starting_managed_export_and_import_operations (which previously could be done from the "datastore admin"). The documentation says:
"The output of a managed export uses the LevelDB log format." and links to this page: https://github.com/google/leveldb/blob/master/doc/log_format.md However this doesn't seem to be entirely accurate. They seem to be doing some funky things with the CRC for one and marking the end of the file isn't in accordance with the documentation. Fortunately the Cloud SDK has code which produces such dumps (see RecordsWriter in google/appengine/ext/mapreduce/records.py) and furthermore has a reader (!). So here is a small python snippet to read the models from such a dump: import sys sys.path.append('/usr/lib/google-cloud-sdk/platform/google_appengine') from google.appengine.datastore import entity_pb from google.appengine.ext.mapreduce import records from google.appengine.ext import ndb class TestModel(ndb.Model): # we need the definition of the model we want to read foobar = ndb.StringProperty(indexed=False) with open(sys.argv[1], 'rb') as f: for r in records.RecordsReader(f, strict=True): entity = TestModel._from_pb(entity_pb.EntityProto(r)) print(entity) -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at https://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/c3273421-3e7f-43cb-b3a0-01ac0e21b87e%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
