To get your data as CSV from a GAE app you need to: 1) enable remote api in app.yaml 2) create bulkloader config file: > $GAE_DIR/appcfg.py create_bulkloader_config --filename=config.yaml -e $EMAIL --url=https://$APP.appspot.com/remote_api > rm bulkloader* 3) edit config.yaml as needed (mainly export_transform functions for your fields) 4) dump specific entity kinds to CSV: KINDS="abc efgh ijk" for KIND in $KINDS: do rm $KIND.csv $GAE_DIR/appcfg.py download_data --filename=$KIND.csv --kind=$KIND \ --config_file=config.yaml -e $EMAIL --url=https://$APP.appspot.com/remote_api done rm bulkloader*
If you have large entities and the get error messages from the csv module: 1) put the following into a file e.g. csv_limits.py: import csv csv.field_size_limit(1048576) 2) import the file in config.yaml using: - import: csv_limit You can also try the following to dump everything (not sure if this still works with new releases): $GAE_DIR/appcfg.py download_data -e $EMAIL --application=$APP --url=https://$APP.appspot.com/remote_api --filename=dump.csv To upload the modified CSV data (after previous download) to GAE do: for KIND in $KINDS do echo "x" | $GAE_DIR/appcfg.py upload_data --filename=$KIND.csv --kind=$KIND \ --config_file=config.yaml -e $EMAIL --passin --url=https://$APP.appspot.com/remote_api \ --application $APP --batch_size=1 done rm bulkloader* Hope this helps :)
