Thanks Mathew --
The --dump --restore is exactly what I've been looking for. However...
While I was able to export some of my entities (mostly those with only
a handful of items), for entities with a LOT of items I'm seeing the
following:
on the command line from bulkloader.py:
HTTPError: HTTP Error 500: Internal Server Error
[INFO ] Backing off due to errors: 256.0 seconds
and in my error logs:
/remote_api
java.nio.BufferOverflowException
at java.nio.HeapByteBuffer.put(Unknown Source)
at java.nio.ByteBuffer.put(Unknown Source)
at com.google.apphosting.runtime.jetty.RpcResponseGenerator.addContent
(RpcResponseGenerator.java:68)
at org.mortbay.jetty.AbstractGenerator$Output.write
(AbstractGenerator.java:624)
at org.mortbay.jetty.AbstractGenerator$Output.write
(AbstractGenerator.java:585)
at com.google.apphosting.utils.remoteapi.RemoteApiServlet.doPost
(RemoteApiServlet.java:139)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:713)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:806)
at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:
487)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter
(ServletHandler.java:1093)
at org.purplex.lbas.servlet.filter.ResponseHeaderFilter.doFilter
(ResponseHeaderFilter.java:49)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter
(ServletHandler.java:1084)
at com.google.apphosting.utils.servlet.ParseBlobUploadFilter.doFilter
(ParseBlobUploadFilter.java:97)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter
(ServletHandler.java:1084)
at com.google.apphosting.runtime.jetty.SaveSessionFilter.doFilter
(SaveSessionFilter.java:35)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter
(ServletHandler.java:1084)
at
com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter
(TransactionCleanupFilter.java:43)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter
(ServletHandler.java:1084)
at org.mortbay.jetty.servlet.ServletHandler.handle
(ServletHandler.java:360)
at org.mortbay.jetty.security.SecurityHandler.handle
(SecurityHandler.java:216)
at org.mortbay.jetty.servlet.SessionHandler.handle
(SessionHandler.java:181)
at org.mortbay.jetty.handler.ContextHandler.handle
(ContextHandler.java:712)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:
405)
at com.google.apphosting.runtime.jetty.AppVersionHandlerMap.handle
(AppVersionHandlerMap.java:238)
at org.mortbay.jetty.handler.HandlerWrapper.handle
(HandlerWrapper.java:139)
at org.mortbay.jetty.Server.handle(Server.java:313)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:
506)
at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete
(HttpConnection.java:830)
at com.google.apphosting.runtime.jetty.RpcRequestParser.parseAvailable
(RpcRequestParser.java:76)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:381)
at
com.google.apphosting.runtime.jetty.JettyServletEngineAdapter.serviceRequest
(JettyServletEngineAdapter.java:135)
at com.google.apphosting.runtime.JavaRuntime.handleRequest
(JavaRuntime.java:235)
at com.google.apphosting.base.RuntimePb$EvaluationRuntime
$6.handleBlockingRequest(RuntimePb.java:5235)
at com.google.apphosting.base.RuntimePb$EvaluationRuntime
$6.handleBlockingRequest(RuntimePb.java:5233)
at com.google.net.rpc.impl.BlockingApplicationHandler.handleRequest
(BlockingApplicationHandler.java:24)
at com.google.net.rpc.impl.RpcUtil.runRpcInApplication(RpcUtil.java:
363)
at com.google.net.rpc.impl.Server$2.run(Server.java:838)
at com.google.tracing.LocalTraceSpanRunnable.run
(LocalTraceSpanRunnable.java:56)
at com.google.tracing.LocalTraceSpanBuilder.internalContinueSpan
(LocalTraceSpanBuilder.java:536)
at com.google.net.rpc.impl.Server.startRpc(Server.java:793)
at com.google.net.rpc.impl.Server.processRequest(Server.java:368)
at com.google.net.rpc.impl.ServerConnection.messageReceived
(ServerConnection.java:448)
at com.google.net.rpc.impl.RpcConnection.parseMessages
(RpcConnection.java:319)
at com.google.net.rpc.impl.RpcConnection.dataReceived
(RpcConnection.java:290)
at com.google.net.async.Connection.handleReadEvent(Connection.java:
466)
at com.google.net.async.EventDispatcher.processNetworkEvents
(EventDispatcher.java:759)
at com.google.net.async.EventDispatcher.internalLoop
(EventDispatcher.java:205)
at com.google.net.async.EventDispatcher.loop(EventDispatcher.java:
101)
at com.google.net.rpc.RpcService.runUntilServerShutdown
(RpcService.java:251)
at com.google.apphosting.runtime.JavaRuntime$RpcRunnable.run
(JavaRuntime.java:394)
at java.lang.Thread.run(Unknown Source)
Any ideas?
...stephan
On Nov 18, 2:51 pm, Matthew Blain <[email protected]> wrote:
> Hi everyone,
> Thanks for the response so far--we'll be getting back to you over the next
> few days.
> I'd like to point out an existing feature which will be helpful to many of
> you: --dump and --restore. This will download or upload all of
> the entities for a particular Kind and save them in a local sqlite database.
> It's useful for general backup (e.g. a weekly backup), and also useful for
> moving data across applications, such as between your app running on the
> dev_appserver and on App Engine, or to load a staging instance with a known
> set of test data.
>
> You can find more information
> here:http://code.google.com/appengine/docs/python/tools/uploadingdata.html...
>
> Also, for Java developers, there is a remote API handler available; adding
> following to your web xml file should work (disclaimer: I have not yet
> personally tested this.)
>
> <servlet>
> <servlet-name>remoteapi</servlet-name>
>
> <servlet-class>com.google.apphosting.utils.remoteapi.RemoteApiServlet</servlet-class>
> </servlet>
> <servlet-mapping>
> <servlet-name>remoteapi</servlet-name>
> <url-pattern>/remote_api</url-pattern>
> </servlet-mapping>
> <security-constraint>
> <web-resource-collection>
> <web-resource-name>remoteapi</web-resource-name>
> <url-pattern>/remote_api</url-pattern>
> </web-resource-collection>
> <auth-constraint>
> <role-name>admin</role-name>
> </auth-constraint>
> </security-constraint>
>
> --Matthew
>
> On Tue, Nov 17, 2009 at 12:53 PM, Matthew Blain
> <[email protected]>wrote:
>
> > Hi App Engine developers,
> > We're working on some improvements and additions to the bulk loader to
> > make it easier to move data between the App Engine Datastore and other
> > data files you may have. If you're doing this right now, we're looking
> > for some beta testers.
> > Two specific issues we're addressing at this time are:
> > * More language-neutral: Java developers, this will help you!
> > * More format-neutral: If you use some format other than CSV, this
> > will help you!
>
> > If you're interested, fill out the form at
>
> >https://spreadsheets.google.com/viewform?formkey=dC15V2hwczhpZ1VVWFhP...
> > to get started.
>
> > --Matthew
--
You received this message because you are subscribed to the Google Groups
"Google App Engine for Java" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
