One question to research is whether the JDO layer is using a batch put () for those 900 entities or doing them each separate. The low level API supports both batch and individual puts, but I'm not sure how JDO decides which to use at any given time.
I think tx.commit() is when the datastore is actually updated and your code blocks, but that should be easy to check with a few log statements. On Dec 3, 12:55 am, Tim Cooper <[email protected]> wrote: > Thanks, Dominik, but I still need to know more - (I understand that > the "1-10 updates per second" is only per entity group so scaleability > comes with having fine-grained entity groups). > > * What is an "update" exactly? Is it a call to "pm.makePersistent > ()"? Is it a call to "tx.commit()"? Is it a call to "pm.flush()"? > Is it an update of a single field in a single Java object? Is it an > invocation of "doGet()/doPost()"? > > * Suppose you want to do multiple updates within the one entity group > within a single Java function. At what point is the 100ms cost > incurred? Which function call blocks for the 100ms? > > * Is it possible to create 900 child entities in an entity group in a > single Java function call, by structuring this "creation of 900 > objects" as a single "update"? > > * If I don't use transactions, does this "1-10 updates per second" > limitation go away? > > I have written an application which uses a single page request to send > an XML document consisting of 900 objects to my GAE app. My GAE app > then puts this list of 900 items into the datastore in a single entity > group. I am happy with the limitation that only 1-10 people can do > this to the same entity group per second. What I'm not entirely happy > about is that the call fails because it hits the 30 second timeout. I > don't want the timeout increased, I want the page request to be > processed quicker. If all this activity is happening within the one > Java function call, without commit()'s or pm.flush()'s inside the > loop, then shouldn't this be regarded as a single "update"? > > It may be relevant to explain the data model: the head object has a > field: "ArrayList<Student> Students". It's set up as an owned, > bijectional one-to-many relationship, using the annotations described > in the GAE documentation. > I'm able to process 400 of these objects in 20 seconds...i.e. 20 per > second. As mentioned elsewhere, the amount of data involved is small, > ~ 300K and there's only one index involved. This is the sort of thing > that would be processed instantaneously in other technologies. > > On Dec 3, 3:29 pm, Dominik Steiner <[email protected]> > wrote: > > > Tim, > > > you might find some information related to that in this thread > > >http://groups.google.com/group/google-appengine-java/browse_thread/th... > > > HTH > > > Dominik > > > On 1 Dez., 23:42, Tim Cooper <[email protected]> wrote: > > > > Can anyone give me a clearer understanding of what an "update" is, in > > > the statement: "Any single entity or entity group can only be updated > > > about five times a second." > > > > What level is an "update"? Is it a single update to a single field? > > > Is it a set of field updates to a single Java object? Is it a single > > > transaction, updating a single entity group? > > > > I have written my application expecting a "update" to mean a set of > > > updates to a single entity group. In one XML request, I am populating > > > my datastore with 900 child entities, all within a single entity > > > group. The total amount of data involved is about 300K. I thought > > > that this size would be quite manageable for AppEngine, and I wanted > > > transactions to be atomic. However, it appears that this means 900 > > > writes and so this call times out. > > > > The frustrating thing was that my application worked perfectly in the > > > development environment. I thought that that was a good indication > > > that it would work when deployed. It has been difficult to debug it > > > using frequent "deploy"s. -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
