John,
It looks like you only need to modify the 'active' user objects. In that
case, you would want to use a Query that filters on the 'active' field,
rather than fetching all User objects via an Extent.
In addition, as your app scales it will probably make sense to break up
your work into smaller batches using the Task Queue and Query Cursors (
http://code.google.com/appengine/docs/java/datastore/jdo/queries.html#Query_Cursors).
Fetch a batch of the user objects. Then, use the query cursor--
obtained from the query results-- to enqueue a task to process the next
batch of objects starting at that cursor point, and so on. To speed things
up, you can enqueue the next task before you start processing the objects
returned from the current query.
On Fri, Nov 11, 2011 at 5:47 AM, John Clarke <clarke...@gmail.com> wrote:
> Hi,
>
> I'm trying to optimize my use of the Datastore. One of my slowest
> operations reads all of the entities from a table and updates the
> information in them all. I currently do this like so:
>
>
> ========================
> PersistenceManager pm = PMF.get().getPersistenceManager();
> Extent extent = pm.getExtent(Users.class);
>
> for(Object userObj : extent) {
> User user = (User)userObj;
>
> // only update active users
> if(user.isActive()) {
> //
> // in here I set some fields for each active user
> //
> }
>
> }
>
> // closs all fields and automatically save all the User objects
> extent.closeAll();
>
> ========================
>
>
> My User class is a POJO with various getters/setters for the data. It
> contains 13 fields and the primary key is a string.
>
> What is the best way to improve this? The User table currently only
> has 100 or so users but it could potentially grow to thousands. This
> operation runs every 10 minutes via cron and it takes about 30-40
> seconds to complete!!
>
> What happens when extent.closeAll() is called? will it do multiple
> writes or is it a batch save?
>
> I'm currently using GAE v1.5.1 but am looking to move to v1.6 in the
> near future.
>
> Thanks,
> J
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine for Java" group.
> To post to this group, send email to
> google-appengine-java@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine-java+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine-java?hl=en.
>
>
--
You received this message because you are subscribed to the Google Groups
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
