hi scott! if you haven't seen the feature request for this, http://code.google.com/p/googleappengine/issues/detail?id=178 , there's lots of related discussion there.
the short answer is, if you want to efficiently enforce that a given property is unique, you need to include it in the key name. you could write a userland library that would do this within entity groups, as described in the issue, but there's no real way to do it across entity groups, since those are the unit of transactionality in the datastore. you can definitely do what you've done, and it will more or less work. however, it's subject to race conditions, since queries aren't transactional, so it will still let occasional duplicate properties slip in. also, as you've noticed, it can't support high volumes of inserts per request. in general, we wouldn't expect you to do hundreds of your get_or_insert() calls, ie two queries plus a put, in serial, within a single request. On Aug 19, 7:55 pm, Scott <[email protected]> wrote: > Ran into a gotcha with my App today. I have implemented a User Model > that has a couple of properties (facebook id and email address) that > need to be unique across all User instances in the datastore. I > implemented my own "get_or_insert" method that I run in a > transaction. The get_or_insert method queries for either or both > properties (fb uid or email) depending on parameters. If no object is > found, a new one is inserted. > > My problem is there are cases where I want to do this for dozens of > items in one request. The execution time I am seeing on the queries > to check for existing items is prohibitively long. I am getting time > outs when trying to process (get_or_insert) 100 items. I've > experiemented with not using the transaction just to see if it would > change the overall time. The impact was minimal. > > I've experiemented with doing bulk puts (putting a list of Model > instances at once). This was great, but does not help as the queries > are still needed to see if the items exist and to try to enforce > uniqueness. > > Any one have any advice on an alternative approach that may work? > I've thought about doing it asynchronously using the task queue, but > that does not work with the user experience and flow of the > application. Right now, I think a new approach is needed that avoids > creating the objects at this point in the application flow. Here's > the code that implements the get_or_insert and is run as a > transaction: > > def _run_user_get_or_insert(q_by_email, q_by_fbuid, confirm_user, > new_user): > user = None > updated = False > if q_by_email: > user = q_by_email.get() > if not user: > if q_by_fbuid: > user = q_by_fbuid.get() > if not user: > user = new_user > updated = True > if confirm_user and not user.is_confirmed: > user.is_confirmed = True > updated = True > if updated: > user.put() > return user > > Running on app engine, it takes ~16s to create 100 new users. > > Thanks for your help. > > -Scott --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~----------~----~----~----~------~----~------~--~---
