Hi Nick, Thanks for the reply. I've got a few question below that I'm hoping you can answer.
On 10 March 2010 22:10, Nick Johnson (Google) <nick.john...@google.com>wrote: > Hi Nickolas, > > On Wed, Mar 10, 2010 at 12:28 AM, Nickolas Daskalou <n...@daskalou.com>wrote: > >> I remember reading a thread not long ago ( >> http://groups.google.com/group/google-appengine/browse_thread/thread/f69fe7dc4a9bc2ec) >> about how put()'ing entities with near-sequential key names or IDs could >> result in Datastore contention, since those entities would most likely be >> saved on the same Bigtable tablet. >> >> One solution in that thread to avoid Datastore contention mentioned >> setting the key names to a hash (thus achieving non-sequential key names). >> >> My question is does this sequential-causes-contention rule also apply to >> the indexes updated when the entities are put()? I'm assuming the answer is >> Yes. >> > > In theory, yes. However, it would take a much higher write rate than for > entity contention before it becomes an issue - so high that you likely do > not have to worry about it. > Is it higher than the "100s of qps" mentioned in that other thread? Is there a rough figure you can give? I've personally observed instances of write contention with sequential key > IDs, where a datetime was also being indexed, and contention with entity > writes was encountered long before any issues with the index would have > become a problem. > What was your qps rate before you encountered contention issues? Have you tried the same thing with non-sequential key IDs? If so, what was the qps rate you were able to achieve without contention issues on the datetime index? Nick > -Nick Johnson > > >> An example of what I mean. Let's say I'm taking snapshots of integer >> values at particular points in time (say, every 5 minutes). So my model >> looks like this: >> >> class Snapshot(db.Model): >> datetime = db.DateTimeProperty() >> value = db.IntegerProperty() >> >> def new_snapshot(d,v): >> return Snapshot(datetime=d, value=v, key_name=sha1('%s-%s-%s %s:%s,%s' % >> (d.year,d.month,d.day,d.hour,d.min,v)) >> >> If I then create 1000 entities which all have the same datetime value but >> all have different value values, will I still encounter Datastore >> contention, due to the datetime index being updated with 1000 values that >> are identical, and hence (if my understanding of indexes is correct) will >> most likely be written onto the same tablet? >> >> Nick >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To post to this group, send email to google-appeng...@googlegroups.com. >> To unsubscribe from this group, send email to >> google-appengine+unsubscr...@googlegroups.com<google-appengine%2bunsubscr...@googlegroups.com> >> . >> For more options, visit this group at >> http://groups.google.com/group/google-appengine?hl=en. >> > > > > -- > Nick Johnson, Developer Programs Engineer, App Engine > Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: > 368047 > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com<google-appengine%2bunsubscr...@googlegroups.com> > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
