I posted this in a different thread for a different reason but here are some numbers on batch putting small entities. Times will vary depending on size of entity, number of properties and indexes on the properties and composite indexes. The first number is the latency or the real world time it took. The second is the (cpu + api) ms cost and the third is the api cost only. So you can see that 1000 items were inserted in just over 1 second in real time.
Results for batch put of 1000 was 1383ms 48493cpu_ms 48400api_cpu_ms Results for batch put of 2000 was 1608ms 97030cpu_ms 96866api_cpu_ms Results for batch put of 10000 was 7791ms 481290cpu_ms 480916api_cpu_ms On Wed, Jan 26, 2011 at 3:45 PM, Stephen Johnson <[email protected]>wrote: > No, the 4 writes per second (rule of thumb) applies to an entity group. If > the items are all in separate entity groups then that doesn't apply. > > > On Wed, Jan 26, 2011 at 3:25 PM, Michael McClain < > [email protected]> wrote: > >> So I can only insert 4 items per second in the database? >> >> I need to insert 1000 items simultaneously for each new document I'm >> inserting in the system... >> Is there a way to do this? >> >> Thank you, >> Michael >> >> On Jan 26, 5:36 pm, Wim den Ouden <[email protected]> wrote: >> > Hi Michael, >> > You need to spread the load as much as possible (parallel taskqueues), >> > keep processor load without extreme spikes. >> > Datastore writes max to about 4 per second (master/slave), 1 per >> > second high replication datastore. >> > When writing the app (triall and error) divide every load as much as >> > possible in smaller chunks, don't load to much libraries etc. (load >> > time) >> > gr >> > wim >> > >> > 2011/1/26 Michael McClain <[email protected]>: >> > >> > >> > >> > > Hi everybody, >> > >> > > I am new to Google Apps Engine. >> > > I'm interested in using a cloud storage service for a large scale >> database >> > > system. >> > >> > > I have to store billions of keys. Where each key has a 12 bytes >> value. >> > > During query time I need to perfome one thousand simultaneous queries >> to >> > > retrieve 5000 thousand keys for each query. So I need to retrieve >> > > 5000*1000*12 bytes = 60 megabytes each time my application is run. >> > >> > > I would like to understand if this is feasible using Google >> Datastore. >> > > What would be the latency in performing one thousand simultaneous >> queries? >> > >> > > Is there a Google representative that could assist me in >> understanding if >> > > my app is feasible for deployment in Google Datastore? >> > > I'm interested in Google Datastore because it can scale usage growth >> and >> > > peak time traffic. >> > >> > > Thank you, >> > > Michael >> > >> > > -- >> > > You received this message because you are subscribed to the Google >> Groups >> > > "Google App Engine" group. >> > > To post to this group, send email to >> [email protected]. >> > > To unsubscribe from this group, send email to >> > > [email protected]<google-appengine%[email protected]> >> . >> > > For more options, visit this group at >> > >http://groups.google.com/group/google-appengine?hl=en. >> > >> > -- >> > gr >> > Wim den Ouden >> > Custom applications,https://e-comm.appspot.com/ >> > Free open source E-commerce framework (web) apps, >> http://code.google.com/p/relat/ >> > Gae developer tips,http://code.google.com/p/relat/wiki/gaetips >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To post to this group, send email to [email protected]. >> To unsubscribe from this group, send email to >> [email protected]<google-appengine%[email protected]> >> . >> For more options, visit this group at >> http://groups.google.com/group/google-appengine?hl=en. >> >> > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
