if data is independent of each other , yes you can easily reach that speed of writing.
and unless it is ok to lost a one or two data point ( in case of memcache eviction ) i recommend directly writing to datastore. On Wed, Oct 24, 2012 at 5:46 AM, Bradley Ford <[email protected]> wrote: > Hi All, > > I hope I can get some advice on a similar topic to that posted above. > > I need to record time series data at high-frequency and timestamps are > very important. > > During a peak period for each data point it might be recording at 1 second > update rates per data point. But during the off-peak periods I expect 10-30 > second update rates are to be expected. > My thinking is that each data-point could be its own tablet; this will > result in between 100,000 to 1,000,000 tablets. > > The other point too note is that it is ok if I have a delay before data is > wrote to the database; so I could dump to the database after I have > collected every 10 or 20 data points. (Possibly using Memcache) > > > From my reading generally and on this thread it should be possible if I > can split each data point onto its own tablet; I am still unsure how I can > make the cost manageable In particular if I can use Memcache as a method > to reduce the number of writes. > > Best Regards > Brad > > On Friday, 27 January 2012 06:57:04 UTC+9, Andrei-Ştefăniţă Cocorean wrote: >> >> Hi, >> >> I want to develop a web service which will be used by a few thousand >> clients to save new pieces of data at a rate of aprox. 1 post/minute. I've >> been looking at what App Engine has to offer but I'm not sure if it's the >> right platform for this kind of system. At a glance it seems the datastore >> is optimized for efficient reads, but not so much for write intensive >> applications. >> >> I'd appreciate any help on deciding whether it's worth to try and build >> this service on App Engine. Also, if you implemented something similar I'd >> be interested in hearing what problems you encountered along the way. >> >> Regards, >> Andrei >> >> >> -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To view this discussion on the web visit > https://groups.google.com/d/msg/google-appengine/-/36DvbBBlMbAJ. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
