Ok, (a bit bummed that nobody bother to reply to my first post to the group) I spent the last few days reading related posts, transcripts of chats with Marzia, went through the related tickets and still I have no answer.
Let me try once more to explain my question: It is not about quotas. It is neither about performance per se. My question is about cost. My app implements a beacon service - my client websites that will be using the beacon have 5-10M pageviews per month and these pageviews will result into actual beacon hits to my app. These requests do not hit the Datastore - I buffer them in memcached doing minimal processing to keep the per beacon-hit cost low and "process them" in batches every few seconds. Still, in spite of all the buffering, if the cheapest write (1 attr table, no indexes) gets charged 250-500msec, this make the app design for such a high throughput service non-obvious. I understand that I can decrease the number of my writes by using a pickled blob attr that contains lots of records inside - that what I am about to do. I just wanted a confirmation from any gurus out there, or from the google team that my understanding about the cost of the "cheapest write" is correct. I tried batch db.put( of 100 single cell objects) - still took 23 seconds of data store CPU , i.e. my cheapest write = 230msec - a bit less than before but not by much. I made all 100 objects children of same parent and performed the same test - again same datastore CPU utilization Is there another method to update/insert multiple records that costs less? I.e. is there any cheaper write? ( I am very flexible :-) ) Is my understanding that the planned 10-12cents per CPU hr will be applied towards the datastore CPU usage as well? Thanks a lot, diomedes On Feb 14, 3:18 pm, diomedes <[email protected]> wrote: > Hi all, > > I have a question with regards to the pricing and how it relates to > the relatively high API CPU associated with each write (400-500msec) > > I have started working on my first GAE app about a month ago - and > overall I am both excited and very satisfied. My initial plan, was to > run it on Amazon's EC2 - but eventually I took the plunge, started > learning python, move to GAE and (almost) never looked back :-) > > My app, a cacti-like webservice that monitors the performance ( think > response time) of a website using google analytics-like beacons, is > rather resource demanding. On top of that GAE best practices imply > that any expensive reports/aggregates etc should be precalculated/ > stored instead of dynamically produced on demand. All that result in > many writes and given that the simplest write (single key-val pair, no > indexes) gets "charged" approx 500msec of API cpu time (see related > thread by > Matijahttp://groups.google.com/group/google-appengine/browse_thread/thread/...) > a normal DB design that would have been meaningful in terms of cost on > EC2 becomes impossible on GAE. > > Because I am a google-aholic I decided to change the app design to > minimize writes - I fetch a bunch of pickled data as a blob, update > in mem and write them back as blob (just like people did before DBs > came along :-) ) > Before I commit to that design I wanted to get the confirmation that > my understanding is correct: > - Google is going to charge 10-12cents per CPU-hour and it will > include in that all the CPU used from APIs etc. (http:// > googleappengine.blogspot.com/2008/05/announcing-open-signups- > expected.html) > - This means that if your site does 10M pageviews a month and does > a couple writes per pageview at 500msec per write it will be "10M CPU > secs/mo just from the writes, i.e. 10M/3600 * $.10/hr = $280/mo just > from the writes. > > Is this correct? > > For the record, I find Google's planned pricing extremely attractive > when compared to Amazon's primarily due to the fact that Amazon > charges 10c for CPU-hr of the machine while google (will) charge 10c > for CPU-hr *actually used* by your requests. This makes a huge > difference -- a server running at 50+% capacity (thats rather > aggressive - but with Amazon/RightScale combination you can be > aggressive) will still use less than 20% of its CPU during that time. > However, when comparing the cost writes between Google and the > corresponding setup of a [high CPU EC2 server + elastic storage] combo > (able to provide quite more than 20-50 "simple" writes per sec) > Amazon is much cheaper than Google. > > Ok, that's all I had to say, > Sorry for the rather long post, > Looking forward to hear comments > > Ah and thank you very very much for lifting the high cpu quota!! > > Diomedes --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~----------~----~----~----~------~----~------~--~---
