It sounds like you would be pushing any architecture to the limits. A few thousand requests per second with shared state is a nontrivial problem, especially if you have a poor read/write ratio. If you have a long-polling scenario you may have chosen the wrong platform; this is something that node.js or twisted python or equivalent asynchronous architectures excel at. But then expect to do a lot of work setting up and maintaining a fleet of servers; it isn't a one-man job anymore.
What percentage of your (present) cost is instance hours vs datastore operations? > and finally I shouldn't be paying for something called experimental! Maybe you shouldn't be *using* something called experimental. You're utilizing GAE in a way that is not efficient. Possibly your application is one that cannot be run efficiently on GAE; it sounds like an async appserver would be more appropriate. But maybe it can be done. Perviously Google hid the real cost of your app from you; now you're getting charged based on a closer approximation of the "real cost" of service. If you want help fitting your application to GAE's architecture then you need to post a lot more details about what you are doing. This post was the first time in your countless messages that you included enough information for any of us to even get a glimpse. Seriously, we (the community) are willing to help, but you need to actually want help. Jeff On Tue, Dec 6, 2011 at 9:08 AM, Andrius A <[email protected]> wrote: > Did you Simon reduced the costs by 1.5-2x? I dont know what are you > building but if I had a blog site running in GAE I wouldn't be complaining. > > I think I am pushing GAE to the limits, my sites do a lot of ajax requests > with some processing on the server (1 request per user each second) and I > have thousands of users, you would say that I should use channels? Yes, I > thought so, but can't since there is no way to broadcast the same message > to all users and looping through each channel will delay sending message! > You would then say use backends to off load sending message through > backends? Ohh yes, but backends do not support channels! So backends are > really good, you can have them running on background and doing time > consuming work which actually saves your money! But wait, they are not > reliable, they restart few times per day and this is what google says in > documentation, so you need to build some mechanism on top to make sure when > backends restart your app do not collapse! > So far I have moved all processing to backends and made all data to be > passed to user requests via memcache and believe me this is where it gets > more complicated. You later realise its nearly impossible to have backends > and frontedns working in local environtment, so develpment process and > debugging is really really painful. Also backends in production restart few > times per day so reliability is very low, so on top you build few master > backends who look after each other and launch other backends when needed > through task queue, and let this make more complicated, we know that task > queue can execute in some occasions the same task few times (which is more > frequent to happen within backends) and then you rely on memcache, and lets > make it a bit more complicated - you know that memcache can be flushed any > time, so then you have datastore which can also time out and then to be it > the most complicated to handle - request can occasionaly fail to load. Do > you see the point? You have a so many API's, we are being charged for them > but they are too unreliable, thats totoally not right. I understand that > nothing is reliable, anything could happen, but at least backends should > not restart few times per day, datastore shoudn't time out every hour, > requests shouldn't fail every day! and finally I shouldn't be paying for > something called experimental! > > > On 6 December 2011 11:55, Simon Knott <[email protected]> wrote: > >> Andrius, >> >> I'm curious as to what changes you've made to your application to fit in >> with the new pricing model? >> >> Whilst a lot of people said they were facing 10x price increase (some >> were 100x) when the model was first announced, a lot of those people have >> since come back and stated how they've managed to reduce that to 1.5-2x by >> re-writing their code. >> >> Cheers, >> Simon >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To view this discussion on the web visit >> https://groups.google.com/d/msg/google-appengine/-/A7uwTwRi69wJ. >> >> To post to this group, send email to [email protected]. >> To unsubscribe from this group, send email to >> [email protected]. >> For more options, visit this group at >> http://groups.google.com/group/google-appengine?hl=en. >> > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- We are the 20% -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
