On Sat, Oct 30, 2010 at 2:35 AM, Tim Hoffman <[email protected]> wrote:
> Hi Bart > > Re Tim ;] > When you have successful startup what sort of average startup time are > you seeing. (I would accumulate a fair number of stats on this > so you can see what a successful varience is.) My startup time for (a > non django) but large stack is typically under 6 secs on a good day > 8-12 secs when things are a bit slow and DEE when it's bad ;-) In the > last few days I am seeing a DEE about 5 times a day on a startup > from a very quiet site (its a actually a test site so no one is using > it). > > At http://jsonspot.appspot.com (running fine it seems) the startup time of a cold boot is: 10-29 05:39PM 02.915 /dispatch/ 200 2387ms 958cpu_ms 25api_cpu_ms rest of the dispatches are at 100-150 msec. I would also have a closer look and see if a request just after a DEE > is also failing but with shorter time. Big frameworks can have > problems > if they don't start up cleanly. > > I think the cluster on which jsonbot runs is bork or something as jsonspot seems to do fine. > Ultimately appengine will fail at some point (and come back). Like the > datastore - you will get a timeout at some point, memcache - stuff > will be > purged) it means you do have to code for or allow for the fact that > things may go wrong for a period. Hopefully not often. The big > upside though > is if you take into account these sorts of things your app will be > more robust. It still beats managing hardware ;-) > > I wonder how much requests i could run on 1 hosted box. Waaaaay easier to program thats for sure. The more i get into the appengine programming (guess about a year now or so) the more i wonder if the limitations of appengine are worth the problems. Actually i'm not into running 1 big instance of my program for millions of users, my plan is to let every user run his/hers own bot. One of the advantadges of open sourcing your code ;] > Profiling your startup will also help you shrink startup time. Maybe > there are things you can defer till you need them. This won't help > if google is having a really bad day. But it will overall improve your > responsiveness when you do have to start new instances. > > I started porting my then console gozerbot to the GAE and had to drastiscally change away from the monolitic load every thing at once, to seperate handlers and on demand loading of my plugins etc etc. I have appstat running so i'll certainly will take a look into profiling startup. One note to make though is that i'm trying not to be a whiner, i love appengine and have learned a lot on this platform the last year. I'm gratefull to google to giving me this opportunity but i must say that i have serious doubts if the CGI method of execution isn't too much of a strain on the programming side compared to the gains i would get from hosting my site with google. I really hate this 30 sec boundary ;[ > Rgds > > Tim > > Thnx for giving your energy and advise Tim, it's much appreciated ;] Bart > On Oct 30, 7:50 am, Bart Thate <[email protected]> wrote: > > On Sat, Oct 30, 2010 at 1:35 AM, Tim Hoffman <[email protected]> wrote: > > > Hi > > > > Hi Tim, > > > > you are right looking at the logs is the first thing i should do .. > > > > 1. 10-29 04:36PM 12.953 > > > > botbase - on demand reloading of commonplugs.forward > > > > 2. D10-29 04:36PM 12.953 > > > > plugins - trying commonplugs.forward > > > > 3. E10-29 04:36PM 50.560 > > > > <class 'google.appengine.runtime.DeadlineExceededError'>: > > Traceback (most recent call last): > > > > Its all deadline errors, notice the time difference between the last > debug > > line and the exception .. 38 seconds ?? > > > > I dont use django but my own framework is getting huge and loading that > > might be a problem. Real problem is though that i cant know for sure > whether > > its my code at fault or its appengine that has troubles. Its getting hard > to > > develop on a unstable platform ;[ > > > > I suggest you do some serious logging in your app so you can see where > > > > > > > > > > > > > > > > > the time is taken up. If its during initial imports > > > then you may be in a pocket of poor performing appengine > > > infrastructure at the moment. (A few people seem to be having > > > onoing problems since the 27th) In addition it will give you a > > > better insight into how you app is behaving during startup. > > > > > It's also possible that you got a DeadLineExceeded during startup that > > > has left you with a broken instance (this can happen if you using > > > things > > > like django). > > > > > Have a look at the system log and see if all the 500's are due to > > > DeadLineExceeded or just the first one followed by > > > ImportError's or even AttributeError in seemingly odd places. > > > > its all DeadLineExceeded errors. > > > > > Mind you all of this only applies if your using python. (I have no > > > idea haw the java platforms behave) > > > > I'm on python. If you're curious about the code its onhttp:// > jsonbot.googlecode.com > > > > > Rgds > > > > > Tim > > > > Thanks for the help Tim ! > > > > Bart > > > > > > > > > > > > > > > > > > > > > On Oct 30, 7:28 am, Bart Thate <[email protected]> wrote: > > > > So i was programming the whole day and night to give my website a new > > > look, > > > > finally got something working and tweeted the website url - > > >http://jsonbot.appspot.com. I was so amazed the bot loaded that fast i > > > > though man i must be king of optimisation, seeing my website work as > it > > > > should (latency ~ 150msec). > > > > > > But then after a few visitors, it just went dead throwing 500's all > the > > > > time. > > > > > > *Instances*QPSLatencyMemory0.01732648.0 ms8.2 MBytes0.01732951.0 > ms8.0 > > > > MBytes0.01732914.0 ms7.9 MBytes0.01733.0 ms10.3 MBytes > > > > > > ENOCLUE whats going on here, but it sure is disappointing. Seems to > me > > > that > > > > coldstart is a problem on GAE right now and scaling doesnt work at > all > > > cause > > > > that always needs to cold boot new instances up to be able to scale ? > > > > > > Error: Server ErrorThe server encountered an error and could not > complete > > > > your request. > > > > > > If the problem persists, please > > > > report<http://code.google.com/appengine/community.html> your > > > > problem and mention this error message and the query that caused it. > > > > I hope google can give also us "freerunners" a decent level of > service so > > > > that we can grow as well. > > > > The app is still stuck, i wont upload a new one till you have been > able > > > to > > > > take a look at it. > > > > > > appid: jsonbot > > > > > > Regards, > > > > > > Bart > > > > > > -- > > > > @jsonbot Heerhugowaard, Netherlands > > > > programming schizofrenic -http://jsonbot.appspot.com > > > > > -- > > > You received this message because you are subscribed to the Google > Groups > > > "Google App Engine" group. > > > To post to this group, send email to [email protected] > . > > > To unsubscribe from this group, send email to > > > [email protected]<google-appengine%[email protected]><google-appengine%2Bunsubscrib > [email protected]> > > > . > > > For more options, visit this group at > > >http://groups.google.com/group/google-appengine?hl=en. > > > > -- > > @jsonbot Heerhugowaard, Netherlands > > programming schizofrenic -http://jsonbot.appspot.com > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]<google-appengine%[email protected]> > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- @jsonbot Heerhugowaard, Netherlands programming schizofrenic - http://jsonbot.appspot.com -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
