Re: [google-appengine] Re: Scalability problem in GAE

2010-01-20 Thread Eric Ka Ka Ng
I have also installed this lib and am having fun with it. personally i 'feel' the overhead is really insignificant refer to your diagram, i'm not sure whether the blue bars are actually showing time without the profiler, i think it may mean something else (i also wanna know what the blue bar and

[google-appengine] Re: Scalability problem in GAE

2010-01-20 Thread mani doraisamy
Sorry. I got the degradation wrong. Datastore degrades from 40ms to 1283ms. thanks, mani On Jan 20, 3:48 pm, mani doraisamy mdorais...@orangescape.com wrote: Here are the stats for the same request at 05:37 Vs 10:49 on

[google-appengine] Re: Scalability problem in GAE

2010-01-20 Thread mani doraisamy
Same request, but another query has degraded from 45ms to 846ms: http://247-test.appspot.com/stats/details?time=1263968821144 http://247-test.appspot.com/stats/details?time=1263908256446 thanks, mani On Jan 20, 4:00 pm, mani doraisamy mdorais...@orangescape.com wrote: Sorry. I got the

Re: [google-appengine] Re: Scalability problem in GAE

2010-01-20 Thread Nickolas Daskalou
Hi mani, I have a question. How did you get appstats to not require an admin login in production? Nick 2010/1/20 mani doraisamy mdorais...@orangescape.com Same request, but another query has degraded from 45ms to 846ms: http://247-test.appspot.com/stats/details?time=1263968821144

[google-appengine] Re: Scalability problem in GAE

2010-01-20 Thread mani doraisamy
Is that a technical question or process question? On Jan 20, 4:35 pm, Nickolas Daskalou n...@daskalou.com wrote: Hi mani, I have a question. How did you get appstats to not require an admin login in production? Nick 2010/1/20 mani doraisamy mdorais...@orangescape.com Same request,

Re: [google-appengine] Re: Scalability problem in GAE

2010-01-20 Thread Nickolas Daskalou
Technical. I tried editing the appstats code to turn off this admin user requirement, but I kept getting a 500 error. 2010/1/21 mani doraisamy mdorais...@orangescape.com Is that a technical question or process question? On Jan 20, 4:35 pm, Nickolas Daskalou n...@daskalou.com wrote: Hi

Re: [google-appengine] Re: Scalability problem in GAE

2010-01-20 Thread Robert Kluin
I think I just edited the main function in appstats/ui.py. I can not recall how the original looked, but mine is now: def main(): Main program. Auth check, then create and run the WSGIApplication. if not os.getenv('SERVER_SOFTWARE', '').startswith('Dev'): if users.get_current_user() is

[google-appengine] Re: Scalability problem in GAE

2010-01-20 Thread mani doraisamy
It uses webapp (with routes) instead of Django and with custom authentication. That's the reason, i guess. thanks, mani On Jan 20, 8:46 pm, Nickolas Daskalou n...@daskalou.com wrote: Technical. I tried editing the appstats code to turn off this admin user requirement, but I kept getting a 500

Re: [google-appengine] Re: Scalability problem in GAE

2010-01-20 Thread Danny Tuppeny
2010/1/20 Eric Ka Ka Ng ngk...@gmail.com i'm not sure whether the blue bars are actually showing time without the profiler, i think it may mean something else (i also wanna know what the blue bar and red rectangle refer to, and will dig into the doc to find out what it actually means. anyone

Re: [google-appengine] Re: Scalability problem in GAE

2010-01-20 Thread Danny Tuppeny
2010/1/20 Danny Tuppeny da...@tuppeny.com 2010/1/20 Eric Ka Ka Ng ngk...@gmail.com i'm not sure whether the blue bars are actually showing time without the profiler, i think it may mean something else (i also wanna know what the blue bar and red rectangle refer to, and will dig into the

Re: [google-appengine] Re: Scalability problem in GAE

2010-01-20 Thread Danny Tuppeny
2010/1/20 mani doraisamy mdorais...@orangescape.com Same request, but another query has degraded from 45ms to 846ms: http://247-test.appspot.com/stats/details?time=1263968821144 http://247-test.appspot.com/stats/details?time=1263908256446 Which calls are you looking at? They look like

Re: [google-appengine] Re: Scalability problem in GAE

2010-01-20 Thread Robert Kluin
The blue line is wall-clock time. The red line is CPU or API time. As Danny mentioned, the blue line indicates blocking time of each call. The red line indicates what you are charged for each. Robert On Wed, Jan 20, 2010 at 12:43 PM, Danny Tuppeny da...@tuppeny.com wrote: 2010/1/20

[google-appengine] Re: Scalability problem in GAE

2010-01-20 Thread mani doraisamy
Oops! The url seems to redirect to newer requests now. It no longer refers to the same request that i had referred to. Enhancement request for Appstats: Generating url with surrogate key for each request (instead of time=1264004278534) would be of great help for bookmarking/reference. Anyway, i

Re: [google-appengine] Re: Scalability problem in GAE

2010-01-20 Thread Robert Kluin
Mani, Just a note. Guido's AppStats uses memcache to store the stats info so it will sometimes disappear. I think it also only stores the most recent 100 (or something like that) requests. You might want to take screen-shots or something to be sure people will see what you want to show them.

Re: [google-appengine] Re: Scalability problem in GAE

2010-01-20 Thread Eric Ka Ka Ng
yes exactly. I think this is the correct interpretation of the blue line and red line. - eric 2010/1/21 Robert Kluin robert.kl...@gmail.com: The blue line is wall-clock time.  The red line is CPU or API time. As Danny mentioned, the blue line indicates blocking time of each call.  The red

[google-appengine] Re: Scalability problem in GAE

2010-01-19 Thread mani doraisamy
Thanks for the link, Danny. This is the best profiling tool i have used. Well done, Guido! Will get back with the performance data for off-peak Vs peak hours. thanks, mani On Jan 17, 9:37 pm, Danny Tuppeny da...@tuppeny.com wrote: Have you identified what part of your page is taking additional

[google-appengine] Re: Scalability problem in GAE

2010-01-19 Thread mani doraisamy
I guess the reason why this is so damn good is because it gives an integrated picture across datastore, memcache and web request, without much overhead! Given that the variations are fixed (GAE's limitation of not allowing all libraries/api/rpc calls), this gives the full picture of the

Re: [google-appengine] Re: Scalability problem in GAE

2010-01-19 Thread Eric Ka Ka Ng
wow!! this is THE TOOL that I'm looking for long time!! previously i have to write codes to do similar profiling and analysis with much limited scales and functions. Now, THIS provides almost everything I want already without any efforts from developers! I could say it is one of the best tool on

Re: [google-appengine] Re: Scalability problem in GAE

2010-01-19 Thread Danny Tuppeny
I haven't used this in production yet (I only read that you can), though I'd avoid leaving it on for long periods if you don't need it. When running in production, it will show two bars - the one showing how much time was taken without the profiler, and one showing how long was actually taking