My app (appid: mr-october) is experiencing lots of problems with latency spike. I am paying for always on service. I am seeing frequent but seemingly random spikes of latency on user requests. This is causing my users lots of unexpected problems and giving the app a reputation as unreliable. Currently I get 20-50 users to the site everyday spread out over the day.
Looking at my logs I ca find an example of this. When users land on my site I present them with a list of available options. this is done by query the data store for objects that have an end date in the future and then doing a little additional filtering on the list. The current list is about 8 returned from the data store and then filtered on the cpu and sanatized to be sent to the user. Over the last 3 hours I find the following latencys for this call, (which i can identify by the first log entry of :com.shinypurplepony.reggie.server.servlets.RegistrationServlet execute: Action: com.shinypurplepony.reggie.shared.action.GetRegistrationSeasons) 2011-02-25 12:36:09.581 /reggie/dispatch 200 4829ms 4763cpu_ms 236api_cpu_ms 6kb Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/534.16 (KHTML, like Gecko) Chrome/10.0.648.82 Safari/ 534.16,gzip(gfe 2011-02-25 12:35:31.683 /reggie/dispatch 200 159ms 405cpu_ms 125api_cpu_ms 6kb Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; GTB0.0; SLCC2; .NET CLR 2.0.50727; 2011-02-25 12:35:16.764 /reggie/dispatch 200 354ms 615cpu_ms 125api_cpu_ms 6kb Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; GTB0.0; SLCC2; .NET CLR 2.0.50727; 2011-02-25 12:20:37.208 /reggie/dispatch 200 232ms 452cpu_ms 125api_cpu_ms 6kb Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en- US; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3,gzip(gfe) 2011-02-25 12:19:28.348 /reggie/dispatch 200 6380ms 4600cpu_ms 190api_cpu_ms 6kb Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en- US; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3,gzip(gfe) 2011-02-25 10:47:19.700 /reggie/dispatch 200 171ms 452cpu_ms 125api_cpu_ms 6kb Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv: 1.9.2.13) Gecko/20101203 Firefox/3.6.13,gzip(gfe) 2011-02-25 10:47:15.865 /reggie/dispatch 200 141ms 405cpu_ms 125api_cpu_ms 6kb Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv: 1.9.2.13) Gecko/20101203 Firefox/3.6.13,gzip(gfe) 2011-02-25 10:47:15.731 /reggie/dispatch 200 385ms 475cpu_ms 125api_cpu_ms 6kb Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv: 1.9.2.13) Gecko/20101203 Firefox/3.6.13,gzip(gfe) 2011-02-25 10:46:52.893 /reggie/dispatch 200 37ms 140cpu_ms 0kb Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.13) Gecko/ 20101203 Firefox/3.6.13,gzip(gfe) 2011-02-25 10:37:15.031 /reggie/dispatch 200 177ms 522cpu_ms 125api_cpu_ms 6kb Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en- US; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3,gzip(gfe) 2011-02-25 10:36:48.890 /reggie/dispatch 200 3550ms 732cpu_ms 125api_cpu_ms 6kb Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en- US; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3,gzip(gfe) Which is honestly much better then the past couple of days: 2011-02-25 08:55:09.606 /reggie/dispatch 500 30302ms 86cpu_ms 40api_cpu_ms 0kb Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 2.0.50727; .NET CLR 1.1.4322; InfoPath.2; MS-2011-02-22 22:23:31.143 /reggie/dispatch 200 5043ms 210cpu_ms 0kb Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/534.13 (KHTML, like Gecko) Chrome/9.0.597.98 Safari/534.13,gzip(gfe) Or these where appengine did not even get to my log statement so I have no idea what the request was: 2011-02-23 09:36:35.922 /reggie/dispatch 500 141862ms 1395cpu_ms 65api_cpu_ms 0kb Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_6; en- us) AppleWebKit/533.19.4 (KHTML, like Gecko) Version/5.0.3 Safari/ 533.19.4,gzip(gfe) 2011-02-23 09:00:21.024 /reggie/dispatch 500 102858ms 793cpu_ms 0kb Mozilla/5.0 (iPad; U; CPU OS 4_2_1 like Mac OS X; en-us) AppleWebKit/ 533.17.9 (KHTML, like Gecko) Version/5.0.2 Mobile/8C148 Safari/ 6533.18.5,gzip(gfe) What can be do to address these latency issues? Thanks, Cian. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
