Hi Stefano, This does indeed seem very odd, and thank you for clarifying the behaviour. It's possible some part of your code is catching the errors and not logging them, or your logging level is not set appropriately, but these are small possibilities.
In this case, I think the best thing for you to do would be to open a public issue tracker issue <http://code.google.com/p/google-appengine/issues/list> with your app id, a description of the problem, a time-frame to investigate, and hopefully the issue can be looked at by engineering to determine if there's some issue in the platform or your app. We monitor this forum, stackoverflow, and the public issue tracker, so your issue should see action within a few days of being posted. PS - your english is quite good, and you've done a very thorough job of reporting your issue. I wish you the best of luck with your application. Nick On Monday, August 10, 2015 at 11:15:45 AM UTC-4, Stefano Ciccarelli wrote: > > Hi Nick, > > the two screenshots attached to this post are perfect to explain what > happen everyday to my app: one intance was started 3 hours ago and served > only 22 requests, every hour (or so) 3 instances are killed and replaced by > fresh ones, all at the same time, so all the requests goes to the new > instances (high latencies). > > PS: I have no errors in my app. > > > Il giorno ven 7 ago 2015 alle ore 17:36 Nick (Cloud Platform Support) < > [email protected]> ha scritto: > >> Hi Stefano, >> >> If I understand your post correctly, you're using automatic scaling >> <https://cloud.google.com/appengine/docs/java/modules/#automatic_scaling_1> >> and have only configured the min_idle_instances parameter to be 1. You're >> wondering why you've seen a certain request distribution pattern or dynamic >> instance lifetime on your app. I think that the answer to these questions >> lies in understanding the App Engine Autoscaler. >> >> Putting your scaling settings into default means that you give up control >> over max_idle_instances (that is, there is no max), and so the Autoscaler >> can and will spawn idle instances to preempt traffic or respond to sudden >> load spikes. This is based on past request loads on your app and >> projections based on spikes. >> >> You will probably want to determine your current and projected request >> load and set a sensible maximum to this value, since at that point you >> could be assured that you won't be charged for more idle instances, if >> they are spawned, than your configuration specified >> <https://cloud.google.com/appengine/docs/java/modules/#max_idle_instances> >> . >> >> As I can see from looking at the data you supplied, an instance which was >> only active for 34 minutes handled 1108 requests. While it's certainly >> possible that this load was handled well by the instance, if the same load >> was observed on the other two machines at some point in the past, which the >> overall graph of requests seems to imply, especially given how spiked the >> load can tend to be within even the past 6 hours (the timeframe of the >> graph), it makes sense that the Autoscaler would keep some instances in >> reserve and route requests to them once the load went higher, but not spin >> them down in case a new spike were to arrive. >> >> The Autoscaler scales aggressively in order to ensure that you won't be >> left overloaded, and it's up to your scaling configuration to make sure >> that this matches your expected load. I suggest you look into the max and >> min_pending_latency configuration options as well as min and >> max_idle_instances. >> >> Now, as to your question about resident instances: a resident instance is >> an instance which is always ready for requests, and won't spin down due to >> inactivity. You specified one min idle instance, so you have one resident >> instance. Idle instances ("resident" instances) are meant to allow your app >> to handle traffic while all dynamic instances (if there are any) are >> overloaded, and new dynamic instances need to be created, so generally they >> will only receive traffic during the front edge of a load spike, acting as >> a buffer as opposed to the main force of request-handling instances. >> >> While resident instances won't spin down due to inactivity, they can shut >> down due to errors or memory limits being reached, in which case the >> instance will receive a request to /_ah/stop which allows it to attempt to >> save any delicate state it's holding. Instances can also (on a rare >> occasion) restart due to datacenter events and maintenance. >> >> However more common reasons for seeing a dynamic instance with a smaller >> age than other dynamic instances is either because a change has just been >> made which deploys a new resident instance (such as a configuration change >> to specify 1 resident instance), or because a resident instance which is >> serving traffic and becomes quite busy with that task might transition to >> being a regular serving dynamic instance as a small efficiency gain, and >> another resident instance will be created to take its place as a buffer for >> even higher load spikes. >> >> So, overall, my advice is to keep in mind that the Autoscaler will scale >> your instances to meet spiking loads (which it seems you have), and that >> your scaling configuration can help keep this within certain boundaries. >> >> I hope this has helped explain what you're seeing, and feel free to ask >> any further questions you might have. >> >> Regards, >> >> Nick >> >> >> On Friday, August 7, 2015 at 7:11:49 AM UTC-4, Stefano Ciccarelli wrote: >>> >>> Hello to all! >>> >>> Our application has always suffered from high latencies due to instances >>> respawning continuously. >>> We always used custom settings in "automatic-scaling" to try to save >>> money. >>> >>> Following this post ( >>> http://googlecloudplatform.blogspot.it/2015/08/How-to-Troubleshoot-Latency-in-Your-App-Engine-Application.html) >>> >>> I've decided to reset the "automatic-scheduling" settings of our app to >>> default values. >>> >>> So I've configured only min-idle-instances to 1 (otherwise the >>> /_ah/warmup handler is never called). The instance class is F2. >>> >>> After two days this is our state. >>> >>> So: >>> - why I have 4 instances 2 days old not used? (The logs says that one >>> instance served ~20 request today, but the other instances stopped to serve >>> yesterday) >>> - why the scheduler stop and spawn instances every 40/50 minutes if 4 >>> instances are there to do nothing? >>> - why the resident instance is restarted every 40/50 minutes if it is >>> supposed to be... resident? >>> - is there something wrong with our code? >>> >>> Thanks >>> >>> >>> >>> >>> [image: nimbus-image-1438944970211.png] >>> -- >>> *Stefano Ciccarelli* >>> GAE Application Division >>> / Director >>> [email protected] >>> >>> *M.M.B. s.r.l.* >>> via Granarolo, 177/7 - 48018 Faenza (RA) - Italy >>> tel. +39.0546.637711 - fax +39.0546.46077 >>> www.mmbsoftware.it - [email protected] >>> >>> Le informazioni contenute in questa comunicazione sono riservate e >>> destinate esclusivamente alla/e persona/e o all'ente sopra indicati. E' >>> vietato ai soggetti diversi dai destinatari qualsiasi uso, copia, >>> diffusione di quanto in esso contenuto sia ai sensi dell'art. 616 c.p., sia >>> ai sensi del DL n. 196/03. Se questa comunicazione Vi e' pervenuta per >>> errore, Vi preghiamo di rispondere a questa e-mail e successivamente >>> cancellarla dal Vostro sistema. >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To post to this group, send email to [email protected]. >> Visit this group at http://groups.google.com/group/google-appengine. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/google-appengine/b4e88571-6790-4fb7-b129-afb82d27821b%40googlegroups.com >> >> <https://groups.google.com/d/msgid/google-appengine/b4e88571-6790-4fb7-b129-afb82d27821b%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> For more options, visit https://groups.google.com/d/optout. >> > -- > *Stefano Ciccarelli* > GAE Application Division > / Director > [email protected] > > *M.M.B. s.r.l.* > via Granarolo, 177/7 - 48018 Faenza (RA) - Italy > tel. +39.0546.637711 - fax +39.0546.46077 > www.mmbsoftware.it - [email protected] > > Le informazioni contenute in questa comunicazione sono riservate e > destinate esclusivamente alla/e persona/e o all'ente sopra indicati. E' > vietato ai soggetti diversi dai destinatari qualsiasi uso, copia, > diffusione di quanto in esso contenuto sia ai sensi dell'art. 616 c.p., sia > ai sensi del DL n. 196/03. Se questa comunicazione Vi e' pervenuta per > errore, Vi preghiamo di rispondere a questa e-mail e successivamente > cancellarla dal Vostro sistema. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/67b64bf7-e536-40e6-9ad2-4b8ebf72775c%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
