Quick update: the time has been pushed back 2 hours to 6PM US/Pacific. See the latest message here:
https://groups.google.com/forum/?fromgroups#!topic/google-appengine-downtime-notify/CO_x02OF9Ak -- Ikai Lan Developer Programs Engineer, Google App Engine plus.ikailan.com On Mon, Mar 12, 2012 at 1:59 PM, Ikai Lan (Google) <[email protected]>wrote: > GAH, it's like no matter how many times I read these things over I always > make at least one mistake. > > And that's why code review is a Good Thing. > > -- > Ikai Lan > Developer Programs Engineer, Google App Engine > plus.ikailan.com > > > > On Mon, Mar 12, 2012 at 1:54 PM, Riley Eynon-Lynch <[email protected]>wrote: > >> Thanks a lot. FYI: That post says Wednesday the 19th instead of Monday >> the 19th. >> >> Riley >> >> >> On Mon, Mar 12, 2012 at 3:52 PM, Ikai Lan (Google) <[email protected]>wrote: >> >>> Regarding that maintenance period: >>> >>> >>> https://groups.google.com/forum/?fromgroups#!topic/google-appengine-downtime-notify/CO_x02OF9Ak >>> >>> It's happening next Monday, March 19th at 4pm US/Pacific (19th March, >>> 23:00 GMT). >>> >>> -- >>> Ikai Lan >>> Developer Programs Engineer, Google App Engine >>> plus.ikailan.com >>> >>> >>> >>> On Mon, Mar 12, 2012 at 11:52 AM, Ikai Lan (Google) >>> <[email protected]>wrote: >>> >>>> Hi Riley, >>>> >>>> That's a legitimate question, and one that we haven't officially >>>> answered yet. It's certainly the direction that things have been moving >>>> simply due to the nature of production management. Given that the SLA >>>> applies to HRD and not master/slave applications, you are definitely going >>>> to get a better quality of service migrating to HRD. In fact, I strongly >>>> advise that you do so. >>>> >>>> One challenge that we have when dealing with issues is to decide >>>> whether we should do emergency maintenance that requires downtime. With any >>>> production system, it's not always guaranteed that maintenance will result >>>> in issues being completely resolved, which would be really bad for app >>>> developers. At what threshold do we determine that a downtime with no >>>> guarantee of addressing the issues is worthwhile? Global 0.1% error rate? >>>> 1%? The call is not always clear cut because those errors may not be evenly >>>> distributed, and the impact may be huge, or it may be small. With >>>> master/slave applications, we do what we can to address the short term >>>> symptoms as well as the underlying system issues without impacting serving, >>>> which is often an order of magnitude more difficult (It kind of reminds me >>>> of that scene in Indiana Jones where he takes an artifact, swapping it with >>>> a bag of sand as quickly as possible to try to avoid setting off traps. >>>> Pillaging of historic artifacts is way easier when it's not dangerous, not >>>> speaking from personal experience). When your application runs on High >>>> Replication, the call is easy: there's no downtime required in 99% of >>>> cases, so we perform the maintenance right away because if it doesn't >>>> address the issue, there's no serving downtime for users. >>>> >>>> If you're not subscribed to downtime-notify, I recommend that you do >>>> so. Announcements like this will NOT and never will be moving to >>>> StackOverflow: >>>> >>>> >>>> https://groups.google.com/forum/?fromgroups#!forum/google-appengine-downtime-notify >>>> >>>> We may be announcing a maintenance in the very near future that will >>>> impact the serving of master/slave applications. >>>> >>>> -- >>>> Ikai Lan >>>> Developer Programs Engineer, Google App Engine >>>> plus.ikailan.com >>>> >>>> >>>> >>>> On Mon, Mar 12, 2012 at 9:46 AM, Riley <[email protected]> wrote: >>>> >>>>> Ikai, it sounds like support for HR apps is being prioritized. Is >>>>> that the case? Should we expect that to be the case in the future? Sorry >>>>> if >>>>> that's documented somewhere already~ >>>>> >>>>> Riley >>>>> >>>>> >>>>> On Monday, March 12, 2012 11:44:59 AM UTC-5, Riley wrote: >>>>>> >>>>>> Our appid is activegrade, we use the m/s datastore, and get from 0-10 >>>>>> QPS throughout the day. Normally we have 1-4 instances running, but >>>>>> since >>>>>> this seems *mostly* related to startup, we dedicated 10 idle resident >>>>>> instances to run all the time. This covers us a little, but still, when >>>>>> a >>>>>> user triggers a new instance, they get the 60+ second wait and then an >>>>>> error. Ugh! Our costs are relatively minor - about $20 a day now that we >>>>>> are running these 10 ordinarily unnecessary instances - but this is a big >>>>>> cost for us, and embarrassing too. >>>>>> >>>>>> On Tuesday, March 6, 2012 3:17:37 PM UTC-6, Adam Sherman wrote: >>>>>>> >>>>>>> Am I the only one seeing short duration outages? They are being >>>>>>> reflected at: >>>>>>> >>>>>>> http://code.google.com/status/**appengine<http://code.google.com/status/appengine> >>>>>>> >>>>>>> But I don't see anyone else complaining anywhere, so it makes me >>>>>>> worried. >>>>>>> >>>>>>> A. >>>>>>> >>>>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Google App Engine" group. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msg/google-appengine/-/VzKRK5UG96MJ. >>>>> >>>>> To post to this group, send email to [email protected] >>>>> . >>>>> To unsubscribe from this group, send email to >>>>> [email protected]. >>>>> For more options, visit this group at >>>>> http://groups.google.com/group/google-appengine?hl=en. >>>>> >>>> >>>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Google App Engine" group. >>> To post to this group, send email to [email protected]. >>> To unsubscribe from this group, send email to >>> [email protected]. >>> For more options, visit this group at >>> http://groups.google.com/group/google-appengine?hl=en. >>> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To post to this group, send email to [email protected]. >> To unsubscribe from this group, send email to >> [email protected]. >> For more options, visit this group at >> http://groups.google.com/group/google-appengine?hl=en. >> > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
