Hi Gee,
Can you please reply to me with your app ID and I can look in to it.
-Marzia
On Thu, Nov 13, 2008 at 10:50 AM, gee <[EMAIL PROTECTED]> wrote:
>
>
> Hi Brett,
>
> During this time, it would have been very extremely helpful if CPU/
> response time quotas were relaxed.
>
> All our cache misses are adding up, and we're now over quota.
>
> We've had significant downtime over the last two days due to the
> scheduled downtimes going awry (our app was one of those taken down
> unexpectedly Tuesday night), and it's definitely hurting our users'
> experience with our growing service. Any help would be much
> appreciated.
>
> Thanks
> Gee
>
>
>
>
>
>
> On Nov 13, 10:30 am, "Brett (Google)" <[EMAIL PROTECTED]>
> wrote:
> > Memcache calls should now be failing fast. That should significantly
> > reduce the number of request deadlines people are seeing. Otherwise,
> > we're working to bring the memcache API back online for all
> > applications as soon as possible. Thanks again for your patience,
> > everyone!
> >
> > On Nov 13, 10:12 am, "Brett (Google)" <[EMAIL PROTECTED]>
> > wrote:
> >
> > > Hi Gayle,
> >
> > > Yes, the *request* times out during the API call. It's not the API
> > > call itself that's timing out. But there is no effective difference to
> > > your application if you're caught in this situation (as you pointed
> > > out). I'd assume the first few calls you make to memcache during a
> > > request actually return False and None. But later on in your request
> > > handler, so much time has already passed that your API call gets
> > > interrupted by the DeadlineExceededException for the entire request.
> >
> > > I think for now your best bet is to catch the runtime
> > > DeadlineExceededException and immediately return with some kind of
> > > error code. We're looking into the cause of the delayed memcache calls
> > > on failure. They should be immediately failing instead of waiting up
> > > to a second to return. That's the root cause of the issue here.
> >
> > > I'm sorry for the trouble this is causing for your application. We're
> > > working right now to address this issue. Someone from the team will
> > > follow up when everything has been resolved.
> >
> > > -Brett
> >
> > > On Nov 13, 9:57 am, Gayle Laakmann <[EMAIL PROTECTED]> wrote:
> >
> > > > It was timing out within memcache.get and memcache.set -
> > > > consistently. If I understand you correctly - you're saying that
> > > > memcache isn't technically timing out, on the grounds that it would
> > > > complete if given enough time. But, the cache misses take SO much
> > > > time that it consistently timeouts during the operations of
> > > > memcache.get and memcache.set.
> >
> > > > Ok, sure, that may be the case. I'm afraid I don't really see the
> > > > difference between that and timing out. Either way, memcache.get and
> > > > memcache.set aren't exactly just "return false and letting your
> > > > application proceed." The fact is they are causing timeouts.
> >
> > > > obj = memcache.get(cache_key)
> > > > File
> "/base/data/home/apps/careercup/13.329261908074364999/memcache/
> > > > __init__.py", line 362, in get
> > > > self._make_sync_call('memcache', 'Get', request, response)
> >
> > > > File "/base/python_lib/versions/1/google/appengine/api/
> > > > apiproxy_stub_map.py", line 46, in MakeSyncCall
> > > > stub.MakeSyncCall(service, call, request, response)
> > > > File "/base/python_lib/versions/1/google/appengine/runtime/
> > > > apiproxy.py", line 245, in MakeSyncCall
> >
> > > > rpc.Wait()
> > > > File "/base/python_lib/versions/1/google/appengine/runtime/
> > > > apiproxy.py", line 161, in Wait
> > > > rpc_completed = _apphosting_runtime___python__apiproxy.Wait(self)
> >
> > > > On Nov 13, 9:45 am, Brett <[EMAIL PROTECTED]> wrote:
> >
> > > > > Hi Gayle,
> >
> > > > > I'm sorry to hear you're seeing issues. It's important to note that
> > > > > there are two types of DeadlineExceededErrors. One is defined in
> > > > > apiproxy_errors.py. The other is in runtime/__init__.py:
> >
> > > > >
> http://code.google.com/p/googleappengine/source/browse/trunk/google/a...
> >
> > > > > The names are the same, which makes this confusing. The latter
> error
> > > > > is raised when the request has been processing for too long (~10
> > > > > seconds). Perhaps what's happening here is that the cache misses
> are
> > > > > causing a lot of extra work to be done, causing your application to
> > > > > hit this other deadline.
> >
> > > > > The traceback sent by Pratham confirms this behavior. More detail
> on
> > > > > how to deal with this is here, in the section entitled "The Request
> > > > > Timer":
> >
> > > > >http://code.google.com/appengine/docs/python/requestsandcgi.html
> >
> > > > > -Brett
> >
> > > > > On Nov 13, 9:29 am, Gayle Laakmann <[EMAIL PROTECTED]> wrote:
> >
> > > > > > Despite what the post at the link below says, memcache is
> actually
> > > > > > timing out and throwing a DeadlineExceededError. Furthermore, it
> > > > > > hardly seems accurate to say that our apps should continue
> serving
> > > > > > normally. The quota limits are so ridiculously low that we CAN'T
> > > > > > serve pages without caching.
> >
> > > > > >
> http://groups.google.com/group/google-appengine-downtime-notify/brows...
> > > > > > We will be taking memcache offline tomorrow morning from 9-10am
> PST
> > > > > > (GMT-8) for routine maintenance. Calls to the memcache API will
> *not*
> > > > > > throw exceptions but will instead return false for set() calls
> and
> > > > > > None for get() calls (just like any other cache miss.)
> >
> > > > > > Your app should continue serving normally during this period, and
> > > > > > we'll keep you updated on our progress. "
> >
> > > > > > Additonally, I'd like to make two suggestions:
> > > > > > 1. If you're going to take down memcache, wouldn't it make sense
> to
> > > > > > remove the quota limits? It hardly seems fair to penalized for
> > > > > > exceeding quota when caching is disabled.
> >
> > > > > > 2. If you have to take down our apps for an hour, can you pick
> > > > > > something like 2am - 3am? I know you don't want to go to work at
> 2am,
> > > > > > but it's not really ok to take down our apps for an hour during
> prime-
> > > > > > time.
> >
>
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~----------~----~----~----~------~----~------~--~---
