Richard,
I would also love to see the ability to query a list of
tasks-to-be-executed. You can star issue 1790.
http://code.google.com/p/googleappengine/issues/detail?id=1790
Robert
On Thu, Mar 3, 2011 at 19:26, Richard Arrano <[email protected]> wrote:
> Hi Steve,
> I would certainly agree about some sort of ability to guarantee low-
> volume high-importance storage. I had an idea that it might be able to
> be implemented in the form of something along the lines of 1MB
> unevictable memcache storage; we wouldn't be able to store much there,
> but we would absolutely guarantee that whatever we store there will be
> able to be retrieved until we flush it or delete the keys. I also like
> your idea of some sort of task queue with certain guarantees attached
> to it at the cost of it being called rarely and with smaller amounts
> of payload. Alternatively I wouldn't mind seeing some sort of pay
> option where you could pay for guarantees on latency and write
> availability. This might be covered in the Business SLA, though my
> cursory impression was that it doesn't actually guarantee things it
> simply reimburses you if they have significant outages.
>
> Something that occurred to me that might help you was I was actually
> considering de-isolating the datastore as a single point of failure by
> additionally sending out those really high importance writes via XMPP
> to some other server. The other server could be EC2, it could even be
> a private home server(it's not going to get hammered since we're
> talking very low volume). It would then back up those records even if
> the datastore is having latency/write issues and they could be
> processed in the near future. So in this sense, you could even query
> the private server via XMPP and get a response regarding the payment
> record without GAE having put the record to the datastore yet. I'm not
> sure about this solution, it just occurred to me and I thought it
> might have some benefits. Your solution sounds good too, though I
> suppose you might even be able to combine them. Thanks for the tips!
>
> Robert: regarding using memcache, I do plan to use the memcache in
> this sense to see what's been sold and what hasn't been, but I assume
> because of the volatility of the memcache there's no way to rely on it
> absolutely to check this. Which of course goes back to my prior point
> that I'd love to see some sort of very constrained, low volume segment
> of memcache be non-volatile for purposes like this. Alternatively,
> what about the ability to at least see some record of even just the
> names of tasks yet to be executed in the task queue? This way I could
> construct a record of what's been sold regardless of datastore
> performance and whether or not the memcache has evicted it.
>
> -Richard
>
> On Mar 3, 10:53 am, stevep <[email protected]> wrote:
>> Hi Richard,
>>
>> Thought I'd comment since I started this thread. Think Wim and Robert
>> have helped already more than I could. So just some thoughts from me.
>>
>> First: I hope any GAE engineers reading this thread might have a
>> thought sparked about task queue featured needed for highly important,
>> lower-volume tasks such as yours.
>>
>> My problem is that I've got to record a one-time purchase payment.
>> Yours is more complicated. My issue was the all the steps needed to
>> setup and maintain a payment record for a Paypal for an Express
>> Checkout purchase (digital good).
>>
>> Rather than risk any loss of the update in the handler functions
>> (which already has to deal with sending urlfetch calls to Paypal), I
>> send the responses from Paypal to a high-priority task queue for the
>> put()s.
>>
>> This high-priority queue is pretty strict about very lightweight
>> imports, and record updates (i.e. no complex indexes on records being
>> updated). Hopefully this keeps issues related to slower task handlers
>> low. Look to Wim's link in this thread for many helpful points about
>> "handler weight control".
>>
>> Since I can't move forward with Express Checkout until the payment
>> record has been put() by the high-priority queue, the client starts to
>> send a request to the on-line handler once every second (with a nice
>> dialog explaining the delay for the customer). If the new payment
>> record has not been put() after 10 client calls, then I go into damage
>> control on the client side. Hopefully this happens very infrequently.
>>
>> The alternative for me is to have the on-line handler send the
>> urlfetch call to Paypal, put() the new payment record upon PP's
>> response, and then respond back to the client with "result=OK". If all
>> this happens, then the client knows to proceed because the initial
>> payment put() has been done. I just felt that there was a greater risk
>> of failure having all this functionality in the one on-line handler
>> function. Perhaps with HR, this is unfounded, but I'm not sure there's
>> enough anecdotal evidence to support this yet.
>>
>> This of course encumbers the user with an additional dialog and a bit
>> more wait time, but I don't think it is too much as many purchase
>> processes on-line take a bit of time. Plus, if we can't get this all
>> done within the allotted time, the damage control is very favorable to
>> the customer and penalizes me which is as it should be (Google of
>> course getting its cut either way -- actually a bit more should GAE be
>> running slowly thereby expending more CPU cycles than normal :-).
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to [email protected].
> To unsubscribe from this group, send email to
> [email protected].
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
>
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
