On Sat, Aug 20, 2011 at 2:34 PM, Stephen <[email protected]> wrote: > On Sat, Aug 20, 2011 at 9:50 PM, Jeff Schnitzer <[email protected]> wrote: >> >> Second: The key is being able to get a strongly consistent sum of the >> sharded counters. > > You only really need to know whether widgets are in stock or out of stock. > > While widgets are in stock, record the customers intent to purchase by > placing the order in a pull-queue. Bulk process the queue in batches > of 1000 requests, deducting 1000 from a total-widgets counter. Flip > the out of stock bit when you run out. > > All orders received get an entity in the datastore marked with > success, or failure for the stragglers which entered the queue before > the out of stock bit could be flipped. > > Ajax form submission on the front end, receive a ticket immediately, > poll the ticket until success or failure.
This is a really interesting idea, especially combined with the channel api to push notification to the client. But I'm worried about a couple things: 1) Are there periods when pull task queues go down? There are for push queues. And they back up occasionally. I would hate to have the experience that the user clicks "buy" and then waits for significant times (more than a second or two) before they get an ok to proceed. 2) Wouldn't this require continual looping to see if there is anything on the task queue? If it checks every 2 seconds, that's potentially 2 seconds of user waiting. Or if this is done by kicking off a push task, we're back to long waits or failures when the push task system is backed up or disabled. 3) I'm a little concerned about getting enough parallelism with this. If the task processor leases 1,000 tasks, they'll be a mishmosh of different kinds. At the very least it will require writing 1,000 purchase records, which will probably take a few seconds even with async. Maybe the answer is to do smaller chunks and run multiple processors in parallel, but now we get back to potential collision and sharding issues. And the total throughput of the system will be 1,000 per (however long it takes) rather than 1,000 per purchasable item per (however long it takes). This feels solveable but it might be a balancing act rather than straightforward engineering. Just to add a few bits of complexity that I didn't mention the first time: * The UI must display the # of units left * There's a shopping-cart of sorts whereby the product is deducted from inventory when it gets added to a cart. It gets returned to the pool if the transaction hasn't been completed in 15 mins (similar to the way eventbrite works). However, I don't think this extra complexity is too hard to add to a basic purchase system that can already handle restricting sales to a specific # of units. Jeff -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
