Oh, and my 2 cents. Transfers to process is a list XFERID Jeff to Brandon $100 XFERID Brian to Brandon $100 XFERID Brandon to Safeway $7
Transactions to Process is a list. TXID 51 Transfer $100 From Jeff TXID 52 Transfer $100 From Brian TXID 53 Transfer $7 From Brandon TXID 54 Transfer $100 to Brandon TXID 55 Transfer $100 to Brandon TXID 56 Transfer $7 to Safeway Brandon's account is a list BWID 1 Account Opened with $150 BWID 2 Received $78 via TXID 38 BWID 3 Receive $100 via TXID 51 BWID 4 Receive $100 via TXID 52 BWID 5 Send $7 via TXID 53 Jeffs account is a list JSID 1 Account opened with $5000 JSID 2 Send $100 via TXID 51 You don't move Dollars from X to Y. You always move to and from general fund. You add and remove dollars based on Transaction IDs which must be unique. You rectify accounts in a batch which includes which transactions were rectified. To determine the current balance. I apologize for the brevity, I was typing this quickly between phone calls, but hopefully this demonstrates the solution. This solution creates more moving parts as a total, but fewer moving parts per type, and the unique ids means that if you have two of the same ID you ignore them when processing. Also note that if I were implementing this Transfer requests would be assigned an ID. The User would enter the information. They would be given a confirmation. And then the transfer would be added to the queue. Unconfirmed transfers would still consume a transfer ID. -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Jeff Schnitzer Sent: Saturday, January 21, 2012 3:42 PM To: [email protected] Subject: Re: [google-appengine] Re: Once-only transactions I think we might be talking about two different things. I'm not worried about the user clicking the button twice; I prevent that with javascript on the frontend. I just want an idempotent transaction that safely transfers $5 from one account to another. Without idempotence I can't hide contention (or any other datastore failure) from the user. Creating a txnId entity outside the transaction and deleting it in the transaction gives me idempotence; no matter how many times the transaction runs, only one of those transactions will complete and delete the txnId entity and all the others will either rollback (eg optimistic concurrency failure) or notice that the txnId has been deleted and immediately return success. The problem with calling to an external system (the cc processing service example) is that there's no way to enlist it in the transaction. You need to build up the framework of a 2pc transaction to the cc service. All the cc processing services I know of sidestep the issue by repeatedly posting some sort of IPN to you, which you are expected to handle idempotently yourself. Actually, there's another variation - some of them have separate auth/capture steps which are kind of like a 2pc transaction. Jeff On Sat, Jan 21, 2012 at 6:04 PM, AndyD <[email protected]> wrote: > Yes, I see what you're saying. But in looking at both your and > Robert's approaches, I still have questions. > > Jeff, how does your approach avoid executing the operation twice? > Let's say there's a web page with a submit button that triggers the > process, and the user clicks that button twice. The algorithm you outlined... > > Create TxnID entity instance > Repeat until success { > * start txn > * load TxnID (if null, return success immediately) > * debit $5 from AccountA > * credit $5 to AccountB > * delete TxnID > * commit txn > } > > ...if run serially, would do the operation twice, wouldn't it? The > first execution would create the TxnID entity, then in the loop, it > would load that entity successfully, perform the operation, then > delete the TxnID entity. Then, when the 2nd button click is handled, > the operation would be be performed again. > > Robert, I didn't quite follow your algorithm. Does your marker > indicate that the operation has not yet been performed? If so, > shouldn't 2b say "if the marker is not found, abort? Also, is there > an implied retry mechanism (e.g. a task) encompassing step 2? > > It seems to me there are multiple challenges: > 1) eliminate the potential of concurrency > 2) retry on failure > 3) only do the operation once > > If we don't eliminate concurrency, then two parallel threads, in the > same or different instances, could find the operation not yet > completed and both perform it. If we indicate (by the presence or > absence of a marker) that the operation has been completed, then two > serial executions would both perform it. And if we don't have a retry > mechanism, there is a risk that the operation won't be performed at all. > > It still seems to me that the task queue approach addresses 1 (by not > dequeing a task to two handlers simultaneously and also not accepting > a duplicate task into the queue) and 2 (by automatically retrying on > failure), but as Jeff pointed out, not 3. If you add an "operation > complete" marker entity to the picture that gets written in the same > transaction as the run-once operation, does that cover all the bases? > The marker would still need to be cleaned up at some later point, of course. > > Just to make it interesting, what if the operation to be performed > once was not a datastore operation? What if it was a call to an > external system (like a credit card processing service)? You could > successfully make the external call, but fail to write the "operation complete" entity. > > -Andy > > > -- > You received this message because you are subscribed to the Google > Groups "Google App Engine" group. > To view this discussion on the web visit > https://groups.google.com/d/msg/google-appengine/-/aKzPhfSgF0AJ. > > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
