Hi Praveen, Doing asynchronous commits to the store in the way implemented for the non-transactional sessions is basically not compatible with the use of transactions. Completion of the transaction is the guarantee that the persistent message was put on disk when using a persistent store (which is really the key reason to use persistence+transactions in the first place, guarantees) so the only way to achieve any performance increase in that situation would basically be for the broker to outright lie, which gets you back to having almost the same guarantees (i.e none really) as the non-transactional case.
By removing that guarantee the message was safe on disk you wouldnt really be in much/any better a position than using a non-transacted publishing session, because although you could definitively say the message had hit the broker for processing you would still never actually know whether a given message had made it to disk and onto a queue if the broker were to go to down. Furthermore, you could quite likely still be committing the message to your own database before it was on disk at the broker, in which case you may as well do that before you ever try sending it and just use non-transactional publishing. Whether a message was to be lost during transit to the broker or after reaching the broker but before being safe on disk if using async store commits, your application would still need to be capable of replaying messages from the database and possibly being able to handle duplicates, so it doesnt really change anything from an application implementation point of view either. Given all that, it seems like the changes Rob made to the Java brokers handling of persistent messages on non-transacted sessions should suit your usage just as well if not better than introducing the irregular(/broken from most viewpoints) transactional behaviour would. In fact, the behaviour there now could be noticably faster than your proposal in many cases due to the lack of transactional round trips, again seeming to make it more suited to the general desire of increasing persistent messaging performance with some acceptability of losses. Robbie On 28 January 2012 01:42, Praveen M <[email protected]> wrote: > Hi Rob, > > > Sorry for the late reply. I finally got around to reading your changelist > and also playing with it. > I realize that the change you've done is for sessions with auto_commit mode. > > I was wondering if the async override could be applied to transacted mode > too. > > If you read my earlier mail, > > " My sequence of operations is, > 1) Enqueue messages -> persist message in my database outside Qpid -> > enqueue to Qpid in persistant mode > Since my message is already persisted in my database, I'm ok > with losing amessage in BDB, and would like to enhance throughput of my > overall enqueue" > > Though persisting the message in my database, the reason why I'm using > transacted session is to protect against a case like below. > > If I use auto_commit, the sequeue of operations are. > 1) my enqueue server -> writes to my database outside qpid -> enqueue to > qpid (auto_commit) -> commit my database write outside qpid > > But there is a case where, after i do the enqueue to qpid and commit my > database write, my enqueue server can crash and the auto_committed message > might be lost, as there is > no guarantee that it has reached the broker (could be still in network > buffer or in transit..but not yet reached the broker). > > To avoid this scenario I was thinking of two options. > > To use publish_sync option with auto_commit > 1) my enqueue server -> writes to my database outside qpid -> enqueue to > qpid (auto_commit with publish_sync) -> commit my database write outside > qpid > > The downside to this is, publish_sync isn't really a JMS terminology and > also, on the client side, there is a lock held on the connection with > publish_sync option, avoiding other sessions sharing the connection to make > progress. I spoke to Robbie about this earlier, and his suggestion was to > try and stick to JMS terminology if possible. > > 2) Use transacted mode commit > my enqueue server -> writes to my database outside qpid -> enqueue to qpid > (transacted_mode) -> commit qpid session-> commit my database write > outside qpid > > This looked pretty reliable and conforms to JMS, and I was hoping to use > this, as there is a guarantee that the message is pushed to the broker > before I write to my own database. > The one enhancement that I thought will improve the performance of the > roundtrip for this case was, to set syncCommit to false, so that I get the > guarantee that the broker got my message, but also return back quickly than > wait for the write to happen(which takes most of the roundtrip time in the > transaction). > > I do understand that what I am trying to achieve is quite different from > normal usage. > > I'd like to hear what you think about this problem and what you would > suggest as my best option in this case. > > Thanks, > Praveen > > > > Thanks, > Praveen > > > On Sat, Jan 21, 2012 at 12:00 PM, Rob Godfrey <[email protected]>wrote: > >> Hi Praveen, >> >> I've checked in my changes on trunk if you want to have a play around... >> let me know if you run into any issues, >> >> Cheers, >> Rob >> >> On 20 January 2012 16:57, Praveen M <[email protected]> wrote: >> >> > Hi Rob, >> > >> > Thanks for writing. Please see inline. >> > >> > On Fri, Jan 20, 2012 at 1:27 AM, Rob Godfrey <[email protected] >> > >wrote: >> > >> > > Hi Praveen, >> > > >> > > apologies for the slow response here... >> > > >> > > On 14 January 2012 03:27, Praveen M <[email protected]> wrote: >> > > >> > > > Hi, >> > > > >> > > > I would like to exploit the Berkeley DB's nosyncCommit (lazy >> commit) >> > > > option to exploit enqueue performance. >> > > > I see that in the current BDBMessageStore implementation there is no >> > way >> > > to >> > > > override the syncCommit Boolean value and >> > > > would like to know if this could be made into an option. >> > > > >> > > > Also, that said It'll be nice to know if someone had used the >> > > > nosyncCommit() option that BDB provides and if using it does any >> > > invisible >> > > > harm. (DB corruptions etc.). >> > > > >> > > > >> > > So, I have coincidentally been working on a patch to increase >> throughput >> > > for non transactional messages which makes greater use of no-sync >> > commits. >> > > The patch is basically to allow a certain amount of "out-of-order" >> > > processing while still maintaining the guarantees on durability that >> the >> > > AMQP specification states that you should get. For non transactional >> > > durable messages this is giving me something like a 6.5x increase in >> max >> > > total throughput. >> > > >> > >> > *Ah, this is great. Exactly what I was looking for. * >> > >> > > >> > > Unfortunately this patch did uncover a couple of issues, one being the >> > need >> > > to have some code in Qpid to handle occasional BDB lock timeouts... and >> > the >> > > other a very occasional bug in BDB itself which they guys at Oracle now >> > > have a fix for, but won't be publicly released until their regular >> > update. >> > > There is a reasonably easy workaround for this which I shall >> incorporate >> > > into my patch, but it may take me a couple more days to get it all >> > > complete. >> > > >> > > Note that my patch is a little different to what you are suggesting - >> in >> > > your case a message may go on to a queue in memory and then be >> delivered >> > > without the guarantee that it was sync'd to disk... in my patch the >> > message >> > > won;t be released into the in-memory queue until it is sync'd, however >> I >> > am >> > > batching up more such enqueues to get better performance. In my tests >> > the >> > > total max throughput with these two possible solutions is pretty much >> > > identical, however the latency on individual messages is likely to be a >> > > little different. >> > > >> > >> > *That sounds even better. Your solution fits the bill perfectly.* >> > *I'd prefer the message getting sync'd to disk first before delivery >> too.* >> > * * >> > >> > > >> > > Now, to why do I want to override that value to false? (I do realize >> that >> > > > turning it to false, doesn't guarantee that the message was written >> to >> > > > disk) >> > > > In my use case. I have a database outside Qpid which persists all the >> > > > messages enqueued (for durability and transactionality with message >> > > > callback operations.). >> > > > >> > > > My sequence of operations is, >> > > > >> > > > 1) Enqueue messages -> persist message in my database outside Qpid -> >> > > > enqueue to Qpid in persistant mode >> > > > Since my message is already persisted in my database, I'm ok with >> > losing >> > > a >> > > > message in BDB, and would like to enhance throughput of my overall >> > > enqueue >> > > > operation by using the nosyncCommit option. >> > > > The reason I'm not using non-persistant mode in Qpid is because I >> don't >> > > > want to overwhelm the memory available by flooding the broker with >> > > > messages. >> > > > >> > > > In the event of a broker crash, I plan to restore the messages from >> my >> > > > database outside Qpid. (I realize this could turn out a little >> > > expensive). >> > > > I've been thinking >> > > > that if there is a replication/HA solution to have messages go to a >> > > primary >> > > > and secondary broker (in a 2 instance cluster), in the event of a >> > crash, >> > > > I'll have to >> > > > only restore the most recent messages from my database outside Qpid >> > upto >> > > a >> > > > certain checkpoint to which the primary and secondary are sync'ed >> (Not >> > > sure >> > > > if this is achievable.). >> > > > >> > > > I do realize that my architecture is a little complicated. Please do >> > let >> > > me >> > > > know if you have more questions about what I'm trying to achieve >> here, >> > > I'll >> > > > be happy to >> > > > clarify doubts. >> > > > >> > > > I'd really like to know if I can add an option to override the >> > syncCommit >> > > > to false without any major side-effects apart from what I expect. >> > > > >> > > > >> > > > >> > > Does my patch seem like it would satisfy your requirements? >> > > >> > >> > *Yes. It's quite exactly what I was looking for. I have a couple of >> > questions on the HA* >> > *state of things though. Please do see our other thread on that. It will >> be >> > really * >> > *valuable if you mention what you think.* >> > * >> > * >> > *Thank you Rob.* >> > * * >> > >> > > >> > > Cheers, >> > > Rob >> > > >> > >> > >> > >> > -- >> > -Praveen >> > >> > > > > -- > -Praveen --------------------------------------------------------------------- Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:[email protected]
