Sorry was sick yesterday, catching up on the email now. On Thu, Mar 24, 2011 at 6:42 AM, Marnie McCormack < [email protected]> wrote:
> Hi Rajith, > > Thanks (and oops). > > On the stats for sync-publish vs transactions, guessing these are all > transient -> with the Java broker ? > > Nope these are all durable with the C++ broker. > For persistent messages, which you'd need for reliable delivery with the > Java broker, the figures would be closer together since the I/O write is the > larger cost. > Performance apart, using transactions IMO is not a good idea unless it's extremely necessary. > > Thanks, > Marnie > > On Wed, Mar 23, 2011 at 8:20 PM, Rajith Attapattu <[email protected]>wrote: > >> Marnie, >> >> Looks like you forgot to hit replyAll. >> So I am forwarding your email to the list along with my comments. >> >> >> On Wed, Mar 23, 2011 at 12:47 PM, Marnie McCormack < >> [email protected]> wrote: >> >>> Sorry, which broker are you usiong Sergey ? I'd assumed the Java broker >>> ... >>> >> >> Since the Java broker supports 0-10 now, you could use sync_publish. >> >> >>> >>> I'd probably disagree that you can achieve guaranteed delivery without a >>> transaction or its equivalent (which sync publish effectively is ?). >>> >> >> Synchronous publishing is not equivalent to transactions, but I understand >> what you meant here. You were probably alluding to the fact that in both >> cases messages are published in a synchronous way. i.e When the tx completes >> (or when the send method returns in sync-publish) you know that the broker >> has acked the message. >> >> However transactions have a lot more overhead than a simple sync publish, >> hence a higher performance penalty with transactions. >> For example look at the following relative numbers for a producer/consumer >> pair using the same setup. >> >> sync publish 15k msg/sec >> tx size =2 48 msg/sec (thats right just 48 messages per sec) >> tx size=1000 14k msg/sec >> >> Also from an application point of view, non transactional code is more >> simple and easy to write than transactional code. >> I would normally stay away from transactions unless I really have to. It >> basically negates a lot of benefits of using messaging. >> >> Regards, >> >> Rajith >> >> >>> >>> Regards, >>> Marnie >>> >>> On Wed, Mar 23, 2011 at 4:21 PM, Rajith Attapattu >>> <[email protected]>wrote: >>> >>>> Hi Sergey, >>>> >>>> For Qpid to guarantee 'at-least-once' delivery on publish, you don't >>>> need to use transactions. >>>> >>>> Broker failures >>>> ----------------- >>>> (1) In a stand-alone-broker you need to use persistence to ensure that >>>> messages survive a broker crash. >>>> (2) In a clustered broker you may be able to get away without using >>>> persistence (*) >>>> >>>> (*) Only the c++ broker supports clustering. >>>> If you don't use persistence, then in the event of a total cluster >>>> failure you will loose messages. >>>> Therefore it depends on how much you can tolerate these occasional >>>> failures. >>>> >>>> Application/Client lib failures >>>> --------------------------------- >>>> In the event of a failure in the application that sends the messages, >>>> you may loose messages. >>>> i.e if there are unacked messages in the client libs replay buffer, they >>>> will be lost. >>>> You could avoid that by using >>>> >>>> For JMS client >>>> ------------------- >>>> (1) Using synchronous publishing. Use >>>> -Dsync_publish={persistent|transient|all}. >>>> (2) Transaction - as Marnie pointed out. >>>> >>>> Performance wise there isn't much of a difference between option 1 & 2 >>>> if you use large batches (ex 1000 msg/batch) for transactions. >>>> But ff you use small tx batches then there is an appreciable difference >>>> in performance. >>>> >>>> Therefore I recommend you use -Dsync_ack=persistent. >>>> Also the application code is much simpler when writing non transactional >>>> code. >>>> >>>> For other clients >>>> -------------------- >>>> If you can handle reliable replay at the application level then you >>>> could just use asynchronous publishing without worrying about application >>>> failure. >>>> Ex. If your application is retrieving orders from a queue and processing >>>> it before sending a reply, then you could ack the order messsage after the >>>> broker acks your reply message. >>>> >>>> Hope this helps. >>>> >>>> Rajith >>>> >>>> >>>> On Wed, Mar 23, 2011 at 11:21 AM, Marnie McCormack < >>>> [email protected]> wrote: >>>> >>>>> Hi Sergey, >>>>> >>>>> To achieve guaranteed message delivery you need to be using >>>>> transactions on >>>>> *publish*. Qpid then guarantees 'at least once' delivery. You'd offset >>>>> the >>>>> reliable delivery cost against the cost or message loss during a >>>>> failure >>>>> from sending non-transacted messages. The use of transactions and >>>>> persistence for reliable messaging adds cost in the shape of I/O. >>>>> >>>>> However, you could opt not to use transactions on consume since you >>>>> cannot >>>>> conceptually lose a message during consumption - if there's a failure >>>>> and a >>>>> message ack doesn't get back to the broker then you'll simply get the >>>>> message back again on restart. This assumes that you >>>>> listen/catch/handle >>>>> errors and exception on consume such that you don't have application >>>>> level >>>>> side effects. >>>>> >>>>> Regards, >>>>> Marnie >>>>> >>>>> On Tue, Mar 22, 2011 at 1:03 PM, <[email protected]> wrote: >>>>> >>>>> > Hi there >>>>> > >>>>> > I'm working with qpid 0.8 by means of JMS API and I'm using >>>>> > CLIENT_ACKNOWLEDGE mode to get higher performance comparing to >>>>> > transactinal processing. >>>>> > As a rule performance is rather pure in both cases. In transactional >>>>> mode >>>>> > I can receive about 1.6K messages per second, in CLIENT_ACKNOWLEDGE >>>>> mode >>>>> > I'm able to receive about 3.6K messages per second and in >>>>> AUTO_ACKNOWLEDGE >>>>> > mode I'm able to receive about 70K messages per second. >>>>> > >>>>> > Are there any recommendations how to improve performance and get a >>>>> > guaranted message delivery? >>>>> > >>>>> > >>>>> > Best Regards, >>>>> > Sergey >>>>> > _______________________________________________________ >>>>> > >>>>> > The information contained in this message may be privileged and conf >>>>> > idential and protected from disclosure. If you are not the original >>>>> intended >>>>> > recipient, you are hereby notified that any review, retransmission, >>>>> > dissemination, or other use of, or taking of any action in reliance >>>>> upon, >>>>> > this information is prohibited. If you have received this >>>>> communication in >>>>> > error, please notify the sender immediately by replying to this >>>>> message and >>>>> > delete it from your computer. Thank you for your cooperation. Troika >>>>> Dialog, >>>>> > Russia. >>>>> > If you need assistance please contact our Contact Center (+7495) >>>>> 258 0500 >>>>> > or go to www.troika.ru/eng/Contacts/system.wbp >>>>> > >>>>> > >>>>> >>>> >>>> >>> >> >
