----- Original Message ----- > From: "Fraser Adams" <[email protected]> > To: "Ken Giusti" <[email protected]> > Cc: [email protected], "ke1g nh" <[email protected]> > Sent: Thursday, April 11, 2013 2:40:36 AM > Subject: Re: QMFv2 object update bug [WAS Re: Questions from a novice] >
<snip> > One thing though, I think that we discussed on the main thread (and Bill > mentioned this too) about providing an extension to specify the QMF > version to consume. To be frank - I think any effort expended going forward to make it easier to have both QMFv1 and QMFv2 work well together is an effort wasted given QMFv1 deprecation: http://qpid.2158936.n2.nabble.com/QMF-and-Broker-Management-td7151372.html > For most users it might not be crippling to get the same data twice, but > it's unwise to presuppose application use cases, I was messing around > once with a test case covering a fairly large number of consumers (1000 > connections, the same for queues and a few thousand bindings) I hadn't > really thought much of it previously until I noticed that my MIS traffic > was taking way more bandwidth than my real traffic (I was monitoring > queue, exchange and connection traffic) given that the QMF2 data are > presented as Maps IIRC with a 10 second data interval I was getting MB/s > :-D (ever come across the Heisenberg Uncertainty Principal :-) I > definitely assert that it applies as much to data management as quantum > physics ;->) > Agreed - but the sooner we turn off unsolicited QMFv1 updates as planned, the sooner the broker stops sending the same data twice. > > Actually that reminds me, you said "V2 handles these updates in a > different way: it is optimized to generate only one update that combines > both statistics and properties " - are you sure? > Yes - once QMFv2 updates are fixed (as they are on trunk). For example, here are two consecutive updates sent by the broker for a queue named "fleabag": Object [0-1-1-0-org.apache.qpid.broker:queue:fleabag] ObjProps PROPS=[[(vhostRef, 0-0-1-0-org.apache.qpid.broker:vhost:org.apache.qpid.broker:broker:amqp-broker,/), (name, 'fleabag'), (durable, False), (autoDelete, True), (exclusive, False), (arguments, {})]] ObjProps STATS=[[(msgTotalEnqueues, 4), (msgTotalDequeues, 0), (msgTxnEnqueues, 0), (msgTxnDequeues, 0), (msgPersistEnqueues, 0), (msgPersistDequeues, 0), (msgDepth, 4), (byteDepth, 8), (byteTotalEnqueues, 8), (byteTotalDequeues, 0), (byteTxnEnqueues, 0), (byteTxnDequeues, 0), (bytePersistEnqueues, 0), (bytePersistDequeues, 0), (msgFtdEnqueues, 0), (msgFtdDequeues, 0), (byteFtdEnqueues, 0), (byteFtdDequeues, 0), (msgFtdDepth, 0), (byteFtdDepth, 0), (releases, 0), (acquires, 0), (discardsTtl, 0), (discardsRing, 0), (discardsLvq, 0), (discardsOverflow, 0), (discardsSubscriber, 0), (discardsPurge, 0), (reroutes, 0), (consumerCount, 0), (consumerCountHigh, 0), (consumerCountLow, 0), (bindingCount, 1), (bindingCountHigh, 1), (bindingCountLow, 0), (unackedMessages, 0), (unackedMessagesHigh, 0), (unackedMessagesLow, 0), (messageLatencyMin, 0), (messageLatencyMax, 0), (flowStopped, False), (flowStoppedCount, 0)]] ---- Object [0-1-1-0-org.apache.qpid.broker:queue:fleabag] ObjProps PROPS=[[]] ObjProps STATS=[[(msgTotalEnqueues, 6), (msgTotalDequeues, 0), (msgTxnEnqueues, 0), (msgTxnDequeues, 0), (msgPersistEnqueues, 0), (msgPersistDequeues, 0), (msgDepth, 6), (byteDepth, 12), (byteTotalEnqueues, 12), (byteTotalDequeues, 0), (byteTxnEnqueues, 0), (byteTxnDequeues, 0), (bytePersistEnqueues, 0), (bytePersistDequeues, 0), (msgFtdEnqueues, 0), (msgFtdDequeues, 0), (byteFtdEnqueues, 0), (byteFtdDequeues, 0), (msgFtdDepth, 0), (byteFtdDepth, 0), (releases, 0), (acquires, 0), (discardsTtl, 0), (discardsRing, 0), (discardsLvq, 0), (discardsOverflow, 0), (discardsSubscriber, 0), (discardsPurge, 0), (reroutes, 0), (consumerCount, 0), (consumerCountHigh, 0), (consumerCountLow, 0), (bindingCount, 1), (bindingCountHigh, 1), (bindingCountLow, 1), (unackedMessages, 0), (unackedMessagesHigh, 0), (unackedMessagesLow, 0), (messageLatencyMin, 0), (messageLatencyMax, 0), (flowStopped, False), (flowStoppedCount, 0)]] You'll notice the -second- update does not contain any properties, because the broker knows that the properties have not changed since the previous update was sent. > > Anyway... and as I say it has been a little while. One thing that I seem > to recall was that if my query subscription was based on properties all > was well, but if it included statistics it wasn't doing what I was > expecting. From memory what I *think* happens is that "one update > combines both statistics and properties" indeed, but that update only > contains any statistics that have actually changed. <snip> Yes, you'll see that the broker sends all statistics for a given object rather than just those that changed. While the QMFv2 spec may indicate only the statistics that have changed should be sent - this is very impractical in practice. That spec should be re-written to say "-may- send only those that have changed". To send only those stats that have changed will require the broker to perform lots of extra work that will end up impacting the performance of its primary purpose - transferring messages. Think about how we'd have to implement this - the broker would have to save a snapshot of every object's statistics between update intervals, then check every single counter for a difference at the next poll (all under a lock, btw). Imagine the impact this would have on the broker's performance, especially as we scale out queues, exchanges, connections, etc. Not worth trading off for mgmt application convenience, IMHO. -K --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
