Russel, the tests were made on a Unix (I guess it was Solaris) system. You told, there is a implicit indexing. But it was in a previous version of MQ (5.2 or even 5.1). When is indexing implemented in distributed MQ?
And I indexing is not the problem, why is it possible, that conventional channels need 0.004 seconds whereas cluster channel need 0.014 seconds? This means cluster channel need three times longer than conventional channels! I do not think, that - on distributed QMgrs - the disk performance may be significant different (because all queues are placed on the same disk). Maybe this is different on z/OS, when XmitQs are put on different page sets. I also do not think of network issues, because - clustered or not - each channel will have its own IP sockets. This could be changed, if a system has more than one IP interface. Are there other reasons? Regards Hubert > -----Ursprüngliche Nachricht----- > Von: MQSeries List <[email protected]> > Gesendet: 06.12.06 10:09:19 > An: [email protected] > Betreff: Re: multi-hop within a cluster > Hubert, > > > You have stated that the colleague tested > get-by an identifier, but not stated that the queue was indexed by that > identifier. > > > If the queue is not indexed by the identifier > used as the key in MQGET processing, then yes, it will go slower than get-next > processing. > > > There is a small overhead in maintaining > an index but if the getters are using get-by-id then it is definitely correct > to index the queue. > > > Russell > > Russell Finn > MQSeries System Test > [EMAIL PROTECTED] > > > > > > Hubert Kleinmanns <[EMAIL PROTECTED]> > > Sent by: MQSeries List <[email protected]> > > > 06/12/2006 07:23 > > > Please respond to > MQSeries List <[email protected]> > > > > To > [email protected] > > cc > > Subject > Re: multi-hop within a cluster > > > > > > > > > Russel, > > I do not know how this is realized internally, but I know about tests, > which a collegue did some years ago. These tests have shown, that reading > with message ID or correlation ID became very slow, when a queue filled > up. I did not see this effect for programs, which read a queue without > specifying a key. > > Do you mean, that reading with correlation ID is not slower than unspecified > reading? > > Of course there are other reasons, why parallel reading of several MCAs > from one single XmitQ may reduce the throughput, but I would expect, that > reading with correlation ID is at least ONE effect. > > Regards > Hubert > > > > -----Ursprüngliche Nachricht----- > > Von: MQSeries List <[email protected]> > > Gesendet: 05.12.06 22:52:27 > > An: [email protected] > > Betreff: Re: multi-hop within a cluster > > > > > This means, > > the MCA has to read every message (because there is no indexing on > the > > correlation ID only) > > > > > > I don't know what that statement is > > based on - would you care to elaborate? > > > > > > On distributed platforms, all queues > > are indexed by each of MsgId, CorrelId, GroupId, MsgSeqNumber and > Offset. > > > > > > On z/OS, a queue can be indexed by either > > MsgId, CorrelId or GroupId. SYSTEM.CLUSTER.TRANSMIT.QUEUE is > indexed > > by CorrelId. > > > > > > Russell > > > > Russell Finn > > > Tel: +44 (0) 1962 818062 > > MQSeries System Test Fax +44 (0) 1962 818062 > > [EMAIL PROTECTED] > > > > > > > > > > > > Hubert Kleinmanns <[EMAIL PROTECTED]> > > > > Sent by: MQSeries List <[email protected]> > > > > > > 05/12/2006 11:37 > > > > > > Please respond to > > MQSeries List <[email protected]> > > > > > > > > To > > [email protected] > > > > cc > > > > Subject > > Re: multi-hop within a cluster > > > > > > > > > > > > > > > > > > It is not the question "reading from one queue > > or from multiple queues". > > > > Conventional channels open their XmitQ exclusively. Then the read > the messages > > unspecified one-by-one from the beginning to the end of the queue > - without > > regarding the contents of the messages. > > > > Cluster channels open the SCXQ shared (this is not the problem). Then > the > > channels read the messages with correlation ID. This means, the MCA > has > > to read every message (because there is no indexing on the correlation > > ID only), look into the correlation ID, check, if this contains the > correct > > channel name, and then gets the message (when the correlation ID fits) > > or step over to the next message - this costs time. > > > > Regards > > Hubert > > > > > > > -----Ursprüngliche Nachricht----- > > > Von: MQSeries List <[email protected]> > > > Gesendet: 05.12.06 11:19:11 > > > An: [email protected] > > > Betreff: Re: multi-hop within a cluster > > > > > > > hubert, > > > > > > i was thinking or rather trying to understand why there could > be a > > potential performance impact because of the fact that cluster-qm have > only > > one xmit-queue and mutliple sender-channels reading from that queue. > > > > > > generally, what's the difference reading from one queue or from > multiple > > queues? > > > > > > (the following statements try to answer the question and are > the basis > > for discussion) > > > > > > the qm has only one log file and per queue a queue file. > > > > > > the maximal performance of a queue manager is theoretical defined > > by the performance of logging. > > > > > > it's possible to assign queue files to specific storage devices. > in > > the extreme case assigning each queue file to a dedicated disk would > drive > > to best performance. the question would be, when is the mq log slowing > > down, so that putting more disks for queue files is not worth. > > > > > > because of that it is better to use multiple queues instead of > just > > one queue. important is to separate the queue files on different disks. > > otherwise there is no performance benefit of just using mutliple queues. > > > > > > for performance it's also important to put mq log files and queue > > files on different disks. > > > > > > messages are normally hold in memory, apart of that persistent > messages > > are always logged. i supose that, if messages are not read for a certain > > time, they are written to the queue file. or if memory is full, messages > > are written to the queue file aswell. so, performance gains or losses > because > > of queue files have to be relativated. that is an argument that it's > not > > so important to use one or multiple queues. > > > > > > cluster-sender-channels read by correlId from the xmit-queue. > so, > > the amount of correlIds corresponds to the amount of > > cluster-sender-channels. > > assuming that there is only a small amount of sender-channels, the > performance > > impact is not so bad, as if it would be thousands of differnt correlIds > > in a queue. > > > > > > thanks for any commentaries, correctives! > > > > > > > > > > > > -------- Original-Nachricht -------- > > > Datum: Mon, 4 Dec 2006 15:16:37 +0100 > > > Von: Hubert Kleinmanns <[EMAIL PROTECTED]> > > > An: [email protected] > > > Betreff: Re: multi-hop within a cluster > > > > > > > Hi, > > > > > > > > in principle you are right, the information about the Q > locations, > > cluster > > > > QMgrs and channel definitions are cached locally. So you > will > > have ONLY A > > > > LITTLE performance impact (which you mostly will not see), > because > > these > > > > informations are refreshed sometimes. > > > > > > > > BUT, there are points, which may have a SIGNIFICANT impact: > All > > cluster > > > > sender channels read from ONE XmitQ (the SYSTEM.CLUSTER.TRANSMIT.QUEUE). > > > > These channels do not read one by one, but the read messages > > with correlation > > > > ID (because the name of the channel filled into the correlation > > id field). > > > > On Unix you do not have an indexing on queues, so reading > may > > (and I think > > > > will be) slower, than reading messages one by one. Additionally > > you normally > > > > have more than one channel, which reads from the > > > > SYSTEM.CLUSTER.TRANSMIT.QUEUE, and I guess this will not > increase > > the performance :-(. > > > > > > > > So it may make sense, NOT to use MQ clustering, when you > have > > performance > > > > issues (or you use faster networks, disks or whatever). > As I > > told in one of > > > > my last mails: It depends on ;-). > > > > > > > > Regards > > > > Hubert > > > > > > > > > > > > > > > > > -----Ursprüngliche Nachricht----- > > > > > Von: MQSeries List <[email protected]> > > > > > Gesendet: 30.11.06 18:08:09 > > > > > An: [email protected] > > > > > Betreff: Re: multi-hop within a cluster > > > > > > > > > > > > > does other people have/had performance issues with > mq clustering. > > > > > > > > > > my understanding of mq clustering is: > > > > > > > > > > IT is always a tradeoff between flexibility and performance. > > mq > > > > clustering simplifies administration because remote queues > and > > channels are created > > > > automatically. this costs something. > > > > > > > > > > but, this costs only arises the first time that objects > > are created > > > > automatically. > > > > > > > > > > and, repository look-ups are kept in cache. > > > > > > > > > > that's how mq tries to minimize the performance loss. > > > > > > > > > > please correct me or improve my statements. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -------- Original-Nachricht -------- > > > > > Datum: Tue, 28 Nov 2006 14:52:01 -0800 > > > > > Von: Christopher Warneke <[EMAIL PROTECTED]> > > > > > An: [email protected] > > > > > Betreff: Re: multi-hop within a cluster > > > > > > > > > > > Reasons why I disagree: > > > > > > > > > > > > 1. For all of the simplification that clustering > > > > > > brings, there is a loss of performance. We > see > > times > > > > > > of .004 seconds for unclustered (sdr/rcvr pair) > > > > > > message transmission avg. verses .014 seconds > for > > > > > > cluster message transmission avg. In this > shop, > > 10ms > > > > > > here, 10ms there, means not having transactions > > > > > > complete in less than a second. We need > subsecond > > > > > > response times. > > > > > > > > > > > > 2. It's a lot easier to change 1 routing > rule > > in a > > > > > > control table, than to change n number of mq object > > > > > > definitions. Sure, in some little test environment > > > > > > you can go and change things as you want - on > a > > > > > > production system, the change mangement work to > > > > > > redefine the mq environment will make you wish > you > > > > > > only had to change 1 routing rule. > > > > > > > > > > > > 3. QSGs and Shared Channel solutions are > better > > than > > > > > > clustering - if you have z/OS. If not, maybe > > > > > > performance is not such an issue and you can cluster > > > > > > the thing to death. Most of the MQ performance > > issues > > > > > > that we see, result from designs that work. Messages > > > > > > get where they are supposed to go. Just > not that > > > > > > efficiently - and then things begin to fail under > > > > > > heavy load. > > > > > > > > > > > > 4. What exactly does "breaking a fly > on > > the wheel" > > > > > > mean? Do you mean that it is overkill? There > > is a > > > > > > point in scale where brokering is the easiest > > > > > > approach, and your clustering solution becomes > the > > fly > > > > > > in vaseline. Think about support for this, > as > > if you > > > > > > were not the person supporting it. Maybe > the > > 'next > > > > > > guy' wants to do it the easy way, and have the > > > > > > application people be responsible for routing > changes. > > > > > > > > > > > > > > > > > > So there... > > > > > > > > > > > > > > > > > > --- Hubert Kleinmanns <[EMAIL PROTECTED]> > > wrote: > > > > > > > > > > > > > I think, using a message broker to route > messages > > is > > > > > > > like breaking a fly on the wheel ... > > > > > > > > > > > > > > MQ clustering is designed for simplifying > > > > > > > administration tasks. Now you need not to > define > > any > > > > > > > channel definitions or remote Q definitions > > > > > > > manually. The cluster algorithm propagates > the > > > > > > > neccessary definitions to the cluster members, > > but > > > > > > > the Qs and channels are going the direct > way. > > > > > > > > > > > > > > But, you could define alias queues on a gateway > > > > > > > system, which point on local queues on the > target > > > > > > > systems. In this case alias queues and local > queues > > > > > > > must have different names. I did not test > it, > > but > > > > > > > this should work. > > > > > > > > > > > > > > I would NOT use the mechanism I described > above, > > > > > > > because it is contrary to the cluster concepts. > > > > > > > Instead I would use overlapping clusters > as follows: > > > > > > > > > > > > > > 1. Put the sending QMgrs and the gateway > QMgrs > > into > > > > > > > one cluster (e. G. CL1). > > > > > > > > > > > > > > 2. Put the receiving QMgrs and the gateway > QMgrs > > > > > > > into a second cluster (e. G. CL2). > > > > > > > > > > > > > > 3. Define alias Qs on the gateway QMgrs in > CL1, > > > > > > > which point to local Qs in cluster CL2. > > > > > > > > > > > > > > 4. On the sending QMgrs put the messages > to the > > > > > > > alias Qs in cluster CL1 on the gateway QMgrs. > > > > > > > > > > > > > > 5. The gateway QMgrs now resolve to the local > > Qs in > > > > > > > cluster CL2. > > > > > > > > > > > > > > 6. The sending QMgrs do not see the local > Qs in > > > > > > > cluster CL2, so the names of the alias Qs > on the > > > > > > > gateways and the local Qs on the receiving > QMgrs > > may > > > > > > > have the same name. > > > > > > > > > > > > > > I tested point 6 and it worked, but maybe > when > > you > > > > > > > have to gateways, they could route the messages > > to > > > > > > > the other gateway (and back). In WebSphere > MQ > > > > > > > version 6 this may be prevented by using > the > > > > > > > CLWLRANK attribute. > > > > > > > > > > > > > > Regards > > > > > > > Hubert > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -----Ursprüngliche Nachricht----- > > > > > > > > Von: MQSeries List > > > > > > > <[email protected]> > > > > > > > > Gesendet: 27.11.06 23:46:48 > > > > > > > > An: [email protected] > > > > > > > > Betreff: Re: multi-hop within a cluster > > > > > > > > > > > > > > > > > > > > > > ...or, you could employ/author a message > > broker > > > > > > > with > > > > > > > > clientconns connecting to whatever qmgrs > > are > > > > > > > involved, > > > > > > > > give it http connections though it's > own > > tomcat > > > > > > > server > > > > > > > > - call it your Enterprise Service Buss. > > > > > > > > It depends on how complex you want your > plumbing > > > > > > > to > > > > > > > > be. > > > > > > > > Do you want to change a routing entry > in > > a control > > > > > > > > table, or restructure the mq environment, > > when > > > > > > > routing > > > > > > > > changes are made? > > > > > > > > > > > > > > > > > > > > > > > > --- Neil Casey <[EMAIL PROTECTED]> > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > > > > > my take on this (for what it's > worth). > > > > > > > > > > > > > > > > > > In MQ there is an assumption that > all > > queue > > > > > > > managers > > > > > > > > > within a cluster are > > > > > > > > > able to communicate directly. If > you > > attempt to > > > > > > > put > > > > > > > > > a message to a cluster > > > > > > > > > queue, the FR will notify your > local > > QM of the > > > > > > > > > locations of that queue, > > > > > > > > > and of the channel definitions > required > > to get > > > > > > > to > > > > > > > > > those locations. These > > > > > > > > > definitions are based on the cluster > > receiver > > > > > > > > > channels to the target queue > > > > > > > > > managers, and so direct connection > capability > > is > > > > > > > > > needed. That is, in order > > > > > > > > > for messages to be sent to any > possible > > > > > > > destination > > > > > > > > > within the cluster, > > > > > > > > > the cluster must be fully inter-connected. > > > > > > > > > > > > > > > > > > In an environment where this is > not > > the case, as > > > > > > > in > > > > > > > > > this example, you > > > > > > > > > should not define one cluster, > but rather > > 2 or > > > > > > > more. > > > > > > > > > Each cluster you > > > > > > > > > define should be fully connected, > and > > there > > > > > > > should > > > > > > > > > be one or more queue > > > > > > > > > managers in both clusters. You > can then > > use > > > > > > > standard > > > > > > > > > cluster gateway > > > > > > > > > techniques to forward messages > between > > the > > > > > > > clusters. > > > > > > > > > > > > > > > > > > There is no simple way to implement > > forwarding > > > > > > > of > > > > > > > > > messages via > > > > > > > > > intermediate queue managers in > a single > > cluster, > > > > > > > > > although you might be > > > > > > > > > able to do it by... > > > > > > > > > > > > > > > > > > QM1, QM2 and QM3 are members of > CLUS1. > > They all > > > > > > > have > > > > > > > > > connectivity to the > > > > > > > > > full repositories, and QM2 has > 2 way > > > > > > > connectivity to > > > > > > > > > QM1 and QM3. QM1 and > > > > > > > > > QM3 cannot be connected directly, > and > > are > > > > > > > partial > > > > > > > > > repositories. > > > > > > > > > Set up the target queue (TQ1) on > a queue > > manager > > > > > > > > > (QM1), but NOT visible in > > > > > > > > > the cluster. > > > > > > > > > Set up a QRemote on a fully connected > > queue > > > > > > > manager > > > > > > > > > (QM2), with QUEUE(TQ1) > > > > > > > > > RQMNAME(QM1) XMITQ(' ') RNAME(TQ1) > > > > > > > CLUSTER(CLUS1) > > > > > > > > > DEFBIND(NOTFIXED) > > > > > > > > > Put to TQ1 from your isolated queue > > manager > > > > > > > (QM3). > > > > > > > > > It should resolve to > > > > > > > > > the queue on TQ1 and send the message > > there. > > > > > > > When it > > > > > > > > > gets there, name > > > > > > > > > resolution should allow the message > > to be > > > > > > > forwarded > > > > > > > > > to the correct queue > > > > > > > > > on QM1. > > > > > > > > > > > > > > > > > > I haven't tested this design to > ensure > > that it > > > > > > > > > works, but I think it > > > > > > > > > would. It does not support load > balancing > > across > > > > > > > > > several possible > > > > > > > > > destinations. > > > > > > > > > > > > > > > > > > I would implement this with 2 clusters, > > not with > > > > > > > a > > > > > > > > > single cluster which is > > > > > > > > > somewhat broken. The 2 cluster > design > > does > > > > > > > support > > > > > > > > > load balancing. > > > > > > > > > > > > > > > > > > The 2 cluster design would have > QM1 > > and QM2 in > > > > > > > > > CLUS1, and QM2 and QM3 in > > > > > > > > > CLUS2. > > > > > > > > > > > > > > > > > > The queue definitions to enable > access > > from QM3 > > > > > > > to > > > > > > > > > QM1 via the gateway > > > > > > > > > would be... > > > > > > > > > on QM1 define QL(TQ1) cluster(CLUS1) > > > > > > > > > defbind(notfixed) > > > > > > > > > on QM2 define QA(TQ1.VIA.CLUS2) > targq(TQ1) > > > > > > > > > cluster(CLUS2) > > > > > > > > > defbind(notfixed) > > > > > > > > > on QM3 define QA(TQ1) targq(TQ1.VIA.CLUS2) > > > > > > > > > defbind(notfixed) cluster(' ') > > > > > > > > > > > > > > > > > > QM3 can now put to TQ1 and the > message > > will > > > > > > > traverse > > > > > > > > > the network via QM2 > > > > > > > > > to QM1. > > > > > > > > > > > > > > > > > > The reason for changing the name > of > > the queue on > > > > > > > the > > > > > > > > > gateway queue manager > > > > > > > > > is so that you can have more than > one > > gateway. > > > > > > > We > > > > > > > > > can enhance the design > > > > > > > > > above by adding QM4, which is in > CLUS1 > > and > > > > > > > CLUS2. It > > > > > > > > > has QA(TQ1.VIA.CLUS2) > > > > > > > > > with exactly the same definition > as > > on QM2. This > > > > > > > > > cluster gateway design > > > > > > > > > will now continue to operate, even > if > > one of QM2 > > > > > > > or > > > > > > > > > QM4 are unavailable, > > > > > > > > > > > > > === message truncated === > > > > > > > > > > > > To unsubscribe, write to [EMAIL PROTECTED] > > and, > > > > > > in the message body (not the subject), write: > SIGNOFF > > MQSERIES > > > > > > Instructions for managing your mailing list subscription > > are provided > > > > in > > > > > > the Listserv General Users Guide available at > http://www.lsoft.com > > > > > > Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html > > > > > > > > > > -- > > > > > Der GMX SmartSurfer hilft bis zu 70% Ihrer Onlinekosten > > zu sparen! > > > > > Ideal für Modem und ISDN: http://www.gmx.net/de/go/smartsurfer > > > > > > > > > > To unsubscribe, write to [EMAIL PROTECTED] > > and, > > > > > in the message body (not the subject), write: SIGNOFF > MQSERIES > > > > > Instructions for managing your mailing list subscription > > are provided in > > > > > the Listserv General Users Guide available at http://www.lsoft.com > > > > > Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html > > > > > > > > > > > > > To unsubscribe, write to [EMAIL PROTECTED] > and, > > > > in the message body (not the subject), write: SIGNOFF MQSERIES > > > > Instructions for managing your mailing list subscription > are > > provided in > > > > the Listserv General Users Guide available at http://www.lsoft.com > > > > Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html > > > > > > -- > > > Der GMX SmartSurfer hilft bis zu 70% Ihrer Onlinekosten zu sparen! > > > > > Ideal für Modem und ISDN: http://www.gmx.net/de/go/smartsurfer > > > > > > To unsubscribe, write to [EMAIL PROTECTED] and, > > > in the message body (not the subject), write: SIGNOFF MQSERIES > > > Instructions for managing your mailing list subscription are > provided > > in > > > the Listserv General Users Guide available at http://www.lsoft.com > > > Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html > > > > > > > To unsubscribe, write to [EMAIL PROTECTED] and, > > in the message body (not the subject), write: SIGNOFF MQSERIES > > Instructions for managing your mailing list subscription are provided > in > > the Listserv General Users Guide available at http://www.lsoft.com > > Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html > > > > > > > > ----------------------------------------------------------------- > > List Archive - > > Manage Your List Settings - > > Unsubscribe > > > > > > Instructions for managing your mailing list subscription are provided > in the Listserv General Users Guide available at http://www.lsoft.com > > > > To unsubscribe, write to [EMAIL PROTECTED] and, > in the message body (not the subject), write: SIGNOFF MQSERIES > Instructions for managing your mailing list subscription are provided in > the Listserv General Users Guide available at http://www.lsoft.com > Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html > > > > ----------------------------------------------------------------- > List Archive - > Manage Your List Settings - > Unsubscribe > > > Instructions for managing your mailing list subscription are provided in the > Listserv General Users Guide available at http://www.lsoft.com > -- Hubert Kleinmanns Beratung / Schulung / Projektleitung Tel.: +49 (0) 60 78 / 7 12 21 Fax: +49 (0) 60 78 / 7 12 25 Mobil: +49 (0) 178 / 6 97 22 54 To unsubscribe, write to [EMAIL PROTECTED] and, in the message body (not the subject), write: SIGNOFF MQSERIES Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://listserv.meduniwien.ac.at/archives/mqser-l.html
