It would be good if I get true batch mode where all requests have the same
ID, since my consumer segregates the response output based on the ID only.
The reason why I want to go with batch mode is, I have some set of servers
where I need to collect some data. But in this the data and also the
servers list will keep vary which comes from different subcollectives. So I
don't have specific facts filter to run in broadcast mode and also I can't
make such facts also. Its basically kind of requirement for us to run this
often.
On Sunday, January 28, 2018 at 9:14:52 PM UTC+5:30, R.I.Pienaar wrote:
>
>
> On Sun, 28 Jan 2018, at 15:29, kk21987 wrote:
> > Yes please. If the reply-to works well with batch mode it would be great
> > for me as it will solve my proble :)
>
> yeah fire and forget doesnt support batch mode in the normal client
>
> in your use case does it matter if the replies all have the same request
> id - ie. do you want a true batch mode where all requests have the same ID
> or many seperate requests would be fine?
>
> I still am curious why you need to use direct mode rather than broadcast
> >
> > On Sunday, January 28, 2018 at 6:23:35 PM UTC+5:30, R.I.Pienaar wrote:
> > >
> > >
> > > On 28 Jan 2018, at 13:31, kk21987 <kkarth...@gmail.com <javascript:>>
> > > wrote:
> > >
> > > Yeah I agree it doesn't make sense to run without batch in direct
> address
> > > mode. But there is one problem using batch with reply-to.
> > >
> > > I have set the export for MCOLLECTIVE_EXTRA_OPTS with the values batch
> > > size as 200 and batch sleep as 30 and when I run mco query with
> --reply-to
> > > its returning the result on the console of mco client server itself
> which I
> > > don't want. Since I have separate consumer server configured for this.
> > >
> > > I really don't understand why it behaves like this. Any idea please.
> > >
> > >
> > > Could be batch mode just isn’t supported in the library when using
> reply
> > > to. I can look later when at a computer again
> > >
> > > Would be trivial to write a small custom application that does this
> > > though.
> > >
> > >
> > > On Sunday, January 28, 2018 at 3:46:04 PM UTC+5:30, R.I.Pienaar wrote:
> > >
> > >>
> > >>
> > >> On 28 Jan 2018, at 11:13, kk21987 <kkarth...@gmail.com> wrote:
> > >>
> > >> Understood and thanks for the detailed clarification! I have already
> > >> splitted as different sub collectives (2000 per subcollective) but
> again
> > >> all the subcollectives are connected with Broker-A which will act as
> > >> Central(just for naming convention) and this central will get big
> pressure
> > >> when I run direct addressing against huge number of servers. So is
> there
> > >> any feasibility to connect all subcollectives ActiveMQ brokers and
> run the
> > >> command instead of using Central one by declaring some parameter in
> > >> mcollective client config or using any other tool in between? I think
> which
> > >> this way it will go smooth, since I see mcollective publish the
> message
> > >> very quickly to Broker-A(central) and as you said this is getting big
> > >> pressure.
> > >>
> > >>
> > >> Why do you need to use direct addressing for your use case? If your
> node
> > >> lists arbitrary and not something you can discover against?
> > >>
> > >> You don’t use any of the confirmation features it brings?
> > >>
> > >> Only other option at this scale might be to direct address + batches
> > >>
> > >>
> > >> On Sunday, January 28, 2018 at 3:35:01 PM UTC+5:30, R.I.Pienaar
> wrote:
> > >>>
> > >>>
> > >>>
> > >>> On 28 Jan 2018, at 10:48, kk21987 <kkarth...@gmail.com> wrote:
> > >>>
> > >>> I agree. I read about the Choria and very interested to use it. But
> am
> > >>> running little old version of Ruby, Puppet where I need to plan it
> for
> > >>> upgrade. But still Choria is in active dev am little concern to go
> ahead it
> > >>> on Production.
> > >>>
> > >>> Between I dig into furthermore and found its issue with direct
> > >>> addressing mode only. When I run mco running with -T mcollective or
> > >>> whatever subcollectives it goes very smooth, but when I run mco
> query using
> > >>> flatfile or -I(direct addressing) against high number of servers
> only then
> > >>> the issue occurs.
> > >>>
> > >>> So I suspect direct addressing takes time to execute and would
> curious
> > >>> to understand why it gives such difference. I believe direct
> addressing
> > >>> should be fast compare with broadcast, correct me if am wrong.
> > >>>
> > >>>
> > >>> It’s a trade off because direct has to create and distribute a
> single
> > >>> message per every target. So if you have 10k targets it’s that many
> > >>> messages. Big pressure on activemq especially with the single queue
> and
> > >>> processing the per node selectors
> > >>>
> > >>> Broadcast in turn is much lighter on the middleware and much faster
> all
> > >>> round but you don’t know which nodes will get it
> > >>>
> > >>> UDP cheap and fast because it’s not guaranteed
> > >>> TCP expensive and slow with many layers but guaranteed delivery (or
> at
> > >>> least advisory about non delivery ack)
> > >>>
> > >>> Likewise with mco broadcast and direct.
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>
> > >>> On Sunday, January 28, 2018 at 3:11:34 PM UTC+5:30, R.I.Pienaar
> wrote:
> > >>>
> > >>>>
> > >>>>
> > >>>> On 28 Jan 2018, at 10:35, kk21987 <kkarth...@gmail.com> wrote:
> > >>>>
> > >>>> Some improvement with your debug help! Looks like the reply-to is
> > >>>> working perfect as expected. The actual problem is triggering the
> command
> > >>>> to all the servers.
> > >>>>
> > >>>> I have total of 9824 servers connected with mcollective.nodes and
> when
> > >>>> I run mco query against all the servers to execute some command
> then
> > >>>> dispatching that mco query message all the consumers takes time
> which is
> > >>>> why the pending messages are there and whenever the each consumers
> receives
> > >>>> the message its slowly decreasing.
> > >>>>
> > >>>>
> > >>>> Great
> > >>>>
> > >>>>
> > >>>>
> > >>>> Now I need to look for some tuning to get the message dispatched to
> all
> > >>>> the consumer very quickly, so it will be good. Any suggestion or
> best
> > >>>> tuning parameters available for this. I will try to look for
> ActiveMQ
> > >>>> documentation to get the something.
> > >>>>
> > >>>>
> > >>>> No I gave up making activemq scale this far. I have some large
> rabbitmq
> > >>>> clusters but that requires a ton of hardware too
> > >>>>
> > >>>> Choria with NATS or the Choria broker would do this comfortably on
> a
> > >>>> single small 4GB VM with a tremendous speed improvement.
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>> On Sunday, January 28, 2018 at 2:03:34 PM UTC+5:30, kk21987 wrote:
> > >>>>
> > >>>>> Thanks.
> > >>>>>
> > >>>>> Yes I captured the client debug log as well and could see that it
> > >>>>> publish message with reply-to properly.
> > >>>>>
> > >>>>> Direct Addressing:
> > >>>>>
> > >>>>> D, [2018-01-28T02:24:56.851292 #17287] DEBUG -- :
> activemq.rb:402:in
> > >>>>> `publish' Sending a direct message to ActiveMQ target
> > >>>>> '/queue/mcollective.nodes' with headers
> '{"mc_identity"=>"mco-server-name",
> > >>>>> "timestamp"=>"1517127896000", "expires"=>"1517127966000",
> > >>>>> "reply-to"=>"/queue/data.consumer.all"}'
> > >>>>>
> > >>>>> Broadcast:
> > >>>>>
> > >>>>> D, [2018-01-28T02:25:19.345299 #17340] DEBUG -- :
> activemq.rb:409:in
> > >>>>> `publish' Sending a broadcast message to ActiveMQ target
> > >>>>> '/topic/mcollective.shell.agent' with headers
> '{"expires"=>"1517127989000",
> > >>>>> "reply-to"=>"/queue/data.consumer.all",
> "timestamp"=>"1517127919000"}'
> > >>>>>
> > >>>>> Here is the way I used to exclude the queue consumer destination
> on
> > >>>>> Broker-A.
> > >>>>>
> > >>>>> <queue physicalName="data.consumer.>"/>
> > >>>>>
> > >>>>> I couldn't take the debug logs on ActiveMQ due to the servers
> count is
> > >>>>> little high, so it generate huge logs and couldn't get it exactly.
> But I
> > >>>>> used ActiveMQ-Cli to export the JMS queue and here is a example of
> one
> > >>>>> pending message
> > >>>>>
> > >>>>>
> > >>>>> <jms-message>
> > >>>>> <header>
> > >>>>>
> > >>>>>
> <message-id>ID:broker-A-16618-1517126209254-79:5:-1:1:559</message-id>
> > >>>>> <delivery-mode>1</delivery-mode>
> > >>>>> <destination>queue://mcollective.nodes</destination>
> > >>>>> <expiration>1517127457000</expiration>
> > >>>>> <priority>4</priority>
> > >>>>> <redelivered>false</redelivered>
> > >>>>> <reply-to>queue://data.consumer.all</reply-to>
> > >>>>> <timestamp>2018-01-28T02:16:27</timestamp>
> > >>>>> </header>
> > >>>>> <properties>
> > >>>>> <property>
> > >>>>> <name>content-type</name>
> > >>>>> <value>text/plain; charset=UTF-8</value>
> > >>>>> </property>
> > >>>>> <property>
> > >>>>> <name>mc_identity</name>
> > >>>>> <value>mcollective-server-name</value>
> > >>>>> </property>
> > >>>>> </properties>
> > >>>>> </jms-message>
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>> On Sunday, January 28, 2018 at 12:40:14 PM UTC+5:30, R.I.Pienaar
> wrote:
> > >>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> On 28 Jan 2018, at 06:06, kk21987 <kkarth...@gmail.com> wrote:
> > >>>>>>
> > >>>>>> My apologize. I misunderstood few things.
> > >>>>>>
> > >>>>>> Basically when I run mco query in direct addressing mode then it
> > >>>>>> publish that message as queue where as broadcast mode publish the
> message
> > >>>>>> as topic. So I suspect its not an issue with the modes whatever
> the mco
> > >>>>>> query use. The only thing I want to understand is, lets say I run
> mco query
> > >>>>>> against 1000servers with --reply-to (fire and forget, no
> acknowledgement
> > >>>>>> and nothing. Correct me if am wrong), so all the response message
> will go
> > >>>>>> to the consumer where the active subscriber is present for the
> given queue.
> > >>>>>> The response message is not coming to Broker-A which I have
> configured by
> > >>>>>> starting receiver here as well and perfectly its going to the
> consumer
> > >>>>>> where its active.
> > >>>>>>
> > >>>>>>
> > >>>>>> That sounds about right. Yes it will use a topic and all nodes
> will
> > >>>>>> get it
> > >>>>>>
> > >>>>>>
> > >>>>>> My concern is, at the same time I see count is showing in pending
> > >>>>>> message for the queue "mcollective.nodes" and slowly its
> decreasing(at the
> > >>>>>> same enqueued and dequeued messages count also increasing slowly)
> which I
> > >>>>>> have verified on ActiveMQ UI Jolokia monitoring (8161). So I
> would like to
> > >>>>>> understand why such messages count in pending messages, since all
> I want to
> > >>>>>> just fire the command and forget and don't want to worry about
> response or
> > >>>>>> whether its executed or not.
> > >>>>>>
> > >>>>>> Name ↑Number Of Pending Messages Number Of Consumers Messages
> > >>>>>> Enqueued Messages Dequeued Views Operations
> > >>>>>> mcollective.nodes 9 84 884 875 Browse Active Consumers
> > >>>>>> Active Producers
> > >>>>>> Send To PurgeDelete
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> You will need to double check in the logs. Your client also logs
> and
> > >>>>>> at debug
> > >>>>>> level you can verify where the publish to
> > >>>>>>
> > >>>>>>
> > >>>>>> On Sunday, January 28, 2018 at 7:48:15 AM UTC+5:30, kk21987
> wrote:
> > >>>>>>
> > >>>>>>> Based on my understanding when I run mco query with reply-to I
> > >>>>>>> should see the reply-to queue name on the response header and
> its there on
> > >>>>>>> mcollective server logs, which I have verified by enabling debug
> on the
> > >>>>>>> mcollective servers.
> > >>>>>>>
> > >>>>>>> My worry is, I shouldn't get any response messages back to the
> > >>>>>>> Broker-A and all the response messages should go to the consumer
> server
> > >>>>>>> which is running elsewhere. This is working as expected for
> broadcast based
> > >>>>>>> mco query but when I run direct addressing based mco query then
> the
> > >>>>>>> response message is flowing everywhere instead of going to
> consumer server
> > >>>>>>> only. I have configured the exclude destination on Broker-A to
> make sure
> > >>>>>>> none of the brokers are connected with the reply-to queue name
> except the
> > >>>>>>> consumer server broker.
> > >>>>>>>
> > >>>>>>> On Sunday, January 28, 2018 at 2:08:06 AM UTC+5:30, R.I.Pienaar
> > >>>>>>> wrote:
> > >>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> On Sat, 27 Jan 2018, at 15:03, kk21987 wrote:
> > >>>>>>>> > Hi,
> > >>>>>>>> >
> > >>>>>>>> > Am having some issue with --reply-to behavior and not sure if
> its
> > >>>>>>>> an
> > >>>>>>>> > issue
> > >>>>>>>> > or the behavior itself like that. My setup is drawn in
> > >>>>>>>> > ......
> > >>>>>>>> >
> > >>>>>>>> >
> > >>>>>>>> > My environment is enabled with direct_addressing. Basically I
> > >>>>>>>> always fire
> > >>>>>>>> > the mco and forget it and don't want to consume any messages
> on
> > >>>>>>>> mco client.
> > >>>>>>>> > I have configured separate consumer server where it will
> consume
> > >>>>>>>> the
> > >>>>>>>> > messages on the queue whatever declared during the mco query
> and
> > >>>>>>>> will get
> > >>>>>>>> > result from thie consumer server. The reason for configured
> > >>>>>>>> separate
> > >>>>>>>> > consumer server is some commands whatever I run will take
> more
> > >>>>>>>> time to
> > >>>>>>>> > execute and also doing some logic on the response messages
> based
> > >>>>>>>> on my
> > >>>>>>>> > requirement.
> > >>>>>>>> >
> > >>>>>>>> > I have configured exclude destination to make sure the queue
> > >>>>>>>> whatever am
> > >>>>>>>> > using in reply-to not connected with Broker-A from other
> brokers.
> > >>>>>>>> >
> > >>>>>>>> > When I run mco query using regex filter it pubish message in
> > >>>>>>>> broadcast mode
> > >>>>>>>> > and consumer server consume the response without any issue.
> At
> > >>>>>>>> the same
> > >>>>>>>> > time when I check on the Broker-A ActiveMQ UI monitoring I
> > >>>>>>>> noticed Messages
> > >>>>>>>> > Enqueued Dequeued count is NOT increased which means nothing
> > >>>>>>>> coming back to
> > >>>>>>>> > Broker-A.
> > >>>>>>>> >
> > >>>>>>>> > But when I run mco query using direct addressing mode (-I)
> along
> > >>>>>>>> with
> > >>>>>>>> > reply-to the message is published as direct addressing and
> > >>>>>>>> consumer server
> > >>>>>>>> > consume the response fine here as well. At the same time when
> I
> > >>>>>>>> check on
> > >>>>>>>> > the Broker-A ActiveMQ UI monitoring I noticed Messages
> Enqueued
> > >>>>>>>> Dequeued
> > >>>>>>>> > count is increased which means message is coming to Broker-A
> as
> > >>>>>>>> well at the
> > >>>>>>>> > same time message is available on consumer server.
> > >>>>>>>> >
> > >>>>>>>> > Am not sure such difference between direct addressing and
> > >>>>>>>> broadcast even
> > >>>>>>>> > though both use the same reply-to mechanism. I don't want any
> > >>>>>>>> response
> > >>>>>>>> > messages come back to Broker-A and all the response messages
> > >>>>>>>> should go to
> > >>>>>>>> > Consumer Server only. Can I have some lights on this please?
> > >>>>>>>>
> > >>>>>>>> what you describe should work, I guess the best hints lie in
> the
> > >>>>>>>> server debug log. Does it send a reply and does it do so to
> the right
> > >>>>>>>> target?
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> --
> > >>>>>>>> R.I.Pienaar / www.devco.net / @ripienaar
> > >>>>>>>>
> > >>>>>>> --
> > >>>>>>
> > >>>>>> ---
> > >>>>>> You received this message because you are subscribed to the
> Google
> > >>>>>> Groups "mcollective-users" group.
> > >>>>>> To unsubscribe from this group and stop receiving emails from it,
> > >>>>>> send an email to mcollective-us...@googlegroups.com.
> > >>>>>> For more options, visit https://groups.google.com/d/optout.
> > >>>>>>
> > >>>>>> --
> > >>>>
> > >>>> ---
> > >>>> You received this message because you are subscribed to the Google
> > >>>> Groups "mcollective-users" group.
> > >>>> To unsubscribe from this group and stop receiving emails from it,
> send
> > >>>> an email to mcollective-us...@googlegroups.com.
> > >>>> For more options, visit https://groups.google.com/d/optout.
> > >>>>
> > >>>> --
> > >>>
> > >>> ---
> > >>> You received this message because you are subscribed to the Google
> > >>> Groups "mcollective-users" group.
> > >>> To unsubscribe from this group and stop receiving emails from it,
> send
> > >>> an email to mcollective-us...@googlegroups.com.
> > >>> For more options, visit https://groups.google.com/d/optout.
> > >>>
> > >>> --
> > >>
> > >> ---
> > >> You received this message because you are subscribed to the Google
> Groups
> > >> "mcollective-users" group.
> > >> To unsubscribe from this group and stop receiving emails from it,
> send an
> > >> email to mcollective-us...@googlegroups.com.
> > >> For more options, visit https://groups.google.com/d/optout.
> > >>
> > >> --
> > >
> > > ---
> > > You received this message because you are subscribed to the Google
> Groups
> > > "mcollective-users" group.
> > > To unsubscribe from this group and stop receiving emails from it, send
> an
> > > email to mcollective-us...@googlegroups.com <javascript:>.
> > > For more options, visit https://groups.google.com/d/optout.
> > >
> > >
> >
> > --
> >
> > ---
> > You received this message because you are subscribed to the Google
> > Groups "mcollective-users" group.
> > To unsubscribe from this group and stop receiving emails from it, send
> > an email to mcollective-us...@googlegroups.com <javascript:>.
> > For more options, visit https://groups.google.com/d/optout.
>
>
> --
> R.I.Pienaar / www.devco.net / @ripienaar
>
--
---
You received this message because you are subscribed to the Google Groups
"mcollective-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to mcollective-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
