I agree. Can you suggest any better tuning for ActiveMQ to make Ruby STOMP consumer to consume message more fast. Right now it consumes around 2000 messages per 1-2mins of time, but am looking consumer where it can receive around 1000 messages in seconds.
On Thursday, January 25, 2018 at 12:29:46 AM UTC+5:30, R.I.Pienaar wrote: > > > On 24 Jan 2018, at 19:39, [email protected] <javascript:> wrote: > > My bad :( > > If you don't mind can you please clarify below queries. > > If I use either of OpenSSL or AES security then can't it be possible to > decode it in python? > > > I guess can’t say I tried, with choria you would not have a problem. > > Is there any asynchronous ruby stomp consumer available like Python > Stompest(which will consume n number of messages parallel)? Since I need to > use reply-to with the mco commands and so many mco commands will keep > executed, so I need a very facts consumer which can consume message very > quickly. > > > Normal ruby stomp gem does this fine. You have some activemq problems > which is why it doesn’t work. you far exceed what is possible with it and > refuse suggestions to use something better > > With NATS you can run 100 000 nodes on a small VM > > > > > > On Thursday, January 25, 2018 at 12:00:17 AM UTC+5:30, R.I.Pienaar wrote: > >> >> >> On 24 Jan 2018, at 19:11, [email protected] wrote: >> >> Am not using any security plugin, >> >> securityprovider = psk >> plugin.psk = unset >> >> >> >> Then you are both deeply irresponsible and out of luck - you have a giant >> unauthenticated hugely insecure distributed root shell. Helping you further >> is just plain irresponsible. >> >> That plugin uses a ruby specific encoding format. You can’t decode it in >> python. >> >> >> >> >> >> >> On Wednesday, January 24, 2018 at 11:32:52 PM UTC+5:30, R.I.Pienaar wrote: >> >>> yes, you need body and decode that >>> >>> what security plugin do you use? >>> >>> >>> On Wed, 24 Jan 2018, at 18:38, [email protected] wrote: >>> > Ok managed to process little bit but struggling with the format, >>> > >>> > Looks like the Stompest consumer frame is returning the data in dict >>> format >>> > as below, >>> > >>> > {'command': 'MESSAGE', 'body': >>> > b'\x04\x08{\x0b:\tbody"n\x04\x08{\x08:\tdata{\n:\x0bstdout"\x14output- >>> > data\n:\rexitcodei >>> > \x00:\x0bstderr"\x00:\x0boutput0:\nerror0:\x0estatusmsg"\x07OK: >>> > \x0fstatuscodei >>> > \x00:\rsenderid"\x13server1:\thash"%903ff3bf7e92121051fd6sd23d8f718a: >>> > \x10senderagent"\nshell:\x0cmsgtimel+\x07\x85\xc2hZ: >>> > \x0erequestid"%c6c257193d805de2sd2336e5bea9f87e', >>> > 'rawHeaders': [('content-length', '264'), ('expires', >>> '1516815051335'), >>> > ('destination', '/queue/mcollective.queue'), ('priority', '4'), >>> > ('message-id', 'ID:activemqserver-45532-1516365445248-3:49:-1:1:73'), >>> > ('content-type', 'text/plain; charset=UTF-8'), ('timestamp', >>> > '1516814981335')]} >>> > >>> > If I see the value for the key body in the above dict, looks like not >>> the >>> > perform format of Hash and if you see at the end of body value the >>> closed >>> > curly '}' is missing. But not sure if its missing or its not showing >>> due to >>> > the encryption. Would you please help me to understand why it comes >>> like >>> > this? >>> > >>> > >>> > On Sunday, January 21, 2018 at 11:38:09 PM UTC+5:30, R.I.Pienaar >>> wrote: >>> > >>> > > It’s probably not JSON. >>> > > >>> > > Did you print what comes from the base64? >>> > > >>> > > --- >>> > > R.I.Pienaar >>> > > >>> > > On 21 Jan 2018, at 16:23, [email protected] <javascript:> wrote: >>> > > >>> > > I tried with the method which you suggested and getting below error >>> when >>> > > consumer is trying to consume the message. >>> > > >>> > > 'utf-8' codec can't decode byte 0x9a in position 0: invalid start >>> byte >>> > > >>> > > Any idea please. >>> > > >>> > > >>> > > On Sunday, January 21, 2018 at 6:52:57 PM UTC+5:30, >>> [email protected] >>> > > wrote: >>> > > >>> > >> The data is not corrupted, since I was able to get the data using >>> Ruby by >>> > >> importing mcollective and using native mcollective decode. >>> > >> >>> > >> And this what the code I tried in Python, >>> > >> >>> > >> def run(self): >>> > >> client = Stomp(self.config) >>> > >> yield client.connect() >>> > >> headers = {StompSpec.ACK_HEADER: >>> StompSpec.ACK_CLIENT_INDIVIDUAL, 'activemq.prefetchSize': '10000'} >>> > >> client.subscribe(self.QUEUE, headers, >>> > >> listener=SubscriptionListener(self.consume, >>> errorDestination=self.ERROR_QUEUE)) >>> > >> >>> > >> >>> > >> def consume(self, client, frame): >>> > >> data = json.loads(base64.b64decode(frame.body)) >>> > >> print(data) >>> > >> >>> > >> >>> > >> >>> > >> >>> > >> >>> > >> On Sunday, January 21, 2018 at 4:33:24 PM UTC+5:30, R.I.Pienaar >>> wrote: >>> > >> >>> > >>> >>> > >>> >>> > >>> On Sun, 21 Jan 2018, at 10:50, [email protected] wrote: >>> > >>> > Hi, >>> > >>> > >>> > >>> > Am trying to use different consumer which can consume messages >>> which >>> > >>> is >>> > >>> > being called with reply-to option and the consumer which am >>> trying to >>> > >>> use >>> > >>> > is stompest (https://github.com/nikipore/stompest) which is >>> Python >>> > >>> based. >>> > >>> > Am able to configure this and its receiving the message but >>> unable to >>> > >>> > decode the message using base64. Am not using any security >>> plugin in >>> > >>> my >>> > >>> > mcollective environment so I hope MCollective does plain base64 >>> > >>> encoding >>> > >>> > only, so with that I tried to decode the message in Python using >>> same >>> > >>> > base64 but its not working. Sometimes getting "Incorrect >>> Padding" >>> > >>> issue and >>> > >>> > sometimes byte related errors are appearing. So looking proper >>> way to >>> > >>> > decode the mcollective message using python and get the message >>> > >>> payload >>> > >>> > data. >>> > >>> > >>> > >>> >>> > >>> padding error means some data at the end is missing - often normal >>> - >>> > >>> base64 data has to be dividable by 4, if not add "=" at the end >>> till it is. >>> > >>> Then it should work >>> > >>> >>> > >>> not sure what happens that your data is corrupt though >>> > >>> >>> > >>> -- >>> > >>> R.I.Pienaar / www.devco.net / @ripienaar >>> > >>> >>> > >> -- >>> > > >>> > > --- >>> > > You received this message because you are subscribed to the Google >>> Groups >>> > > "mcollective-users" group. >>> > > To unsubscribe from this group and stop receiving emails from it, >>> send an >>> > > email to [email protected] <javascript:>. >>> > > For more options, visit https://groups.google.com/d/optout. >>> > > >>> > > >>> > >>> > -- >>> > >>> > --- >>> > You received this message because you are subscribed to the Google >>> > Groups "mcollective-users" group. >>> > To unsubscribe from this group and stop receiving emails from it, send >>> > an email to [email protected]. >>> > For more options, visit https://groups.google.com/d/optout. >>> >>> >>> -- >>> R.I.Pienaar / www.devco.net / @ripienaar >>> >> -- >> >> --- >> You received this message because you are subscribed to the Google Groups >> "mcollective-users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> For more options, visit https://groups.google.com/d/optout. >> >> -- > > --- > You received this message because you are subscribed to the Google Groups > "mcollective-users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] <javascript:>. > For more options, visit https://groups.google.com/d/optout. > > -- --- You received this message because you are subscribed to the Google Groups "mcollective-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
