Sorry I have not been on top of the mailing lists like I need to be. In the newer versions of storm zeromq support has been removed, and if you want to use it you need to import it from an external source for example https://github.com/ptgoetz/storm-0mq. That repository is for storm-0.9.0.1 which is a version prior to the changes in question. As part of those changes the APIs for IConnection have changed, and so storm-0mq is no longer compatible with storm. I¹m not sure what if any plans there are to support storm-0mq for newer versions of storm, but if it were updated, it would need to support the batching internally.
- Bobby On 8/25/14, 4:27 PM, "Sean Allen" <[email protected]> wrote: >Bobby, >I just realized that I was rather imprecise on that question. Sorry about >that. > >When you say, "coming from the client side pre batched". I see that with >the netty client but, if you are using 0mq transport, is there batching >going on? >My initial interpretation was you mean any client but I realize now that >you might have meant just the netty client. > >-Sean- > > > >On Mon, Aug 25, 2014 at 5:20 PM, Sean Allen <[email protected]> >wrote: > >> No problems based on it. I'm trying to understand what changed in the >> receive code with the latest release. >> >> Can you point me to where in the newer code the messaging layer is doing >> the batching? >> >> >> >> On Mon, Aug 25, 2014 at 2:56 PM, Bobby Evans >><[email protected]> >> wrote: >> >>> Sean, >>> >>> I think it was an oversight leaving the parameter in place still. >>> Previously the code would take the messages and pull them out of the >>> messaging layer putting them in a batch to be processed. In the newer >>>code >>> the messaging layer is already doing the batching. In fact the >>>messages >>> are coming from the client side pre batched. Are you running into >>> performance or other problems with this change? If not please file a >>>JIRA >>> for this. We should probably remove the parameter completely and either >>> remove or at a minimum deprecate Config.TOPOLOGY_RECEIVER_BUFFER_SIZE. >>> >>> - Bobby >>> >>> On 8/24/14, 10:05 AM, "Sean Allen" <[email protected]> wrote: >>> >>> >It would appear that as of this commit: >>> > >>> >>>https://github.com/apache/incubator-storm/commit/861a92eab8740cfc0f83ac4 >>>d7 >>> >ade9a2ab04a8b9b >>> > >>> >That changing topology.receiver.buffer.size has no effect. >>> > >>> >mk-receive-thread is loader.clj still accepts max-buffer-size as an >>>input >>> >but the value isn't used within the function. >>> > >>> >I'm not sure if this was intentional additional code needs to be >>>updated >>> >to >>> >reflect the lack of the option or if it a bug in the mk-receive-thread >>> >code >>> >that it has no effect. >>> >>> >> >> >> -- >> >> Ce n'est pas une signature >> > > > >-- > >Ce n'est pas une signature
