Bennie, Just FYI, JeroMQ is written with Java NIO as much as possible and has no native code.
Thanks Min On Oct 11, 2013, at 12:56 PM, Bennie Kloosteman <[email protected]> wrote: > Yes compared to native langauges all GC languages incur extra IO costs eg > When writing both languages they need to read/copy the data to the kernel , > but the GC language needs to tell the GC to pin each buffer so it doesnt > reloccate it ,this can be significant for lots of small packets. > > GC pauses can be a huge issue i once had a store and forward message system > which in stress testing ended up having 10 second GC pauses on a 800Meg heap > which caused the system to stop reading incomming packets which then gave > timeout errors . Many get around this by storing the message outside the GC ( > in native or in serialized queues of byte[] ) and then getting the message > when you want. > > GC languages also have a card table cost for wrting to a message. > > Note pure Java solutions like Java NIO may be better than JeroMQ , there was > a discussion on this earlier . I dont know JeroMQs architecture but pure > solutions tend to consider the GC issues rather than trivially wrapping the > API , they reuse byte[] buffers to copy from and to the kernel , and may even > employ unsafe / JNI code to self manage some buffers so you dont need to > pin the GC .. . > > Again except for the GC pauses these issues are only for micro benches and > dwarfed by the cost of any real work / IO. > > My suggestion, build your real work and use a flexible comms API , use the > easiest comms possible ( which would prob be a a Java lib so you dont worry > about deploying zeromq ) then test the whole end to end process if the comms > is too slow then test zeromq and NIO ( which should be easy since you put it > behind an interface) but not before. > > Ben > > > On Fri, Oct 11, 2013 at 9:10 AM, crocket <[email protected]> wrote: > Does it mean JeroMQ incurs marshalling when communicating to the kernel? > > > On Fri, Oct 11, 2013 at 12:21 AM, Bennie Kloosteman <[email protected]> > wrote: > Benchmark it with some real work. > > Marshalling to native is expensive when doing micro benches with no work but > not compared to going over the wire or messages that do real work , > Serialization cost will normally be higher than marshalling to native . If > your doing inter process via shared memory / pipes on the same machine and > dont do anything with the message it may well be more expensive. On the > other hand for large scale solutions where your messages stay in a queue > for a while you can leave the message in native and not fill the GC and > suffer GC pauses. > > Note the JVM also incurs marshaling when communicating to the kernel say for > tcp/ip which zeroMq does not . > > Ben > > > On Thu, Oct 10, 2013 at 9:22 PM, crocket <[email protected]> wrote: > Someone pointed out that ZeroMQ was great at communicaton between two or more > languages but that for communications in one language, internal messaging > solutions like akka and clojure.core.async would be a lot faster. > > He said marshalling was expensive. > > I looked into core.async, and I found it uses queues for asynchronicity. > > How would one compare core.async and ZeroMQ? > > _______________________________________________ > zeromq-dev mailing list > [email protected] > http://lists.zeromq.org/mailman/listinfo/zeromq-dev > > > > _______________________________________________ > zeromq-dev mailing list > [email protected] > http://lists.zeromq.org/mailman/listinfo/zeromq-dev > > > > _______________________________________________ > zeromq-dev mailing list > [email protected] > http://lists.zeromq.org/mailman/listinfo/zeromq-dev > > > _______________________________________________ > zeromq-dev mailing list > [email protected] > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
_______________________________________________ zeromq-dev mailing list [email protected] http://lists.zeromq.org/mailman/listinfo/zeromq-dev
