Re: [zeromq-dev] Does ZeroMQ pay a lot in terms of marshalling compared to akka and clojure.core.async?

2013-10-10 Thread Bennie Kloosteman
Benchmark it with some real work. Marshalling to native is expensive when doing micro benches with no work but not compared to going over the wire or messages that do real work , Serialization cost will normally be higher than marshalling to native . If your doing inter process via shared memory

Re: [zeromq-dev] Does ZeroMQ pay a lot in terms of marshalling compared to akka and clojure.core.async?

2013-10-10 Thread Bennie Kloosteman
:21 AM, Bennie Kloosteman bkloo...@gmail.comwrote: Benchmark it with some real work. Marshalling to native is expensive when doing micro benches with no work but not compared to going over the wire or messages that do real work , Serialization cost will normally be higher than marshalling

Re: [zeromq-dev] Does ZeroMQ pay a lot in terms of marshalling compared to akka and clojure.core.async?

2013-10-10 Thread Bennie Kloosteman
to copy from and to the kernel , and may even employ unsafe / JNI code to self manage some buffers so you dont need to pin the GC .. . Ben On Fri, Oct 11, 2013 at 12:46 PM, Tony Arcieri basc...@gmail.com wrote: On Thu, Oct 10, 2013 at 8:56 PM, Bennie Kloosteman bkloo...@gmail.comwrote

Re: [zeromq-dev] tcp vs zeromq

2013-07-12 Thread Bennie Kloosteman
If you run it without a network stack zeromq can be very fast so its not so much the protocol. You can see this in the tests but note in those cases zeromq is using C++. For lots of small messages 100 bytes zeroMq does need a lot more cpu than tcp but that is not a common scenario. There is

Re: [zeromq-dev] RFC 32 - Z85

2013-07-01 Thread Bennie Kloosteman
Correct, it's only for source code and config files and perhaps sending by email. For email you would always use Base64 , there is no efficiency gain due to the header costs and every system uses it. For source and config , it would only apply to older languages , most modern languages have all

Re: [zeromq-dev] Pros and cons of using Cassandra for storage?

2013-04-17 Thread Bennie Kloosteman
The other question is do you want persistant messaging ...To which i would say no ( in 95% of cases) ...good in theory , crap in the field and builds the expectation that things just work and when things go pear shape and it doesnt just work you dont have the systems that deal with failure.

Re: [zeromq-dev] Actor Model

2012-12-15 Thread Bennie Kloosteman
I wouldn't like to build an actor model directly on ZeroMQ as you can do better for in memory transport than ZeroMQ and you may want to place some actors on the same server .. as the implementation for an abstract Actor framework is fine just don't tie it to the implementation. Regarding threads

Re: [zeromq-dev] Actor Model

2012-12-15 Thread Bennie Kloosteman
1. Each thread has one socket. You hash the unique name of the actor and assign it accordingly to a thread. advantages: no contention. disadvantages: it cannot work if actors have different processing needs. You rely on the randomization of the hash function to balance the work. this is pretty

Re: [zeromq-dev] UDP performance

2012-11-27 Thread Bennie Kloosteman
On Wed, Nov 28, 2012 at 11:19 AM, Pieter Hintjens p...@imatix.com wrote: I did quite a lot of experimentation with UDP for 0MQ patterns a year or two ago. The results weren't impressive, performance wise. Semantically, it maps quite nicely, if you're OK with short messages and a high rate of

Re: [zeromq-dev] Distributed Q with **lots** of consumers

2012-11-12 Thread Bennie Kloosteman
This is a fairly standard bus pattern , though personally i HATE persistent queues with a passion , they kill all performance , you have to manage the infrastructure and the possibility of poison messages - for most business cases there are better ways to guarantee delivery eg send update events

Re: [zeromq-dev] epgm performance numbers

2012-10-29 Thread Bennie Kloosteman
Ethernet contention ? It is broadcasting On Tue, Oct 30, 2012 at 3:19 AM, Rohan Bedarkar r_bedar...@yahoo.comwrote: Steve, I found something interesting. ZMQ roundtrip over epgm drops suddenly at one point.. Thoughts? .. ... 88: 143us 89: 144us 90: 145us 91: 146us 92:

Re: [zeromq-dev] epgm performance numbers

2012-10-29 Thread Bennie Kloosteman
On Oct 29, 2012, at 9:14 PM, Bennie Kloosteman bkloo...@gmail.com wrote: Ethernet contention ? It is broadcasting On Tue, Oct 30, 2012 at 3:19 AM, Rohan Bedarkar r_bedar...@yahoo.comwrote: Steve, I found something interesting. ZMQ roundtrip over epgm drops suddenly at one point

Re: [zeromq-dev] zmq_close() has no effect after fork()

2012-10-09 Thread Bennie Kloosteman
Why not create the context and socket after you fork ? Ben On Tue, Oct 9, 2012 at 2:57 PM, John Khvatov iva...@gmail.com wrote: Hello all! I'd like to get some feedback about my report: https://zeromq.jira.com/browse/LIBZMQ-441 My be there is any workarounds about this problem? -- WBR,

Re: [zeromq-dev] broken pipe on remote shutdown

2012-09-19 Thread Bennie Kloosteman
Doesnt the kernel send an EPIPE if you do a read on a socket where the other end has sent a TCP RST , lots of routers send this so you need to handle it. On Thu, Sep 20, 2012 at 12:22 AM, Yi Ding yi.s.d...@gmail.com wrote: On Wed, Sep 19, 2012 at 10:49 AM, Chuck Remes li...@chuckremes.com

Re: [zeromq-dev] Odd numbers with zeromq

2012-09-18 Thread Bennie Kloosteman
If you wait for responses you have to wait on latency which can get much worse eg international 250ms your limit is 4 messages per second. You cannot do better That why people build asynch systems...and why Doom/Quake use to spam UDP position updates - without waiting for an answer. Ben On

Re: [zeromq-dev] server with zmq_xrep closes connection automaticly

2012-09-06 Thread Bennie Kloosteman
Unix or windows server Ben On Thu, Sep 6, 2012 at 6:02 PM, HP010170 hp010...@gmail.com wrote: Zhong: If you can provide the source code, it will be easier for the various 0mq experts on the list to assist with your problem. HP On 06/09/2012 06:29, Zhong Weilin wrote: Hi, zeromq

Re: [zeromq-dev] Licence

2012-09-06 Thread Bennie Kloosteman
Ahh thanks, looks like the anti add bar in brain needs adjustment Ben On Fri, Sep 7, 2012 at 10:44 AM, Pieter Hintjens p...@imatix.com wrote: On Fri, Sep 7, 2012 at 10:41 AM, Bennie Kloosteman bkloo...@gmail.com wrote: I see the 15/ZMTP spec is covered by the GPL licence

Re: [zeromq-dev] Too much ZeroMQ overhead versus plain TCP Java NIO Epoll (with measurements)

2012-08-30 Thread Bennie Kloosteman
On Thu, Aug 30, 2012 at 11:56 PM, Julie Anderson julie.anderson...@gmail.com wrote: As I said in the text you quoted: I will try to come up with a simple version to do the same thing. But Stuart did that for me in C. My thanks to him. I am not complaining about anything... Just trying to

Re: [zeromq-dev] Too much ZeroMQ overhead versus plain TCP Java NIO Epoll (with measurements)

2012-08-29 Thread Bennie Kloosteman
2) Do people agree that 11 microseconds are just too much? Nope once you go cross machine that 11 micro seconds become irrelevant . The fastest exchange im aware of for frequent trading is 80 micro seconds (+ transport costs) best case , so who are you talking to and if your not doing

Re: [zeromq-dev] Too much ZeroMQ overhead versus plain TCP Java NIO Epoll (with measurements)

2012-08-29 Thread Bennie Kloosteman
On Thu, Aug 30, 2012 at 10:35 AM, Julie Anderson julie.anderson...@gmail.com wrote: See my comments below: They appear to be single threaded synchronous tests which seems very unlike the kinds of applications being discussed (esp. if you're using NIO). More realistic is a network

Re: [zeromq-dev] C# client doesnt recieve messages from C++ server

2012-08-20 Thread Bennie Kloosteman
What error...is it a unicode error since your not using wchar eg no match ? Ben On Mon, Aug 20, 2012 at 3:55 PM, Alexander Voron voron.alexan...@gmail.comwrote: I've tried to program sending messages from one server to multiple clients. I have to use C# on client side, C++ on server side. I

Re: [zeromq-dev] C# client doesnt recieve messages from C++ server

2012-08-20 Thread Bennie Kloosteman
:10, Bennie Kloosteman bkloo...@gmail.com wrote: What error...is it a unicode error since your not using wchar eg no match ? Ben On Mon, Aug 20, 2012 at 3:55 PM, Alexander Voron voron.alexan...@gmail.com wrote: I've tried to program sending messages from one server to multiple

Re: [zeromq-dev] Relation between - HI_WATER_MARK and SEND_BUFFER_SIZE of zmq ?

2012-08-20 Thread Bennie Kloosteman
Do you need messages replayed from the database or just stored ? If replayed to me its sounds like a queue like MSMQ ( or the IBM one) is a better fit as it does everything you want , personally i hate queues , the performance especially for persitant ones is bad and the superficial simplicty

Re: [zeromq-dev] does Pub/sub guarantee delivering within Local Net ?

2012-08-07 Thread Bennie Kloosteman
Just because the socket is guaranteed does not guarantee the receiving app / service has received it... Only your own app can do that with some sort of ack. Ben On Wed, Aug 8, 2012 at 6:16 AM, Danil Gazizov danil.gazi...@gmail.comwrote: Soon.. I can answer to myself - yes, it does, like socket

Re: [zeromq-dev] looking inside ZMQ queue

2012-07-27 Thread Bennie Kloosteman
Like most systems ZeroMQ will just send the message to the transport when the transport has received the message ( eg TCP/IP )it is removed from ZeroMQ. There are no guarantees it has even been sent , yet alone that the transport layer on the other side has the message. This is pretty standard

Re: [zeromq-dev] Websockets as a Transport ?

2012-07-07 Thread Bennie Kloosteman
On Sat, Jul 7, 2012 at 8:16 AM, Mark Farnan mark.far...@petrolink.comwrote: Hi Ben, ** ** HttpS (more specifically SSL) is mandatory requirement in the Industry, just forget trying to do anything without it that involves the Internet. I know but you said minimal authentication , if

Re: [zeromq-dev] Websockets as a Transport ?

2012-07-06 Thread Bennie Kloosteman
Hi Mark, Web socket uses tcp underneath so getting through the firewall is no issue just use port 80 , and in fact web sockets ( and things that use port 80 ) have issues with some ISP html proxy servers and load balancers and it will take time for these products to mature to handle web sockets.

Re: [zeromq-dev] I've written a completely new C# API for ZeroMQ

2012-05-02 Thread Bennie Kloosteman
1. Ensure the DLL has permissions to run on your local machines ( right click properties) . This is not just for .net but exes also. 2. Make sure you are running in full trust . Regards, Ben On Wed, May 2, 2012 at 1:18 PM, Thomas Fee angry...@hotmail.com wrote: Hey Alex, sounds great but no