Dhammika,
> We can add scatter/gather IO API to zmq_msg. Proper readv/writev on
> sockets will need some changes in encoder/decoder, but we can use
> current encoder/decoder and read/write each iov entry separately.
>
> typedef struct
> {
> void *content;
> unsigned char shared;
> unsigned char vsm_size;
> unsigned char vsm_data [ZMQ_MAX_VSM_SIZE];
> struct iovec iov [ZMQ_MAX_IOV_SIZE];
> } zmq_msg_t;
>
> int zmq_msg_init_iov (zmq_msg_t *msg, int idx, void *data,
> size_t size, zmq_free_fn *ffn, void *hint);
Yes. It's doable, however, it would require quite an unpleasant change
on the receiving end. Each receiving application would have to handle
the case when the message is delivered in several non-contiguous chunks.
Think of the added complexity needed to process data randomly split into
several pieces!
I've been following different train of thought: When sending data like
those in Michael's scenario, the splits are not random. Rather they
mirror business logic of the application. For example, the structure of
the message may look like this (matrixA, matrixB, matrixC).
In such a case - if we preserve the splits when message is delivered to
the receiving end - it would be pretty easy to process the data. You'll
simply process individual matrices, each one being guaranteed to reside
in a contiguous memory region.
That brings me to the concept of "message groups" that's already
provided by some messaging implementations. "Message group" is a group
of messages that is guaranteed to be delivered in atomic fashion, i.e.
you'll either get none of the messages in the group or you'll get all of
them.
What about this:
zmq_send (s, msg1, ZMQ_GROUP);
zmq_send (s, msg2, ZMQ_GROUP);
zmq_send (s, msg3, ZMQ_GROUP);
zmq_flush (s);
It's less trivial to implement (at very least it would require tweaking
the wire protocol), however, the added value is much greater than in the
simple iovec case.
Thoughts?
Martin
_______________________________________________
zeromq-dev mailing list
[email protected]
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
