Re: [zeromq-dev] Zmq 4.2.0 aligned memory

[KIU Shueng Chuan]

A common use case for me is sending an array of floats.

First message part is some small metadata. Second message part is the float
array.

On reception, zmq_msg_data is cast to float* and accessed directly.

This non-alignment would be problematic.
Or perhaps there never was any alignment guarantee?

On 15 Nov 2016 3:34 a.m., "Jens Auer"  wrote:

> Hi,
>
> I think I have an idea why you are seeing unaligned messages, but this
> only applies to messages where the payload is not stored in the msg_t
> object itself. I think the threshold for this is 64 bytes. In ZeroMQ 4.1,
> receiving messages was done by first receiving from the socket into a
> static 8kb buffer, and then a new message object was created that allocated
> memory externally by calling malloc.  The payload was then copied from the
> receive buffer to the message buffer. The malloced message buffer was
> aligned probably.
>
> In ZeroMQ 4.2, this is changed to reduce the number of malloc calls and
> copy operations. The receive buffer is now dynamically allocated as a 8kb
> block, and messages are constructed as zero-copy messages using the part of
> the receive buffer containing the payload. This saves malloc calls and copy
> operations and increases performance. However, the payload may now start at
> basically arbitrary addresses. As an example, let's assume that we receive
> a small message of 10 bytes and a large message of 1kb, both received in a
> single call to recv on the socket. The engine allocates a new buffer of
> 8kb, calls recv(socket, buffer) and the data is written to the buffer. A
> small message is then created which contains the data from byte 2-11 in the
> msg_t, byte 1 contains the header. At byte 12 starts the header of the next
> message, and at byte 22(?) starts the payload. The large message is created
> as a zero-copy message using the pointer to byte 22 as storage. This is not
> aligned to a 4-byte address.
>
> Could you provide some more information about the sizes of the messages
> that you receive? How do you decode the buffer content?
>
> Best wishes,
> Jens
>
> -Ursprüngliche Nachricht-
> Von: zeromq-dev [mailto:zeromq-dev-boun...@lists.zeromq.org] Im Auftrag
> von Emmanuel Taurel
> Gesendet: Montag, 14. November 2016 16:49
> An: zeromq-dev@lists.zeromq.org
> Betreff: [zeromq-dev] Zmq 4.2.0 aligned memory
>
> Hello all,
>
> We are using zeromq since years now without troubles. We have recently
> tried our software using Zmq 4.2.0 (on linux hosts).
> For our application, we are using multipart messages with 4 parts in
> publish/subscribe mode.
> With Zmq 4.0.5, on the subscriber side, when we get the last message part,
> the received buffer was memory aligned (at least on 0x4 border).
> Unfortunately, with Zmq 4.2.0, the buffer is not aligned any more.
>  For instance with Zmq 4.0.5, the buffer was at address xxx08 while with
> Zmq 4.2.0, it is at address xxx23.
>
> I don't know if it is relevant but our messages are relatively small
> messages (few tens of bytes) This is a problem for us in decoding the
> buffer content.
>
> Is there something to be done to have memory aligned buffers?
>
> Thank's in advance for your answers
>
>  Emmanuel
> ___
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
> ---
> Diese E-Mail wurde von Avast Antivirus-Software auf Viren geprüft.
> https://www.avast.com/antivirus
>
> ___
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Zmq 4.2.0 aligned memory

[Jens Auer]

Hi,

I think I have an idea why you are seeing unaligned messages, but this only 
applies to messages where the payload is not stored in the msg_t object itself. 
I think the threshold for this is 64 bytes. In ZeroMQ 4.1, receiving messages 
was done by first receiving from the socket into a static 8kb buffer, and then 
a new message object was created that allocated memory externally by calling 
malloc.  The payload was then copied from the receive buffer to the message 
buffer. The malloced message buffer was aligned probably.

In ZeroMQ 4.2, this is changed to reduce the number of malloc calls and copy 
operations. The receive buffer is now dynamically allocated as a 8kb block, and 
messages are constructed as zero-copy messages using the part of the receive 
buffer containing the payload. This saves malloc calls and copy operations and 
increases performance. However, the payload may now start at basically 
arbitrary addresses. As an example, let's assume that we receive a small 
message of 10 bytes and a large message of 1kb, both received in a single call 
to recv on the socket. The engine allocates a new buffer of 8kb, calls 
recv(socket, buffer) and the data is written to the buffer. A small message is 
then created which contains the data from byte 2-11 in the msg_t, byte 1 
contains the header. At byte 12 starts the header of the next message, and at 
byte 22(?) starts the payload. The large message is created as a zero-copy 
message using the pointer to byte 22 as storage. This is not aligned to a 
4-byte address. 

Could you provide some more information about the sizes of the messages that 
you receive? How do you decode the buffer content? 

Best wishes,
Jens

-Ursprüngliche Nachricht-
Von: zeromq-dev [mailto:zeromq-dev-boun...@lists.zeromq.org] Im Auftrag von 
Emmanuel Taurel
Gesendet: Montag, 14. November 2016 16:49
An: zeromq-dev@lists.zeromq.org
Betreff: [zeromq-dev] Zmq 4.2.0 aligned memory

Hello all,

We are using zeromq since years now without troubles. We have recently tried 
our software using Zmq 4.2.0 (on linux hosts).
For our application, we are using multipart messages with 4 parts in 
publish/subscribe mode.
With Zmq 4.0.5, on the subscriber side, when we get the last message part, the 
received buffer was memory aligned (at least on 0x4 border). Unfortunately, 
with Zmq 4.2.0, the buffer is not aligned any more.
 For instance with Zmq 4.0.5, the buffer was at address xxx08 while with Zmq 
4.2.0, it is at address xxx23.

I don't know if it is relevant but our messages are relatively small messages 
(few tens of bytes) This is a problem for us in decoding the buffer content.

Is there something to be done to have memory aligned buffers?

Thank's in advance for your answers

 Emmanuel
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev


---
Diese E-Mail wurde von Avast Antivirus-Software auf Viren geprüft.
https://www.avast.com/antivirus

___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

[zeromq-dev] SEHException 0x80004005 from ZeroMQ/libzmq

[Aaron Friesen]

All,

Getting an SEHException 0x80004005 from ZeroMQ (4.1.0.21) / libzmq (4.1.5.0)

Multiple processes went down with the same exception at the same time.

Was not able to get a dump but the application logs showed the following stack 
trace:

System.Exception System.Runtime.InteropServices.SEHException (0x80004005): 
External component has thrown an exception.
at ZeroMQ.lib.zmq.zmq_msg_send(IntPtr msg, IntPtr socket, Int32 flags)
at ZeroMQ.ZSocket.SendFrame(ZFrame frame, ZSocketFlags flags, ZError& error)
at ZeroMQ.ZSocket.SendFrames(IEnumerable`1 frames, Int32& sent, ZSocketFlags 
flags, ZError& error)
at ZeroMQ.ZSocket.SendFrames(IEnumerable`1 frames, ZSocketFlags flags, ZError& 
error)
at ZeroMQ.ZSocket.SendMessage(ZMessage msg, ZSocketFlags flags, ZError& error)
at ZeroMQ.ZSocket.SendMessage(ZMessage msg, ZSocketFlags flags)
at ZeroMQ.ZSocket.SendMessage(ZMessage msg)
at xx.SocketsThread(Object eventWaitHandle)

No line numbers available, but based on the logged message, it would have 
occurred in the following code.  Because the stack trace does not include any 
of the calls within the try block (PollIn, ProcessRequest, 
ProcessSubscription), I am at a loss as to what exactly was executing at the 
time of the exception that was calling SendMessage.

Does anyone have any ideas as to what I might be doing wrong, or what the 
problem might be and how to avoid it?



ZSocket[] sockets = new ZSocket[] { _requestSocket, 
_subscriberSocket };
ZPollItem[] pollItems = new ZPollItem[] { 
ZPollItem.CreateReceiver(), ZPollItem.CreateReceiver() };
ZMessage[] messages = null;

try
{
TimeSpan timeout = TimeSpan.FromMilliseconds(100);

while (_run)
{
if (ZPollItems.PollIn(sockets, pollItems, out messages, 
out error, timeout))
{
if (error == ZError.EAGAIN)
continue;

if (error == ZError.ETERM)
break;

if (messages == null)
continue;

if (messages[0] != null)// Request
ProcessRequest(messages[0]);

if (messages[1] != null)// Subscription
ProcessSubscription(messages[1]);
}
else
{
if (error == ZError.EAGAIN)
continue;

if (error != ZError.None)
break;
}
}
}
catch (Exception ex)
{
if (!(ex is ThreadAbortException))
{
_logger.FatalException(string.Format("Exception 
encountered while polling for messages on sockets. Thread '{0}' shutting 
down.", threadName), ex);

Environment.Exit(-1);
}
}

Thank you in advance,

Aaron Friesen

___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Zmq 4.2.0 aligned memory

[Luca Boccassi]

On Mon, 2016-11-14 at 16:48 +0100, Emmanuel Taurel wrote:
> Hello all,
> 
> We are using zeromq since years now without troubles. We have recently
> tried our software using Zmq 4.2.0 (on linux hosts).
> For our application, we are using multipart messages with 4 parts in
> publish/subscribe mode.
> With Zmq 4.0.5, on the subscriber side, when we get the last message
> part, the received buffer was memory aligned
> (at least on 0x4 border). Unfortunately, with Zmq 4.2.0, the buffer is
> not aligned any more.
>  For instance with Zmq 4.0.5, the buffer was at address xxx08 while with
> Zmq 4.2.0, it is at address xxx23.
> 
> I don't know if it is relevant but our messages are relatively small
> messages (few tens of bytes)
> This is a problem for us in decoding the buffer content.
> 
> Is there something to be done to have memory aligned buffers?
> 
> Thank's in advance for your answers
> 
>  Emmanuel

Hi,

Since 4.0.5 a couple things have changed externally:

- zmq_msg_t has increased to 64 bytes from 32 bytes

- zmq_msg_t is now aligned to pointer size, to fix SIGBUS crashes on
some architectures

Exactly how large are your payloads?

It's possible that before they didn't fit in the inline buffer, since it
was really small, but they do now and that means they are not aligned
anymore since the inline buffer is not the first element of the struct:

struct {
   metadata_t *metadata;
   unsigned char data [max_vsm_size];
   unsigned char size;
   unsigned char type;
   unsigned char flags;
   char group [16];
   uint32_t routing_id;
} vsm;

Although an xxx23 address is a bit strange. In theory it should be
aligned to ptr size + 8.

If they are bigger than the very small size and they end up on the heap,
then the content_t is not aligned at all:

struct content_t
{
void *data;
size_t size;
msg_free_fn *ffn;
void *hint;
zmq::atomic_counter_t refcnt;
};

You could try to add the alignment attribute there in src/msg.hpp and
see if it makes a difference for you.


signature.asc
Description: This is a digitally signed message part
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

[zeromq-dev] Zmq 4.2.0 aligned memory

[Emmanuel Taurel]

Hello all,

We are using zeromq since years now without troubles. We have recently
tried our software using Zmq 4.2.0 (on linux hosts).
For our application, we are using multipart messages with 4 parts in
publish/subscribe mode.
With Zmq 4.0.5, on the subscriber side, when we get the last message
part, the received buffer was memory aligned
(at least on 0x4 border). Unfortunately, with Zmq 4.2.0, the buffer is
not aligned any more.
 For instance with Zmq 4.0.5, the buffer was at address xxx08 while with
Zmq 4.2.0, it is at address xxx23.

I don't know if it is relevant but our messages are relatively small
messages (few tens of bytes)
This is a problem for us in decoding the buffer content.

Is there something to be done to have memory aligned buffers?

Thank's in advance for your answers

 Emmanuel
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

[zeromq-dev] Docker logs via Zeromq

[Benjamin Henrion]

Hi,

Does someones runs this kind of solution to ship docker logs over zeromq?

https://tomasz.janczuk.org/2015/09/from-kafka-to-zeromq-for-log-aggregation.html

His solution is barely documented in terms of code, the only thing
there is a gist:

https://gist.github.com/tjanczuk/f133cc65977f5a8c4a7f

Best,

-- 
Benjamin Henrion 
FFII Brussels - +32-484-566109 - +32-2-3500762
"In July 2005, after several failed attempts to legalise software
patents in Europe, the patent establishment changed its strategy.
Instead of explicitly seeking to sanction the patentability of
software, they are now seeking to create a central European patent
court, which would establish and enforce patentability rules in their
favor, without any possibility of correction by competing courts or
democratically elected legislators."
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev