Hi,
I would have the same conclusion than Ellie regarding the proposal for a custom
patch here, we indeed use Cyrus 3.0.x in production and the code mentioned in
this issue is present in this version without any issue for long events (length
> 1400 bytes for us)
>From which version of Cyrus Imap do you come from for encountering this
>regression ? Or is it a change in your system ?
Also, the socket(7) man page describes how SO_RCVBUF and SO_SNDBUF are defined
by default :
SO_SNDBUF
Sets or gets the maximum socket send buffer in bytes. The
kernel doubles this value (to allow space for bookkeeping
overhead) when it is set using
setsockopt(2)<https://man7.org/linux/man-pages/man2/setsockopt.2.html>, and this
doubled value is returned by
getsockopt(2)<https://man7.org/linux/man-pages/man2/getsockopt.2.html>. The
default
value is set by the /proc/sys/net/core/wmem_default file
SO_RCVBUF
Sets or gets the maximum socket receive buffer in bytes.
The kernel doubles this value (to allow space for
bookkeeping overhead) when it is set using
setsockopt(2)<https://man7.org/linux/man-pages/man2/setsockopt.2.html>,
and this doubled value is returned by
getsockopt(2)<https://man7.org/linux/man-pages/man2/getsockopt.2.html>. The
default value is set by the
/proc/sys/net/core/rmem_default file
In our Linux system hosting Cyrus we configure /proc/sys/net/core/rmem_default
and /proc/sys/net/core/wmem_default with the same value of 212992
To have an understanding of the reason for dividing by 10 minus 32 there is a
good technical response here : https://stackoverflow.com/a/22007520
Indeed, SO_SNDBUF is the size of the buffer for the socket, not the size of a
single message send with sendto.
Cdt,
Sébastien
On Thu, 23 Feb 2023, at 8:04 AM, Matteo A. wrote:
In imap/notify.c, data is written to the socket using a buffer sized at 556; in
nodityd/notifyd.c the buffer at reading is limited to 480 by the following:
bufsiz = MIN(bufsiz / 10 - 32, NOTIFY_MAXSIZE);
(https://github.com/cyrusimap/cyrus-imapd/issues/1674<https://fra01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fcyrusimap%2Fcyrus-imapd%2Fissues%2F1674&data=05%7C01%7Csebastien.michel%40worldline.com%7C0b913c4bf1a74f3e46de08db15f2daa3%7Cfda9decfe89243ac9d9f1a493f9f98d0%7C0%7C0%7C638127905027293261%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=NX7WibeAvZAmGAcJrCxcl8F1IcpC3Okvdk0lI%2BE4fdw%3D&reserved=0>)
Applying a little math, for us seems that the value of SO_RCVBUF read by
getsockopt is 5120.
So, after patching the two sources, removing the division by 10, MessageMove
events come back to work again.
Ahh! I did see that code and wondered later if maybe it was the culprit...
It seems like there's an alternate code path in there that uses dlist instead
(of ??), and it looks like it relies on "notifysocket" in imapd.conf having a
value prefixed with "dlist:" -- have a look at notify_dlist() and its caller in
imap/notify.c. It doesn't seem to be limited in length, so it might avoid this
truncation.
I don't see it documented anywhere though, just noticed it in the code while I
was rummaging around, so I don't know what it's intended for or how to set it
up properly. I imagine your external_event.pl script would need to be updated
to parse dlist instead of whatever the usual format is. There's a Cyrus::DList
perl module in perl/imap/Cyrus/DList.pm that will help with that, and it's used
by the cassandane tests, so you can look there for working examples of its
usage. That might be preferable to custom patches tinkering with datagram
lengths?
If you end up chasing that path, I'd appreciate a writeup that can be
incorporated into the docs! (Or even better, a PR that updates the docs
directly.)
Cheers,
ellie
Worldline, Cardlink, GoPay and Santeos are registered trademarks and trade
names owned by the Worldline Group. This e-mail and any documents attached are
confidential and intended solely for the addressee. It may also be privileged.
If you are not the intended recipient of this e-mail, you are not authorized to
copy, disclose, use or retain it. Please notify the sender immediately and
delete this e-mail (including any attachments) from your systems. As e-mails
may be intercepted, amended or lost, they are not secure. Therefore,
Worldline's and its subsidiaries' liability cannot be triggered for the message
content. Although the Worldline Group endeavors to maintain a virus-free
network, we do not warrant that this e-mail is virus-free and do not accept
liability for any damages, losses or consequences resulting from any
transmitted virus if any. The risks are deemed to be accepted by anyone who
communicates with Worldline or its subsidiaries by e-mail.
------------------------------------------
Cyrus: Info
Permalink:
https://cyrus.topicbox.com/groups/info/T84dc31747d015b6c-M9daa39dc1d5b922b6064d38a
Delivery options: https://cyrus.topicbox.com/groups/info/subscription
