On 2021-06-15, Claudio Jeker <[email protected]> wrote:
> On Tue, Jun 15, 2021 at 04:24:08PM +0200, Julien Pivotto wrote:
>> Hello,
>>
>> I am a Prometheus maintainer and we have received a bug regarding
>> Prometheus - prometheus would no longer work on OpenBSD since we
>> introduced MMAP:
>>
>> https://github.com/prometheus/prometheus/issues/8877
>> https://github.com/prometheus/prometheus/issues/8799
>>
>> I would like to know if the facts here are accurate and, on the
>> opposite, if there are happy openbsd users of Prometheus 2.19+.
>>
>> I see that Prometheus 2.24 is packaged upstream, so I guess there are
>> users. Can you please interact with us so we can better understand the
>> situation at play.
>>
>
> Unlike other OS OpenBSD does not automatically sync between mmap-ed memory
> of a file with any write() to the same file (OpenBSD has no unified
> cache). It requries use of msync(2) to make sure that mappings are
> properly updated.
The other option is to ensure that i/o is all done through mmap and not
use file access at all. For openldap/lmdb we deal with this by enforcing
setting MDB_WRITEMAP (based on a suggestion from Howard Chu); there are
trade-offs
"This is faster and uses fewer mallocs, but loses protection from
application bugs like wild pointer writes and other bad updates
into the database. Incompatible with nested transactions. Do not
mix processes with and without MDB_WRITEMAP on the same environment.
This can defeat durability mdb_env_sync etc)."
Some other software (dovecot, sqlite) wants to use mmap but has
alternative code; where we have noticed this we disable use of mmap.
Cyrus imapd had problems with this too, I'm not sure where they
ended up, that was handled 10+ years ago.
> While prometheus works, it also does not.
It is quite likely this is the case for some other software in ports.
Adding syncs feels a bit problematic to me. It's what is needed when
mixing file/mmap on OpenBSD, but is very difficult to ensure every case
is handled, especially in the face of changing code, and the problems
from missing them simply don't show up on most other OS. But there is
not an alternative without coherent mmap/file access (not *necessarily*
UBC, though that's a common way to do it).
