Thank you for your answer!
On 5/4/22 13:33, Ben Kochie wrote:
That's not correct, the author seems to be unaware of the difference
between fsync and fdatasync[0].
The TSDB's WAL calls an internal sync(), which is implemented as
fdatasync() by default every minute on a timer.
https://github.com/prometheus/prometheus/blob/2381d7be57314df63c17291f22e5e770b44a70f3/tsdb/wal.go#L699-L702
It also calls the sync() when closing out individual WAL segments.
[0]: https://linux.die.net/man/2/fdatasync
On Wed, May 4, 2022 at 12:23 PM Volker Dormeyer <[email protected]>
wrote:
Hello all,
I have serious question regarding the following article:
https://valyala.medium.com/wal-usage-looks-broken-in-modern-time-series-databases-b62a627ab704
It is about the statement as follows from the author:
“Prometheus calls fsync
<https://github.com/prometheus/tsdb/blob/bc3b0bd429153ab54662a930df3817e4f29d169e/wal/wal.go#L390>
only after big chunk of data (aka |segment|) is written into WAL,
so all
the |segment| data may be lost / corrupted on power loss before
|fsync|.
The data may be corrupted if the OS flushes a few pages with the
written
data to disk, but doesn’t flush the remaining pages. Prometheus
|fscync|’s segments every 2 hours by default, so a lot of data may be
corrupted on hardware reset.”
Is it still true? Can we shorten the interval between fsync calls?
Does
this make sense?
Thanks!
Best Regards,
Volker
--
You received this message because you are subscribed to the Google
Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to [email protected]
<mailto:prometheus-users%[email protected]>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/4a8ac708-57c7-ea33-8973-ac2ce0ab1ca7%40ixolution.de.
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/1166bca5-7535-4d4a-2f7b-0779cb9b80c2%40ixolution.de.
