That's not correct, the author seems to be unaware of the difference between fsync and fdatasync[0].
The TSDB's WAL calls an internal sync(), which is implemented as fdatasync() by default every minute on a timer. https://github.com/prometheus/prometheus/blob/2381d7be57314df63c17291f22e5e770b44a70f3/tsdb/wal.go#L699-L702 It also calls the sync() when closing out individual WAL segments. [0]: https://linux.die.net/man/2/fdatasync On Wed, May 4, 2022 at 12:23 PM Volker Dormeyer <[email protected]> wrote: > Hello all, > > I have serious question regarding the following article: > > > https://valyala.medium.com/wal-usage-looks-broken-in-modern-time-series-databases-b62a627ab704 > > It is about the statement as follows from the author: > > “Prometheus calls fsync > < > https://github.com/prometheus/tsdb/blob/bc3b0bd429153ab54662a930df3817e4f29d169e/wal/wal.go#L390> > > only after big chunk of data (aka |segment|) is written into WAL, so all > the |segment| data may be lost / corrupted on power loss before |fsync|. > The data may be corrupted if the OS flushes a few pages with the written > data to disk, but doesn’t flush the remaining pages. Prometheus > |fscync|’s segments every 2 hours by default, so a lot of data may be > corrupted on hardware reset.” > > Is it still true? Can we shorten the interval between fsync calls? Does > this make sense? > > Thanks! > > Best Regards, > Volker > > > -- > You received this message because you are subscribed to the Google Groups > "Prometheus Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/prometheus-users/4a8ac708-57c7-ea33-8973-ac2ce0ab1ca7%40ixolution.de > . > -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/CABbyFmqPcWKu-KxYouqZ7f41CXUTUdoNKdy8KRWwXx%3DKsXw_SQ%40mail.gmail.com.
