[ https://issues.apache.org/jira/browse/KAFKA-3968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jun Rao resolved KAFKA-3968. ---------------------------- Fix Version/s: 3.0.0 Assignee: Cong Ding Resolution: Fixed Merged to trunk. Thanks Cong for fixing this long lasting issue. > fsync() is not called on parent directory when new FileMessageSet is flushed > to disk > ------------------------------------------------------------------------------------ > > Key: KAFKA-3968 > URL: https://issues.apache.org/jira/browse/KAFKA-3968 > Project: Kafka > Issue Type: Bug > Components: log > Affects Versions: 0.9.0.1, 0.10.0.0 > Environment: Linux, ext4 filesystem > Reporter: Andrey Neporada > Assignee: Cong Ding > Priority: Major > Labels: reliability > Fix For: 3.0.0 > > > Kafka does not call fsync() on directory when new log segment is created and > flushed to disk. > The problem is that following sequence of calls doesn't guarantee file > durability: > fd = open("log", O_RDWR | O_CREATE); // suppose open creates "log" > write(fd); > fsync(fd); > If system crashes after fsync() but before parent directory have been flushed > to disk, the log file can disappear. > This is true at least for ext4 on Linux. > Proposed solution is to flush directory when flush() is called for the first > time. -- This message was sent by Atlassian Jira (v8.3.4#803005)