[
https://issues.apache.org/jira/browse/CASSANDRA-10421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14959979#comment-14959979
]
Stefania commented on CASSANDRA-10421:
--------------------------------------
bq. So what I think I see is that when the LogTransaction completes it first
writes to the log the commit record, and then starts making permanent changes
to the the files on disk (deleting the old ones). But if it hasn't actually
synced the log to disk then on a restart we could have a partial log and
attempt to roll back, but it is too late because before the crash we had
already deleted parts of the before state. At the end we should sync the log
files before deleting the obsolete files right?
Yes this is correct, that's why we flush after writing every record but if we
want to survive a power cut then we should call {{fsync}}.
bq. Was the intent to sync the folder when creating the log file or when adding
a record which indicates the addition of other data files?
The intent was to sync the folder when creating the file and to sync the
content of the file with a flush when appending data to it. However flushing
only passes the data to the operating system; it won't protect us from a power
cut. This wasn't clear to me yesterday. At a minimum we should {{fsync}} after
adding the final record and probably also when adding new records as you
correctly pointed out. Therefore, I would strongly prefer to leave it as it was
before and call {{fsync}} on the folder whenever we add a record, so in case of
a power cut we have the most up-to-date data. This is what I did in the latest
commit. Let me know if you have performance concerns.
I also had to add several {{log.txnFile().close();}} to the unit tests
(whenever we test {{removeUnfinishedLeftovers}}) because on Windows we cannot
delete files that are open. This is a bit ugly so maybe we should also go back
to using {{FileUtils::appendLine}}, unless again you have performance concerns.
I've rebased on 3.0 and started a new CI run on both Linux and Windows.
> Potential issue with LogTransaction as it only checks in a single directory
> for files
> -------------------------------------------------------------------------------------
>
> Key: CASSANDRA-10421
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10421
> Project: Cassandra
> Issue Type: Bug
> Reporter: Marcus Eriksson
> Assignee: Stefania
> Priority: Blocker
> Fix For: 3.0.0 rc2
>
>
> When creating a new LogTransaction we try to create the new logfile in the
> same directory as the one we are writing to, but as we use
> {{[directories.getDirectoryForNewSSTables()|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/lifecycle/LogTransaction.java#L125]}}
> this might end up in "any" of the configured data directories. If it does,
> we will not be able to clean up leftovers as we check for files in the same
> directory as the logfile was created:
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/lifecycle/LogRecord.java#L163
> cc [~Stefania]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
