Abhishek Rai updated ZOOKEEPER-2574:
    Attachment: ZOOKEEPER-2574.4.patch

Thanks for the review [~abrahamfine].  I've applied your comments and uploaded 
a new patch set, please take another look.

> PurgeTxnTest.java:224 Can we change ArrayList<File> logsToPurge back to 
> List<File> logsToPurge?
I switched logsToPurge from a List<File> to an ArrayList<File> so I can simply 
use remove(0) to remove the first element in the list on line 239.  However, as 
you pointed out, this is probably not obvious given that all other lists around 
it are List<File>, so I've added a comment explaining the choice.

> PurgeTxnLog.java:138 Do we need to use the FileFilter here since we do 
> "filtering" on line 142?
Both filtering are required.  The FileFilter used in lines 134-138 are useful 
for listing all snapshot and log files with zxid >= leastZxidToBeRetain.  The 
check on 142 is to skip deletion of the newest log file that comes before the 
oldest retained snapshot.  However, I agree that the logic would be simpler if 
all filtering logic is in one place, in MyFileFilter.accept().  I've moved it 
there now.

> PurgeTxnLog.java:148 We do logging and System.out.println for the same 
> String, do we need both?

My goal here was to capture the output in the log file generated by the ant 
test run.  System.out.println wasn't useful in this context.  However, I needed 
to retain System.out.println cause PurgeTxnLog can also be invoked 
interactively from a console.  Is there a way to achieve both goals, logging 
and console output (preferably stdout) without any duplication.

> PurgeTxnLog can inadvertently delete required txn log files
> -----------------------------------------------------------
>                 Key: ZOOKEEPER-2574
>                 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2574
>             Project: ZooKeeper
>          Issue Type: Bug
>          Components: server
>    Affects Versions: 3.4.7, 3.4.8, 3.5.0, 3.5.1, 3.5.2
>         Environment: Zookeeper 3.4.8, standalone, and 3-server quorum
>            Reporter: Abhishek Rai
>            Assignee: Abhishek Rai
>            Priority: Blocker
>             Fix For: 3.4.10, 3.5.3
>         Attachments: ZOOKEEPER-2574.2.patch, ZOOKEEPER-2574.3.patch, 
> ZOOKEEPER-2574.4.patch, ZOOKEEPER-2574.patch
> As part of the fix for ZOOKEEPER-1797, the call to 
> FileTxnSnapLog.getSnapshotLogs() was removed from PurgeTxnLog.java.  As a 
> result, some old-looking but required txn log files can be deleted, resulting 
> in data corruption or loss.
> For example, consider the following:
> 1. Configuration:
> autopurge.snapRetainCount=3
> 2. Following files exist:
> log.100 spans transactions from zxid=100 till zxid=140 (inclusive)
> snapshot.110 - snapshot as of zxid=110
> snapshot.120 - snapshot as of zxid=120
> snapshot.130 - snapshot as of zxid=130
> Above scenario is possible when snapshotting has happened multiple times but 
> without accompanying log rollover, which is possible if the server was 
> running as a learner.
> 3. PurgeTxnLog retains all snapshots but deletes log.100 because its zxid is 
> older than the zxid of the oldest snapshot (110).  This results in loss of 
> transactions in the range 131-140.
> Before the fix for ZOOKEEPER-1797, this was avoided by the call to 
> FileTxnSnapLog.getSnapshotLogs() which finds and retains the newest txn log 
> file with starting zxid < oldest retained snapshot's highest zxid.

This message was sent by Atlassian JIRA

Reply via email to