Hi guys, I am working on implementing a Byzantine Fault Tolerant module on top of Zookeeper. As part of our work, we run experiments on the unmodified version of Zookeeper. We noticed yesterday that the PurgeTxnLog does not work properly, and were surprised to find that you noticed it at the same time! :) I tried to apply the latest patch that Mahadev posted, but it does not work properly. Most probably, I am missing something from the development. I applied the patch over version 3.0.1 and get the following compile errors:

[javac] Compiling 3 source files to /home/manos/zookeeper-3.0.1_orig/build/classes [javac] /home/manos/zookeeper-3.0.1_orig/src/java/main/org/apache/zookeeper/server/PurgeTxnLog.java:98: cannot find symbol
   [javac] symbol  : method getDataDir()
[javac] location: class org.apache.zookeeper.server.persistence.FileTxnSnapLog [javac] Arrays.asList(txnLog.getDataDir().listFiles(new MyFileFilter("log."))));
   [javac]                                     ^
[javac] /home/manos/zookeeper-3.0.1_orig/src/java/main/org/apache/zookeeper/server/PurgeTxnLog.java:100: cannot find symbol
   [javac] symbol  : method getSnapDir()
[javac] location: class org.apache.zookeeper.server.persistence.FileTxnSnapLog [javac] files.addAll(Arrays.asList(txnLog.getSnapDir().listFiles(new MyFileFilter("snapshot."))));
   [javac]                                          ^
   [javac] 2 errors

I checked and the txnLog (a FileTxnSnapLog object) does not have a getDataDir() method. The only getDataDir method is in ServerConfig, returning the static instance.dataDir string. The patch refers to revision 737951, so I guess these methods are already there in that revision. Is there any patch available that can be applied to 3.0.1 directly? If not, I can try and use the dataDir directly (but I guess there was a reason why it was changed).

Thank you,

Mahadev konar (JIRA) wrote:

Mahadev konar updated ZOOKEEPER-252:

    Attachment: ZOOKEEPER-252.patch

had a bug in the last patch.

PurgeTxnLog is not handling the new dataDir directory structure

                Key: ZOOKEEPER-252
                URL: https://issues.apache.org/jira/browse/ZOOKEEPER-252
            Project: Zookeeper
         Issue Type: Bug
         Components: server
   Affects Versions: 3.0.0, 3.0.1
           Reporter: Patrick Hunt
           Assignee: Mahadev konar
           Priority: Critical
            Fix For: 3.1.0

        Attachments: ZOOKEEPER-252.patch

org.apache.zookeeper.server.PurgeTxnLog class has not been updated to handle the new 
directory structure imposed by the upgrade from v2 to v3 of ZooKeeper. In particular the 
dataDir now has a "version-2" subdirectory that stores all of the 
snaps/transactionallogs for version2 of the persistence layer.
I also note that the documentation of this class is particularly poor. I'm 
working on ZOOKEEPER-229 and would like to point to the API docs for this class 
regarding usage but they api docs are nonexistent
Also - I think it's important for the user to be able to specify the number of "backup" 
snaps and logs that should be kept -- right now it seems we delete all but the current 
snaps/txlogs. Either by count or by date -- ie "remove anything 5 days or older, with a minum 
of 3 most recents snaps (and accompanying txlogs)" seems like a pretty common user case 
(assuming the operator is doing system backups every X days, etc...)
in general this class needs some tlc - the formatting should also be cleaned up.
Also - the API docs for this and LogFormatter are not included in the build.xml 
"javadoc" target. These are user utilities so javadoc for these two classes 
should be included. I will fix this issue as part of ZOOKEEPER-229. I'm also updateing 
the forrest documention in 229 so don't worry about that either.

Reply via email to