[
https://issues.apache.org/jira/browse/ZOOKEEPER-3826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17140059#comment-17140059
]
Michael Han commented on ZOOKEEPER-3826:
----------------------------------------
{quote}It seems that since the new version requires a snapshot to be present,
it would make sense for it to create one immediately upon successful startup
{quote}
Yes this is the expected behavior and because of this, disabling the
snapshot.trust.empty after upgrade should be ok because upgrade should involve
at least one restart after which the new version of ZK should kick in and
generate the snapshot.
bq. I'm not sure why it doesn't on an upgrade?
As we discussed earlier, this should not happen if the upgrade was successful.
Can we check the following:
* Was the upgrade successful? What's the version of the running ZK process
after upgrade?
* If ZK version is correct, check the file system to make sure it's indeed the
snapshot file was missing, instead of some IO / permission issue.
* If snapshot file is missing but ZK process is in right version (3.5.x or
3.6.x), this should be a bug and we'd need look further - a full server log
will be helpful in this case to help investigation.
> upgrade from 3.4.x to 3.5.x
> ---------------------------
>
> Key: ZOOKEEPER-3826
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3826
> Project: ZooKeeper
> Issue Type: Bug
> Components: server
> Affects Versions: 3.5.7
> Environment: Kuberenetes
> Reporter: Aldan Brito
> Priority: Critical
>
> upgrade of zookeeper from 3.4.14 to 3.5.7
> We faced the snapshot issue which is described in
> https://issues.apache.org/jira/browse/ZOOKEEPER-3056
> After setting the property "snapshot.trust.empty=true" the upgrade was
> successful.
> while reverting the "snapshot.trust.empty=false" flag and restart of the
> zookeeper pods, one of the zookeeper server is failing with the similar stack
> trace no snapshot found.
> {code:java}
> {"type":"log", "host":"zk-testzk-0", "level":"ERROR",
> "neid":"zookeeper-4636c00bfc3849e0be179bc71cef17f8", "system":"zookeeper",
> "time":"2020-05-12T08:32:17.685Z", "timezone":"UTC", "log":{"message":"main -
> org.apache.zookeeper.server.quorum.QuorumPeer - Unable to load database on
> disk"}}
> java.io.IOException: No snapshot found, but there are log entries. Something
> is broken!
> at
> org.apache.zookeeper.server.persistence.FileTxnSnapLog.restore(FileTxnSnapLog.java:240)
> at
> org.apache.zookeeper.server.ZKDatabase.loadDataBase(ZKDatabase.java:240)
> at
> org.apache.zookeeper.server.quorum.QuorumPeer.loadDataBase(QuorumPeer.java:901)
> at
> org.apache.zookeeper.server.quorum.QuorumPeer.start(QuorumPeer.java:887)
> at
> org.apache.zookeeper.server.quorum.QuorumPeerMain.runFromConfig(QuorumPeerMain.java:205)
> at
> org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:123)
> at
> org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:82)
> {"type":"log", "host":"zk-testzk-0", "level":"ERROR",
> "neid":"zookeeper-4636c00bfc3849e0be179bc71cef17f8", "system":"zookeeper",
> "time":"2020-05-12T08:32:17.764Z", "timezone":"UTC", "log":{"message":"main -
> org.apache.zookeeper.server.quorum.QuorumPeerMain - Unexpected exception,
> exiting abnormally"}}
> java.lang.RuntimeException: Unable to run quorum server
> at
> org.apache.zookeeper.server.quorum.QuorumPeer.loadDataBase(QuorumPeer.java:938)
> at
> org.apache.zookeeper.server.quorum.QuorumPeer.start(QuorumPeer.java:887)
> at
> org.apache.zookeeper.server.quorum.QuorumPeerMain.runFromConfig(QuorumPeerMain.java:205)
> at
> org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:123)
> at
> org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:82)
> Caused by: java.io.IOException: No snapshot found, but there are log entries.
> Something is broken!
> at
> org.apache.zookeeper.server.persistence.FileTxnSnapLog.restore(FileTxnSnapLog.java:240)
> at
> org.apache.zookeeper.server.ZKDatabase.loadDataBase(ZKDatabase.java:240)
> at
> org.apache.zookeeper.server.quorum.QuorumPeer.loadDataBase(QuorumPeer.java:901)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
