RE: Re: Two issues in ZooKeeper that might cause data inconsistency or committed data loss

[Sirius Alpha]

Hi Enrico, 

Thanks for your response! I will try to provide the code fix as soon as 
possible!

Best wishes,
Yang

On 2022/12/24 12:31:14 Enrico Olivelli wrote:
> Yang,
> Thanks for your report
> 
> 
> Il Mar 13 Dic 2022, 18:16 Yang Sirius  ha
> scritto:
> 
> > Hi everyone!
> >
> > Recently we discovered two issues in the ZooKeeper’s latest versions that
> > might cause data inconsistency or committed data loss. Details and analysis
> > of the issues are presented on JIRA:
> >
> >
> >- ZOOKEEPER-4643  :
> > Committed txns may be improperly truncated if follower crashes right
> >after updating currentEpoch but before persisting txns to disk.
> >- ZOOKEEPER-4646 
> > : Committed txns may still be lost if followers crash after replying
> >ACK-LD but before writing txns to disk. (This issue is related to the fix
> >of ZOOKEEPER-3911
> >)
> >
> >
> > The issues seem to be critical since they lead to data loss or
> > inconsistency, which violate the properties that ZAB is supposed to
> > satisfy. I wonder whether the bugs should get a fix since data consistency
> > is of prime importance of ZooKeeper. If so, I will try to fix the code
> > together with further testing and verification techniques.
> >
> 
> Help is always welcome!
> 
> I personally don't have time to investigate and code a fix, but I will be
> happy to review your work
> 
> Thank you very much
> 
> Sharing problems and solutions is fundamental for an OSS community like
> Apache ZooKeeper
> 
> Cheers
> Enrico
> 
> 
> 
> > Thanks!
> >
> > Attached are example traces of these two issues that have been generated
> > in multiple versions such as 3.8.0 & 3.7.1. (The traces are also provided
> > on JIRA.)
> > Trace-ZK-4643:
> > Trace-ZK-4646:
> >
> 

Re: Two issues in ZooKeeper that might cause data inconsistency or committed data loss

[Enrico Olivelli]

Yang,
Thanks for your report


Il Mar 13 Dic 2022, 18:16 Yang Sirius  ha
scritto:

> Hi everyone!
>
> Recently we discovered two issues in the ZooKeeper’s latest versions that
> might cause data inconsistency or committed data loss. Details and analysis
> of the issues are presented on JIRA:
>
>
>- ZOOKEEPER-4643  :
> Committed txns may be improperly truncated if follower crashes right
>after updating currentEpoch but before persisting txns to disk.
>- ZOOKEEPER-4646 
> : Committed txns may still be lost if followers crash after replying
>ACK-LD but before writing txns to disk. (This issue is related to the fix
>of ZOOKEEPER-3911
>)
>
>
> The issues seem to be critical since they lead to data loss or
> inconsistency, which violate the properties that ZAB is supposed to
> satisfy. I wonder whether the bugs should get a fix since data consistency
> is of prime importance of ZooKeeper. If so, I will try to fix the code
> together with further testing and verification techniques.
>

Help is always welcome!

I personally don't have time to investigate and code a fix, but I will be
happy to review your work

Thank you very much

Sharing problems and solutions is fundamental for an OSS community like
Apache ZooKeeper

Cheers
Enrico



> Thanks!
>
> Attached are example traces of these two issues that have been generated
> in multiple versions such as 3.8.0 & 3.7.1. (The traces are also provided
> on JIRA.)
> Trace-ZK-4643:
> Trace-ZK-4646:
>

Two issues in ZooKeeper that might cause data inconsistency or committed data loss

[Yang Sirius]

Hi everyone!

Recently we discovered two issues in the ZooKeeper’s latest versions that might 
cause data inconsistency or committed data loss. Details and analysis of the 
issues are presented on JIRA:


  *   ZOOKEEPER-4643 :  
Committed txns may be improperly truncated if follower crashes right after 
updating currentEpoch but before persisting txns to disk.
  *   ZOOKEEPER-4646 : 
Committed txns may still be lost if followers crash after replying ACK-LD but 
before writing txns to disk. (This issue is related to the fix of 
ZOOKEEPER-3911)

The issues seem to be critical since they lead to data loss or inconsistency, 
which violate the properties that ZAB is supposed to satisfy. I wonder whether 
the bugs should get a fix since data consistency is of prime importance of 
ZooKeeper. If so, I will try to fix the code together with further testing and 
verification techniques.

Thanks!

Attached are example traces of these two issues that have been generated in 
multiple versions such as 3.8.0 & 3.7.1. (The traces are also provided on JIRA.)
Trace-ZK-4643:
Trace-ZK-4646:

Two issues in ZooKeeper that might cause data inconsistency or committed data loss

[Sirius Yang]

Hi!

Recently we discovered two issues in the ZooKeeper’s latest versions that might 
cause data inconsistency or committed data loss. Details and analysis of the 
issues are presented on JIRA:


  *   ZOOKEEPER-4643 :  
Committed txns may be improperly truncated if follower crashes right after 
updating currentEpoch but before persisting txns to disk.
  *   ZOOKEEPER-4646 : 
Committed txns may still be lost if followers crash after replying ACK-LD but 
before writing txns to disk. (This issue is related to the fix of 
ZOOKEEPER-3911)

The issues seem to be critical since they lead to data loss or inconsistency, 
which violate the properties that ZAB is supposed to satisfy. I wonder whether 
the bugs should get a fix since data consistency is of prime importance of 
ZooKeeper. If so, I will try to fix the code together with further testing and 
verification techniques.

Thanks!

Attached are example traces of these two issues that have been generated in 
multiple versions such as 3.8.0 & 3.7.1. (The traces are also provided on JIRA.)