nickva commented on issue #5916: URL: https://github.com/apache/couchdb/issues/5916#issuecomment-4025967432
Another issue is how purge_seq is handled by clouseau/dreyfus indexes. When we initialize the purge checkpoint in https://github.com/apache/couchdb/blob/25e1269170e7d056d9c5242946224900eb5f759f/src/dreyfus/src/dreyfus_index.erl#L126 we also set the purge seq in the index: https://github.com/apache/couchdb/blob/25e1269170e7d056d9c5242946224900eb5f759f/src/dreyfus/src/dreyfus_util.erl#L352-L354 That is done with the idea that later on in https://github.com/apache/couchdb/blob/25e1269170e7d056d9c5242946224900eb5f759f/src/dreyfus/src/dreyfus_index_updater.erl#L90-L91 we can read that purge_seq from the index and pass that as the initial start purge sequence. However setting a purge_seq on an index then immediately reading it back, we won't get the value we set. Until it commits we'll get the old committed value. That is incorrect behavior as far how purging logic is concerned because of this sequence of steps when the purge sequence is non-0 for a db. For example is db purge seq is 42, the minimum value may be 12 1. When index is first created we set clouseau's purge seq = 42 2. In purge_index https://github.com/apache/couchdb/blob/25e1269170e7d056d9c5242946224900eb5f759f/src/dreyfus/src/dreyfus_index_updater.erl#L91 we read it right back from the index but instead of 42 we'll get 0 3. Folding purges starting from 0 will fail, because the minimum purge seq is 12, and to the purge checkpoint logic it looks like the index just skipped a bunch of purges and is now out of sync. Maybe instead we should use the local doc checkpoints to keep track of the index purge seq instead of using the index. We already have to track them in the checkpoints anyway. Or, we have to force the index to properly commit and a set_purge_seq checkpoint is initialized (I am not sure if we a synchronous commit on purge_seq set though...) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
