rakeshadr commented on PR #3379: URL: https://github.com/apache/ozone/pull/3379#issuecomment-1117489674
Thanks a lot @aswinshakil for the continuous efforts in analysing and digging out the root cause. It looks like OM TableCache is tricky for the FSO or non-FSO listing status code path, which would create inconsistency across batches. _Please go through the following scenario:_ Say, below keys exists in OM. These keys `"x/y/z/a1"...."x/y/z/a1023"` and `"x/y/z/b1"..."x/y/z/b1023"` in DBTable. Assume, `"x/y/z/b1024"` only exists in TableCache. Now, Ozone client Listing Status with keyName = `"x/y/z/"`: BatchSize = 1024. It will take two iteration to fetch all the values from OM. **Iteration-1:** keyName = `"x/y/z/"` and startKey = `""` step-1) Seek all the elements from TableCache if its not a deleted key. Now, the Batch-1 [`"x/y/z/b1024"`] step-2) Seek all the elements from Now, the Batch-1 [`"x/y/z/a1"...."x/y/z/a1023"`] FinalList of Batch-1 will be a combined values : [`"x/y/z/a1"...."x/y/z/a1023, "x/y/z/b1024"`]. **Iteration-2:** keyName = `"x/y/z/"` and startKey = `"x/y/z/b1024"`, which is the last element in the batch-1. FinalList of Batch-2 : Empty list because there is no elements after `"x/y/z/b1024".` Since `"x/y/z/b1"..."x/y/z/b1023"` exists before `"x/y/z/b1024"` start key, all these keys would be skipped and won't get fetched/seeks from DBTable. **Proposal:** How about to use TableCache only for checking the deleted keys and do not add the cached values into the listing keys ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
