Liulietong commented on PR #8354: URL: https://github.com/apache/paimon/pull/8354#issuecomment-4800050935
Thanks for checking this. I looked at the lookup compaction logic again, and I think your understanding is reasonable. When I opened #7221, I mainly checked the local logic in `LookupMergeFunction.pickHighLevel()`. If we manually construct a case like `L1 seq=7, L2 seq=8, L0 seq=6`, then picking L1 by level looks wrong when `sequence.field` is configured. But I did not have a real case from normal write and compaction flow. In current lookup compaction, lower high level file should already be the merged result with higher level files. So if L2 has seq=8, and L0 is compacted to L1, the new L1 should already contain the result with seq=8. So I agree the test in #7221 is not enough, because it bypasses the real writer / level manager / compaction flow. I am OK to revert #7221. If later we can find a real reproduce case from normal write and compaction, we can add an integration test and fix it again. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
