discivigour commented on PR #7866: URL: https://github.com/apache/paimon/pull/7866#issuecomment-4550992166
Thank you for sharing this feature. I believe my feature can already support the following functions well: 1. The LSM Tree was introduced to achieve incremental sorting, uniformly sorting existing files and newly arrived files. 2. Integrate the three functions of merging small files, eliminating add and delete pairs, and manifest sorting into this feature. The original full compaction and minor compaction would undermine the sorting effect. Moreover, I think the current PR has the following issues: 1. It does not reorder the large number of existing manifests. 2. The manifest sort function is limited by the trigger frequency of full compaction; if only minor compaction is performed, manifest sort cannot be triggered. 3. Manifest sort only sorts small files and delete-related files, so the sorting is not sufficient. 4. Minor compaction will destroy the sorting effect. To avoid conflicts and duplicated implementation, I’ll continue to improve my PR and port any useful parts if needed, with credit. Thanks for understanding! However, thank you very much for your contribution! If you have any suggestions or find areas that need improvement in my PR #7842 , please feel free to continue providing review comments. Your feedback is very helpful for my work -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
