wangzhigang1999 commented on PR #10319: URL: https://github.com/apache/paimon/pull/10319#issuecomment-6073937918
Thanks for the update. I independently tested `1156834035` against `80c5ab9593`. The PR's Python reader passed all 256 tables in my cross-version, mixed-writer matrix, including the original composite-partition failure. Java still cannot read some historical Python layouts; that is an existing limitation, but the compatibility boundary needs to be explicit. The OSS performance regression is a concern. With 128 small Parquet files, median planning + full-read time increased from **1.71 to 2.24 seconds (+31%)** for a DATE composite partition, and from **1.47 to 2.47 seconds (+68%)** for an escaped STRING partition without DATE. These were three measured runs after warm-up on the same frozen files. Local full-read performance was roughly unchanged. The head adds path-existence checks even for canonical files, so the description's claim of no additional lookups needs updating. I do not think this is ready to merge. I closed my own attempt in #10045 because these compatibility and performance tradeoffs were unresolved. Please align on the design before making further code changes: which version combinations we support, how readers locate historical files, and what lookup cost is acceptable. Adding another fallback for each reported failure is not a sufficient compatibility strategy, especially when it slows down tables that already use canonical paths. Once the design is agreed, please implement it and provide reproducible correctness and performance results. Please also change the added Chinese comments and docstrings to English, consistent with the surrounding code. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
