JingsongLi commented on PR #10319: URL: https://github.com/apache/paimon/pull/10319#issuecomment-6135839850
The current head still has end-to-end compatibility value; I reran 51 focused partition/shard/REST FileIO tests and the real Java writer → Python reader/append → Java reader/rollback chain in both naming modes, with standard JDK 8 Maven checks. These passed. I also repeated the exact-baseline/current resolver comparison with production RESTTokenFileIO → PyArrowFileIO delegation and a counted, present-object backend: 128 canonical files add 0→128 get_file_info calls for DATE+escaped partitions and escaped STRING partitions, while plain STRING stays 0→0. This measures calls, not OSS latency. The previously reported production performance gate remains unresolved. For the strategy question, I recommend preserving a probe-free canonical fast path and explicitly opting affected tables into historical-layout compatibility, or adding a reliable per-file layout signal. A table-wide inferred writer version is insufficient for mixed layouts: current DataFileMeta has no historical/canonical directory marker, and schema IDs or the latest snapshot writer cannot identify the origin of every retained file. An explicit compatibility mode can probe each mixed file while new/already-migrated tables avoid the cost. Please cover the transition and retained snapshots/tags/rollback as well as mixed append. The current mixed tests establish current Java ↔ current Python only. Please declare and test the intended historical client/version matrix; synthetic old-directory coverage does not establish historical Java-reader support. I cannot infer a project-wide version floor or an accepted OSS latency budget from the current implementation. Until that boundary is explicit, keep this as a supported compatibility proposal with the existing production gate, rather than claiming broad historical interoperability or accepting the measured lookup cost implicitly. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
