skyyyy955 commented on PR #11317: URL: https://github.com/apache/paimon/pull/11317#issuecomment-7059756507
Thanks for clarifying the scope of the writer changes. I found a related TopN case on an older snapshot. With PyPaimon at `7a30ef580e` and `metadata.stats-mode=full`, I wrote `[NaN, 1.0]` and `[10.0]` in separate commits. The manifest recorded file maxima of `1.0` and `10.0`. On Spark 3.5.9 with the published Paimon connector `2.2-20261120.000749-68`, I ran `ORDER BY score DESC LIMIT 1` against snapshot 2. With TopN in the Paimon scan, Spark read only the second file and returned `10.0`. Adding `REPARTITION(1)` to the input view kept TopN out of the scan. The same query then read both files and returned `NaN`. Removing the hint reproduced `10.0`. Writing the same rows with `e916d09c12` produced byte-identical Parquet files, but the manifest left the NaN file's upper bound unknown. Both plans returned `NaN` for that table. Would it make sense to track TopN on these existing manifests in a separate issue? I can include the two physical plans and the reproducer there. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
