skyyyy955 opened a new pull request, #11346:
URL: https://github.com/apache/paimon/pull/11346

   ### Purpose
   
   Document an incorrect result that can occur when Spark pushes descending 
TopN into a scan of an older PyPaimon snapshot. If the stored file maximum is 
finite but the file contains `NaN`, the scan can skip that file. The Spark SQL 
guide now shows how to repartition the query input and verify the resulting 
plan. [Reproduction and 
discussion](https://github.com/apache/paimon/pull/11317#issuecomment-7065402886)
   
   ### Tests
   
   - Matched the documented SQL to the Spark 3.5.9 reproduction. On the same 
snapshot, the pushed-down scan read one file and returned `10.0`. With the 
repartition hint, Spark read both files and returned `NaN`. Verified the Paimon 
scan and physical plan for both cases.
   - Ran `yarn build` on Windows with the generated-HTML path matcher 
temporarily adjusted to accept backslashes. The REST OpenAPI checks and site 
build passed, and I verified the new section and SQL in the rendered page. 
Restored the build configuration afterward.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to