JingsongLi commented on PR #8574: URL: https://github.com/apache/paimon/pull/8574#issuecomment-5083336466
The new implementation calculates the number of rows by subtracting the DV cardinality from the original number of rows in the data file; however, the Chain Table still merges and deduplicates rows across snapshot and delta files based on the primary key, so this count is not the final row count. Furthermore, `buildChainSplits` always constructs a non-empty list of deletion-files; even if all elements are null, this causes almost all ChainSplits to return a “definite” row count. Based on this, `AbstractBatchTableScan` assumes that the first few splits have already met the LIMIT and may discard subsequent splits, resulting in the final actual merged row count falling short of the LIMIT. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
