Akash3121 opened a new issue, #9982: URL: https://github.com/apache/paimon/issues/9982
### Search before asking - [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar. ### Paimon version master (1d368b4a5932f8221fd28e2555001abdb8fb12ee) ### Compute Engine Flink SQL. The problem is in Paimon's Flink source limit pushdown. ### Minimal reproduce step ```SQL CREATE TABLE t ( id INT, data STRING ) WITH ( 'connector' = 'paimon' ); INSERT INTO t VALUES (1, 'a'), (2, 'b'), (3, 'c'); SELECT * FROM t LIMIT 4294967297; ``` `FlinkTableSource.applyLimit(long)` accepts and stores a `long` , but `FlinkSourceBuilder#createReadBuilder` narrows it with: ``` readBuilder.withLimit(limit.intValue()); ``` For 4294967297 , Long.intValue() returns 1 . The internal scan can therefore stop after one row even though the requested limit is greater than the total row count. Relevant code: - paimon-flink/paimon-flink-common/.../FlinkTableSource.java : applyLimit(long) - paimon-flink/paimon-flink-common/.../FlinkSourceBuilder.java : createReadBuilder - paimon-core/.../ReadBuilderImpl.java : the internal limit is stored as int ### What doesn't meet your expectations? A valid large LIMIT must not reduce the result set. Since the table contains only three rows and the requested limit is 4294967297, all three rows should be returned. ### Anything else? A safe fix would push the limit into the int-based ReadBuilder only when the value is less than or equal to Integer.MAX_VALUE. For larger values, Paimon can skip the internal optimization and retain the long-valued Flink/downstream limit. Suggested tests: 1. LIMIT Integer.MAX_VALUE 2. LIMIT Integer.MAX_VALUE + 1 3. LIMIT 4294967297, whose narrowed int value is 1 4. A multi-split table to ensure the source does not stop early ### Are you willing to submit a PR? - [x] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
