kz930 opened a new pull request, #7247: URL: https://github.com/apache/texera/pull/7247
### What changes were proposed in this PR? `JSONLScanSourceOpExec.open()` applies the requested offset twice. It drops `offset` lines from the reader, so the iterator it keeps already begins at the requested row; it then computes the worker's slice bounds as `offsetValue + …`, which are positions in the original file, and applies those bounds to that already-offset iterator. The rows in between are skipped, and when what remains is shorter than the offset the operator emits nothing at all. Neither case raises an error or writes a log line, so the operator reports success with rows missing. This PR resolves the bounds against the iterator they are actually applied to, by dropping the `offsetValue +` from both expressions. That is the first of the two options the issue describes; it keeps the reader-side `drop` doing the offset and leaves the slice to do only what it exists for, which is dividing the window across workers. ### Any related issues, documentation, discussions? Closes #7245 ### How was this PR tested? Two cases were added to `JSONLScanSourceOpExecSpec`, which previously covered the row limit but never set an offset: an offset on its own, and an offset together with a limit — the combination that used to yield no rows at all. Both fail against the current code and pass with the change, and the three existing cases are unaffected. Beyond those, the operator was driven over a five-line file across every combination of offset (unset, 0, 1, 2, 4, 5, 7), limit (unset, 0, 2, 3, 9) and worker count (1, 2, 3), with each result compared against `drop(offset).take(limit)` — the reading that `ArrowSourceOpExec` and this operator's own `sourceSchema()` already follow. 36 of the 105 combinations were wrong before the change and none after. Every one of the 36 had an offset of 1 or more, which is why an unset offset was unaffected and the gap went unnoticed. That sweep was a scratch harness for checking the fix and is not part of the diff; the two added cases are the representatives of it that are worth keeping. ### Was this PR authored or co-authored using generative AI tooling? Generated-by: Claude Code (Claude Opus 5) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
