malinjawi opened a new pull request, #12563: URL: https://github.com/apache/gluten/pull/12563
## What changes are proposed in this pull request? Three fixes in the ValueStream scan path: 1. **Unbounded recursion on empty batches.** `ValueStreamDataSource::next()` retried by calling itself — an exhausted iterator, a null batch, or a batch fully eliminated by a dynamic filter each recursed, so a long run of filtered-out batches could grow the native stack without bound. The retry is now a loop. 2. **Finished-split signaling.** When no iterators remain, return an explicit engaged-null `RowVectorPtr` so `TableScan` reads it as "current split finished", instead of relying on the implicit conversion of `nullptr` into the `std::optional` return type. 3. **Null/zero-row batch normalization.** `RowVectorStream::next()` converted every batch unconditionally: a null batch from the upstream iterator would crash the conversion (latent today, given the iterator's peek-and-cache contract), and zero-row batches cost an extra scan round-trip each. Both are now normalized to `nullptr` and skipped by the data-source loop. Also adds the stream-index bounds check to `constructValueStreamNode()` that `constructValuesNode()` and `constructCudfValueStreamNode()` already perform (skipped in validation mode, where no input iterators are attached). ## How was this patch tested? New cases in `ValueStreamDynamicFilterTest` cover consecutive fully-filtered batches, finished-split signaling, and a filter/project pipeline over a value stream; they ride the existing `velox_operators_test` target. ## Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Code (reviewed and tested before submission) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
