kecookier opened a new issue, #12985: URL: https://github.com/apache/gluten/issues/12985
### Backend VL (Velox) ### Bug description ### Description We hit two bugs in the rss_sort shuffle reader (`VeloxRssSortShuffleReaderDeserializer::VeloxInputStream`) in our production environment with Celeborn (rss_sort + hash partitioning). Both reproduce on current main — regression tests submitted in #12983 fail: https://github.com/apache/gluten/pull/12983 ### Bug 1: infinite loop on EOS mid-page `GlutenByteInputStream::readBytes()` drives `next(true)` in a `for(;;)` loop. When a page header declares more bytes than the stream actually holds (truncated partition data), the reader hits EOS mid-page — but `VeloxInputStream::next()` ignores its `throwIfPastEnd` argument and silently returns on EOS, so the loop never exits. Symptom in production: the task appears stuck (thread in RUNNABLE, no progress, no error surfaced). CI evidence from #12983: ``` C++ exception with description "possible infinite loop: Read() returned 0 for 100 consecutive calls" thrown in the test body. ``` (the test's fake stream cuts the loop short after 100 consecutive EOS reads instead of hanging) Additionally, `next()` stores the `Read()` result into an unsigned `offset_` without a signed guard, so a negative return from a buggy upstream corrupts `setRange` instead of failing fast. ### Bug 2: corrupted deserialization when a page spans multiple read windows `VeloxInputStream` is a single-window refill stream: each `next()` overwrites the sole `ByteRange` with serde assumes stable multi-range data — for nested types it does `tellp()` → pre-scan the page →`seekp()` back, and it scans the payload via `nextView()` for checksum verification. When an uncompressed page (nested structs) spans multiple ~1MB read windows, the serde's backward seek reads window data already overwritten by a refill, corrupting the stream. CI evidence from #12983: ``` Exception: VeloxRuntimeError Reason: (-673841146 vs. 0) Invalid serialized string size: -673841146 Function: readLengthPrefixedString # 3 ...readLengthPrefixedString(ByteInputStream*) # 6 ...PrestoVectorSerde::deserialize(...) # 9 gluten::VeloxRssSortShuffleReaderDeserializer::next() ``` ### Impact - Bug 1: task hangs forever with no error — requires external timeout to kill, hard to diagnose from logs. - Bug 2: wrong data / spurious deserialization errors for pages larger than one read window (~1MB) containing nested types. ### Fix We have a fix ready and will submit a PR shortly (referencing this issue). ### Gluten version main branch, Gluten-1.3 ### Spark version Spark-3.5.x ### Spark configurations _No response_ ### System information _No response_ ### Relevant logs ```bash ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
