acrites commented on PR #41020: URL: https://github.com/apache/beam/pull/41020#issuecomment-6666256542
I asked for a comparison of `DecodingIterator` vs. `DataStreamDecoder`: | Feature | UncachedStateIterable.DecodingIterator | DataStreams.DataStreamDecoder | | --- | --- | --- | | Stream Concatenation | ❌ None. Decodes directly from single-chunk InputStream. | ✅ Continuous stream. Custom Inbound InputStream stitches chunks together. | | Cross-Chunk Elements | ❌ Fails. Throws EOFException / IllegalStateException if an element crosses chunk boundaries. | ✅ Supported. Reads across chunk boundaries seamlessly. | | Elements Larger Than Chunk Size | ❌ Fails. Cannot decode any element larger than a single gRPC buffer chunk. | ✅ Supported. Can read multi-chunk elements of arbitrary size. | | Zero-Byte Elements | ❌ Risks infinite loop. Does not detect when a coder consumes 0 bytes. | ✅ Handled. Detects 0-byte consumption and reads a sentinel byte to advance. | | Empty Chunks in isReady() | ⚠️ Can return true when next chunk is empty ByteString. | ✅ Loops until a non-empty chunk, EOF, or unready stream is encountered. | | Batch Boundary Decoding | ❌ Only supports element-by-element next(). | ✅ Provides decodeFromChunkBoundaryToChunkBoundary() for page-level caching. | | Reusability & Testing | ❌ Private static class in StateFetchingIterators; 0% dedicated unit tests. | ✅ Public utility in sdks/java/core; full dedicated test suite (DataStreamsTest). | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
