SteNicholas opened a new pull request, #3748:
URL: https://github.com/apache/celeborn/pull/3748
### What changes were proposed in this pull request?
The C++ `ShuffleClientImpl::readPartition` now returns an empty stream when
a shuffle has no locations for the requested partition, instead of constructing
a `CelebornInputStream` over zero locations. Specifically:
- Add a `CelebornInputStream::empty()` factory that builds a no-op stream
whose `read()` immediately yields `-1`.
- In `readPartition`, when `locations` is empty, log a warning and return
`CelebornInputStream::empty()`; the shuffle key is now built only on the
non-empty path.
Tests:
- A unit test for `CelebornInputStream::empty()` (immediate EOF, repeated
reads, zero-length read).
- Two `readPartition` tests (empty file groups, missing partition) backed by
a `@VisibleForTesting` seam that seeds the cached file-group response so no RPC
to the `LifecycleManager` is needed.
### Why are the changes needed?
For a partition with no shuffle data, the previous code constructed a full
reader over an empty location list (also building the decompressor when
compression is enabled). Returning an empty stream short-circuits this and
makes the empty-shuffle case explicit, consistent with the read path's intent.
### Does this PR resolve a correctness bug?
- [ ] Yes
### Does this PR introduce _any_ user-facing change?
- [ ] Yes
### How was this patch tested?
Added unit tests under `cpp/celeborn/client/tests/`:
- `CelebornInputStreamTest.emptyStreamReadsReturnEof`
- `ShuffleClientImplTest.readPartitionEmptyWhenFileGroupsEmpty`
- `ShuffleClientImplTest.readPartitionEmptyWhenPartitionMissing`
Run with:
```
cd cpp && mkdir -p build && cd build && cmake .. && make celeborn_client_test
./celeborn/client/tests/celeborn_client_test \
--gtest_filter='CelebornInputStreamTest.*:ShuffleClientImplTest.readPartition*'
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
