igorbernstein2 commented on code in PR #30147:
URL: https://github.com/apache/beam/pull/30147#discussion_r1473272446
##########
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableServiceImpl.java:
##########
@@ -395,7 +395,14 @@ public void onResponse(Row response) {
@Override
public void onError(Throwable t) {
- future.setException(t);
+ if (byteLimitReached) {
+ // When the byte limit is reached we cancel the stream in
onResponse.
+ // In this case we don't want to fail the request with
cancellation
+ // exception. Instead, we construct the next request.
+ onComplete();
Review Comment:
In general, here is an expected scenario for server streaming rpcs:
- client consumes all of the messages, but before it receives the grpc
trailers with an ok status, the deadline timer expires.
In this case, there is no reason to throw an error, as the customer got all
of their data.
In this particular case, there is no data loss as the next segment read will
because the next segment read will start from the last complete row (ie
truncateRequest() will start from the last complete row. So calling onComplete
when byteLimitReached is correct.
However there is another bug that I noticed: if a segment returns an empty
result, then you will end up with index out of bounds exception
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
