Hello Michael Ho, Thomas Tauber-Marshall, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/14677
to look at the new patch set (#4).
Change subject: IMPALA-9137, IMPALA-9138: Mark failed RPCs as retryable and
update blacklist
......................................................................
IMPALA-9137, IMPALA-9138: Mark failed RPCs as retryable and update blacklist
Introduces two optional fields to TStatus: TStatusProperties and
TRetryInfo. TStatusProperties is a struct of properties used to describe
the TStatus object. TRetryInfo is a struct containing all error
information related to query retries. It currently has only one field:
TRPCErrorMessage, which is set if the TStatus was created as the result
of a failed RPC call. It contains the TNetworkAddress of the destination
node of the failed RPC. When a Status object is created, SetIsRetryable()
can be used to mark the Status as retryable and
SetRPCErrorMessage(RPCErrorMessage) can be used to add a TRPCErrorMessage
to a Status object.
When the Coordinator updates the state of an Impala backend
(Coordinator::UpdateBackendExecStatus), if it receives a Status where
Status::HasRPCErrorMessage() is true, it takes the RPCErrorMessage
destination node, and adds it to the blacklist.
Currently, if a Status is marked as retryable (Status::SetIsRetryable()),
nothing happens. The change is simply meant to lay the groundwork
for future changes in IMPALA-9124.
Only RPC failures in KrpcDataStreamSender are marked as retryable and
have a RPCErrorMessage set.
Re-factored the Thrift files a bit and added a Common.thrift file for
all commonly used Thrift structures. Started doing some re-factoring of
Status to break up all the information encapsulated in TErrorCode.
Testing:
* Ran core tests
* Planning to add more tests after IMPALA-8138 is merged
Change-Id: I733cca13847fde43c8ea2ae574d3ae04bd06419c
---
M be/generated-sources/gen-cpp/CMakeLists.txt
M be/src/common/status.cc
M be/src/common/status.h
M be/src/runtime/coordinator.cc
M be/src/runtime/coordinator.h
M be/src/runtime/krpc-data-stream-sender.cc
M be/src/service/client-request-state.cc
M be/src/util/container-util.h
M be/src/util/error-util.h
M common/protobuf/common.proto
M common/thrift/CMakeLists.txt
M common/thrift/CatalogObjects.thrift
M common/thrift/CatalogService.thrift
A common/thrift/Common.thrift
M common/thrift/Frontend.thrift
M common/thrift/ImpalaInternalService.thrift
M common/thrift/StatestoreService.thrift
M common/thrift/Status.thrift
M common/thrift/Types.thrift
M common/thrift/generate_error_codes.py
M tests/custom_cluster/test_custom_statestore.py
M tests/statestore/test_statestore.py
22 files changed, 318 insertions(+), 36 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/77/14677/4
--
To view, visit http://gerrit.cloudera.org:8080/14677
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I733cca13847fde43c8ea2ae574d3ae04bd06419c
Gerrit-Change-Number: 14677
Gerrit-PatchSet: 4
Gerrit-Owner: Sahil Takiar <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Michael Ho <[email protected]>
Gerrit-Reviewer: Thomas Tauber-Marshall <[email protected]>
