Dan Hecht has posted comments on this change. Change subject: IMPALA-5890: Abort queries if scanner hits IO errors ......................................................................
Patch Set 5: (3 comments) http://gerrit.cloudera.org:8080/#/c/8011/5/be/src/exec/base-sequence-scanner.cc File be/src/exec/base-sequence-scanner.cc: PS5, Line 177: if (status.IsCancelled() || status.IsMemLimitExceeded()) return status; : : // Log error from file format parsing. : state_->LogError(ErrorMsg(TErrorCode::SEQUENCE_SCANNER_PARSE_ERROR, : stream_->filename(), stream_->file_offset(), : (stream_->eof() ? "(EOF)" : ""))); : : // Make sure errors specified in the status are logged as well : state_->LogError(status.msg()); : : // If abort on error then return, otherwise try to recover. : if (state_->abort_on_error()) return status; : : // Abort scan range for I/O related errors : if (status.code() == TErrorCode::SCANNER_CONTEXT_WRAPPED_IO_ERROR) { : eos_ = true; : return Status::OK(); : } it does seem like this could be written in terms of RuntimeState::LogOrReturnErrror(), right? I guess the one complication is that we're trying not to fail entirely, but instead mark eos_ in that case. But are we sure that returning error from GetNextInternal() won't do the same thing? I thought the scanners have a check higher up the callstack that would go ahead to the next scan range if abort_on_error==false and GetNext() returns (recoverable) error, but I could be mistaken. http://gerrit.cloudera.org:8080/#/c/8011/5/be/src/exec/scanner-context.cc File be/src/exec/scanner-context.cc: PS5, Line 194: Internal " would be best to keep "Internal error" on the same line so that it's more easily grep-able. http://gerrit.cloudera.org:8080/#/c/8011/5/be/src/exec/scanner-context.h File be/src/exec/scanner-context.h: Line 277: Status WrapIoError(const Status& status); rather than wrapping, is it not possible to make our IO code return a single (or some set of) unrecoverable IO error codes directly, that we can then identify as IO errors? -- To view, visit http://gerrit.cloudera.org:8080/8011 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I44dc95184c241fbcdbdbebad54339530680d3509 Gerrit-PatchSet: 5 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Lars Volker <l...@cloudera.com> Gerrit-Reviewer: Dan Hecht <dhe...@cloudera.com> Gerrit-Reviewer: Joe McDonnell <joemcdonn...@cloudera.com> Gerrit-Reviewer: Lars Volker <l...@cloudera.com> Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com> Gerrit-HasComments: Yes
