Philip Zeyliger has uploaded a new patch set (#2).
Change subject: IMPALA-3642: Adding backend addresses to error statuses for
some scratch failures.
......................................................................
IMPALA-3642: Adding backend addresses to error statuses for some scratch
failures.
Adds GetBackendAddress() (which is host:port) to error messages stemming
from SCRATCH_LIMIT_EXCEEDED, SCRATCH_READ_TRUNCATED, and
SCRATCH_ALLOCATION_FAILED messages.
Testing:
* Unit tests assert the string is updated for SCRATCH_LIMIT_EXCEEDED
and SCRATCH_ALLOCATION_FAILED. SCRATCH_READ_TRUNCATED doesn't
have an existing test, and I didn't add a new one.
* Manually testing a query that spills after "chmod 000 /tmp/impala-scratch":
$ chmod 000 /tmp/impala-scratch
$ impala-shell
[dev:21000] > set mem_limit=100m;
MEM_LIMIT set to 100m
[dev:21000] > select count(*) from tpch_parquet.lineitem join
tpch_parquet.orders on l_orderkey = o_orderkey;
Query: select count(*) from tpch_parquet.lineitem join tpch_parquet.orders
on l_orderkey = o_orderkey
Query submitted at: 2017-09-11 11:07:06 (Coordinator: http://dev:25000)
Query progress can be monitored at:
http://dev:25000/query_plan?query_id=5c48ff8f4103c194:1b40a6c00000000
WARNINGS: Could not create files in any configured scratch directories
(--scratch_dirs=/tmp/impala-scratch) on backend 'dev:22002'. See logs for
previous errors that may have prevented creating or writing scratch files.
Opening
'/tmp/impala-scratch/5c48ff8f4103c194:1b40a6c00000000_08e8d63b-169d-4571-a0fe-c48fa08d73e6'
for write failed with errno=13 description=Error(13): Permission denied
Opening
'/tmp/impala-scratch/5c48ff8f4103c194:1b40a6c00000000_08e8d63b-169d-4571-a0fe-c48fa08d73e6'
for write failed with errno=13 description=Error(13): Permission denied
Opening
'/tmp/impala-scratch/5c48ff8f4103c194:1b40a6c00000000_08e8d63b-169d-4571-a0fe-c48fa08d73e6'
for write failed with errno=13 description=Error(13): Permission denied
Change-Id: If31a50fdf6031312d0348d48aeb8f9688274cac2
---
M be/src/runtime/bufferpool/buffer-pool-test.cc
M be/src/runtime/tmp-file-mgr-test.cc
M be/src/runtime/tmp-file-mgr.cc
M common/thrift/generate_error_codes.py
4 files changed, 13 insertions(+), 9 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/16/7816/2
--
To view, visit http://gerrit.cloudera.org:8080/7816
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: If31a50fdf6031312d0348d48aeb8f9688274cac2
Gerrit-PatchSet: 2
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Philip Zeyliger <[email protected]>
Gerrit-Reviewer: Philip Zeyliger <[email protected]>
Gerrit-Reviewer: Tim Armstrong <[email protected]>
