Hello Daniel Becker, Riza Suminto, Gabor Kaszab, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/21148
to look at the new patch set (#2).
Change subject: IMPALA-12903: Querying virtual column FILE__POSITION for TEXT
and JSON tables crashes Impala
......................................................................
IMPALA-12903: Querying virtual column FILE__POSITION for TEXT and JSON tables
crashes Impala
Impala generates segmentation fault when it queries the virtual column
FILE__POSITION for TEXT or JSON tables. When the scanners that do not
support the FILE__POSITION virtual column detect its presence they
try to report an error and close themselves. The segfault is in the
scanners' Close() method when they try to dereference a NULL stream
object.
This patch simply adds NULL-checks in Close().
Alternatively we could detect the presence of FILE__POSITION during
planning in the HdfsScanNode, but doing it in the scanners lets us
handle more queries, e.g. queries that dynamically prune partitions
and the surviving partitions all have file formats that support
FILE__POSITION.
Testing:
* added negative tests to properly report the errors
* added tests for mixed file format tables
Change-Id: I8e1af8d526f9046aceddb5944da9e6f9c63768b0
---
M be/src/exec/json/hdfs-json-scanner.cc
M be/src/exec/text/hdfs-text-scanner.cc
M
testdata/workloads/functional-query/queries/QueryTest/virtual-column-file-position-generic.test
A
testdata/workloads/functional-query/queries/QueryTest/virtual-column-file-position-negative.test
M tests/query_test/test_scanners.py
5 files changed, 92 insertions(+), 3 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/48/21148/2
--
To view, visit http://gerrit.cloudera.org:8080/21148
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I8e1af8d526f9046aceddb5944da9e6f9c63768b0
Gerrit-Change-Number: 21148
Gerrit-PatchSet: 2
Gerrit-Owner: Zoltan Borok-Nagy <borokna...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <daniel.bec...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <gaborkas...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <riza.sumi...@cloudera.com>
