Impala Public Jenkins has submitted this change and it was merged. Change subject: IMPALA-4539: fix bug when scratch batch references I/O buffers ......................................................................
IMPALA-4539: fix bug when scratch batch references I/O buffers The bug occurs when scanning uncompressed plain-encoded Parquet string data. Testing: I could manually reproduce it reliably with ASAN and --disable_mem_pools=true using the following steps: # The repro is more predictable when there is no row batch queue. set mt_dop=1; # A unique string column should switch to plain encoding set compression_codec=none; create table big_uuids stored as parquet as select uuid() from tpch_20_parquet.lineitem; # The repro requires that some rows are filtered out, so that we end # up in a state where the output batch is full before all rows are # copied from the scratch batch select ndv(_c0) from big_uuids where substring(_c0, 1, 2) != 'a1' limit 10; After the fix it no longer reproduces. I do not yet have a practical test case that triggers the bug on a normal ASAN setup. I will continue to try to create one. Change-Id: Ic27e7251e0f633cb694b506f6eb62beed6e66ad9 Reviewed-on: http://gerrit.cloudera.org:8080/5406 Reviewed-by: Tim Armstrong <[email protected]> Tested-by: Impala Public Jenkins --- M be/src/exec/hdfs-parquet-scanner.cc 1 file changed, 10 insertions(+), 8 deletions(-) Approvals: Impala Public Jenkins: Verified Tim Armstrong: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/5406 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: Ic27e7251e0f633cb694b506f6eb62beed6e66ad9 Gerrit-PatchSet: 5 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Tim Armstrong <[email protected]> Gerrit-Reviewer: Alex Behm <[email protected]> Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Marcel Kornacker <[email protected]> Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Michael Ho <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]>
