Repository: impala
Updated Branches:
  refs/heads/master 95f166630 -> 5f7599687


IMPALA-6489: use correct template tuple size

The bug was that we mixed up the size of the top-level tuple with the
size of the nested tuple. The upshot in this case was that the wrong
amount of data was memcpy()ed over and we read past the bounds of the
original allocation.

Testing:
TestParquetArrayEncodings.test_avro_primitive_in_list reliably
reproduced the problem under ASAN. After the fix it not longer
reproduces.

Change-Id: I8193c04673f15e5057f457cc8a3a91a8fef64be2
Reviewed-on: http://gerrit.cloudera.org:8080/9288
Reviewed-by: Alex Behm <alex.b...@cloudera.com>
Reviewed-by: Dan Hecht <dhe...@cloudera.com>
Tested-by: Impala Public Jenkins


Project: http://git-wip-us.apache.org/repos/asf/impala/repo
Commit: http://git-wip-us.apache.org/repos/asf/impala/commit/f4d7e264
Tree: http://git-wip-us.apache.org/repos/asf/impala/tree/f4d7e264
Diff: http://git-wip-us.apache.org/repos/asf/impala/diff/f4d7e264

Branch: refs/heads/master
Commit: f4d7e2644a7c3ab7450bc4fd7dfd274803fbf8dd
Parents: 95f1666
Author: Tim Armstrong <tarmstr...@cloudera.com>
Authored: Mon Feb 12 14:23:17 2018 -0800
Committer: Impala Public Jenkins <impala-public-jenk...@gerrit.cloudera.org>
Committed: Tue Feb 13 03:13:47 2018 +0000

----------------------------------------------------------------------
 be/src/exec/hdfs-scanner.h | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/impala/blob/f4d7e264/be/src/exec/hdfs-scanner.h
----------------------------------------------------------------------
diff --git a/be/src/exec/hdfs-scanner.h b/be/src/exec/hdfs-scanner.h
index e3c186f..6497457 100644
--- a/be/src/exec/hdfs-scanner.h
+++ b/be/src/exec/hdfs-scanner.h
@@ -428,7 +428,7 @@ class HdfsScanner {
   void IR_ALWAYS_INLINE InitTuple(
       const TupleDescriptor* desc, Tuple* template_tuple, Tuple* tuple) {
     if (has_template_tuple(template_tuple)) {
-      InitTupleFromTemplate(template_tuple, tuple, tuple_byte_size());
+      InitTupleFromTemplate(template_tuple, tuple, tuple_byte_size(*desc));
     } else {
       tuple->ClearNullBits(desc->null_bytes_offset(), desc->num_null_bytes());
     }
@@ -479,6 +479,9 @@ class HdfsScanner {
 
   /// Not inlined in IR so it can be replaced with a constant.
   int IR_NO_INLINE tuple_byte_size() const { return tuple_byte_size_; }
+  int IR_NO_INLINE tuple_byte_size(const TupleDescriptor& desc) const {
+    return desc.byte_size();
+  }
 
   /// Returns true iff 'template_tuple' is non-NULL.
   /// Not inlined in IR so it can be replaced with a constant.

Reply via email to