Repository: impala
Updated Branches:
  refs/heads/2.x c3e993327 -> 3d7d8209e


IMPALA-6489: use correct template tuple size

The bug was that we mixed up the size of the top-level tuple with the
size of the nested tuple. The upshot in this case was that the wrong
amount of data was memcpy()ed over and we read past the bounds of the
original allocation.

Testing:
TestParquetArrayEncodings.test_avro_primitive_in_list reliably
reproduced the problem under ASAN. After the fix it not longer
reproduces.

Change-Id: I8193c04673f15e5057f457cc8a3a91a8fef64be2
Reviewed-on: http://gerrit.cloudera.org:8080/9288
Reviewed-by: Alex Behm <alex.b...@cloudera.com>
Reviewed-by: Dan Hecht <dhe...@cloudera.com>
Tested-by: Impala Public Jenkins


Project: http://git-wip-us.apache.org/repos/asf/impala/repo
Commit: http://git-wip-us.apache.org/repos/asf/impala/commit/3660122c
Tree: http://git-wip-us.apache.org/repos/asf/impala/tree/3660122c
Diff: http://git-wip-us.apache.org/repos/asf/impala/diff/3660122c

Branch: refs/heads/2.x
Commit: 3660122ca2911e187bc4e957f7d3cefdfec9f2f5
Parents: c3e9933
Author: Tim Armstrong <tarmstr...@cloudera.com>
Authored: Mon Feb 12 14:23:17 2018 -0800
Committer: Impala Public Jenkins <impala-public-jenk...@gerrit.cloudera.org>
Committed: Tue Feb 13 04:10:14 2018 +0000

----------------------------------------------------------------------
 be/src/exec/hdfs-scanner.h | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/impala/blob/3660122c/be/src/exec/hdfs-scanner.h
----------------------------------------------------------------------
diff --git a/be/src/exec/hdfs-scanner.h b/be/src/exec/hdfs-scanner.h
index e3c186f..6497457 100644
--- a/be/src/exec/hdfs-scanner.h
+++ b/be/src/exec/hdfs-scanner.h
@@ -428,7 +428,7 @@ class HdfsScanner {
   void IR_ALWAYS_INLINE InitTuple(
       const TupleDescriptor* desc, Tuple* template_tuple, Tuple* tuple) {
     if (has_template_tuple(template_tuple)) {
-      InitTupleFromTemplate(template_tuple, tuple, tuple_byte_size());
+      InitTupleFromTemplate(template_tuple, tuple, tuple_byte_size(*desc));
     } else {
       tuple->ClearNullBits(desc->null_bytes_offset(), desc->num_null_bytes());
     }
@@ -479,6 +479,9 @@ class HdfsScanner {
 
   /// Not inlined in IR so it can be replaced with a constant.
   int IR_NO_INLINE tuple_byte_size() const { return tuple_byte_size_; }
+  int IR_NO_INLINE tuple_byte_size(const TupleDescriptor& desc) const {
+    return desc.byte_size();
+  }
 
   /// Returns true iff 'template_tuple' is non-NULL.
   /// Not inlined in IR so it can be replaced with a constant.

Reply via email to