Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/18528 )
Change subject: IMPALA-11208: Fix uninitialized counter of CollectionItemsRead in orc-scanner ...................................................................... IMPALA-11208: Fix uninitialized counter of CollectionItemsRead in orc-scanner CollectionItemsRead in the runtime profile counts the total number of nested collection items read by the scan node. Only created for scans that support nested types, e.g. Parquet or ORC. Each scanner thread maintains its local counter and merges it into HdfsScanNode counter for each row batch. However, the local counter in orc-scanner is uninitialized, leading to weird values. This patch simply initializes it to 0 and adds test coverage. Tests: Add profile verification for this counter on some existing query tests. Note that there are some implementation difference between Parquet and ORC scanners (e.g. in predicate pushdown). So we will see different counter results in some query. I just pick some queries that have consistent counters. Change-Id: Id7783d1460ac9b98e94d3a31028b43f5a9884f99 Reviewed-on: http://gerrit.cloudera.org:8080/18528 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M be/src/exec/hdfs-orc-scanner.h M testdata/workloads/functional-query/queries/QueryTest/nested-types-scanner-array-materialization.test M testdata/workloads/functional-query/queries/QueryTest/nested-types-tpch.test M testdata/workloads/tpch/queries/tpch-q1.test M testdata/workloads/tpch/queries/tpch-q10.test M testdata/workloads/tpch/queries/tpch-q11.test M testdata/workloads/tpch/queries/tpch-q12.test M testdata/workloads/tpch/queries/tpch-q13.test M testdata/workloads/tpch/queries/tpch-q14.test M testdata/workloads/tpch/queries/tpch-q15.test M testdata/workloads/tpch/queries/tpch-q16.test M testdata/workloads/tpch/queries/tpch-q17.test M testdata/workloads/tpch/queries/tpch-q18.test M testdata/workloads/tpch/queries/tpch-q19.test M testdata/workloads/tpch/queries/tpch-q2.test M testdata/workloads/tpch/queries/tpch-q20.test M testdata/workloads/tpch/queries/tpch-q21.test M testdata/workloads/tpch/queries/tpch-q22.test M testdata/workloads/tpch/queries/tpch-q3.test M testdata/workloads/tpch/queries/tpch-q4.test M testdata/workloads/tpch/queries/tpch-q5.test M testdata/workloads/tpch/queries/tpch-q6.test M testdata/workloads/tpch/queries/tpch-q7.test M testdata/workloads/tpch/queries/tpch-q8.test M testdata/workloads/tpch/queries/tpch-q9.test 25 files changed, 73 insertions(+), 19 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/18528 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Id7783d1460ac9b98e94d3a31028b43f5a9884f99 Gerrit-Change-Number: 18528 Gerrit-PatchSet: 4 Gerrit-Owner: Quanlong Huang <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Daniel Becker <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]>
