Paul Rogers has uploaded this change for review. ( http://gerrit.cloudera.org:8080/12248
Change subject: IMPALA-8095: Detailed expression cardinality tests ...................................................................... IMPALA-8095: Detailed expression cardinality tests Cardinality is a critical input to the query planning process, especially join planning. Impala has many high-level end-to-end tests that implicitly test cardinality at the "wholesale" level: A test will produce a wrong result if the cardinality is badly wrong. Until this patch, no detailed unit tests exist for this topic. Tests here include: * Table cardinality, NDV values and null count in metadata retrieved from HMS. * Table cardinality, NDV values and null counts in metadata presented to the query. * Expression NDV and selectivity values (which derive from table cardinality and column NDV.) The test illustrate a number of bugs. This patch simply identifies the bugs, comments out the tests that fail because of the bugs, and substitutes tests that pass with the current, incorrect, behavior. Future patches will fix the bugs. Reviewers can note the difference between the original, incorrect behavior shown here, and the revised behavior in those additional patches. Since none of the existing "functional" tables provide the level of detail needed for these tests, added a new test table specifically for this task. This set of tests was a good time to extend the test "fixture" framework created earlier. The FrontendTestBase class was refactored to use a new FrontendFixture which represents a (simulated) Impala and HMS cluster. The previoius SessionFixture represents a single user session (with session options) and the QueryFixture represents a single query. As part of this refactoring, the fixture classes moved into "common" alongside FrontendTestBase. Testing: This patch includes only tests: no "production" code was changed. Change-Id: I3da58ee9b0beebeffb170b9430bd36d20dcd2401 --- M fe/src/main/java/org/apache/impala/analysis/Expr.java M fe/src/main/java/org/apache/impala/analysis/SelectListItem.java M fe/src/main/java/org/apache/impala/analysis/SlotDescriptor.java D fe/src/test/java/org/apache/impala/analysis/AnalysisSessionFixture.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeAuthStmtsTest.java A fe/src/test/java/org/apache/impala/analysis/ExprCardinalityTest.java M fe/src/test/java/org/apache/impala/analysis/ExprNdvTest.java M fe/src/test/java/org/apache/impala/analysis/ExprRewriteRulesTest.java A fe/src/test/java/org/apache/impala/common/AbstractFrontendTest.java A fe/src/test/java/org/apache/impala/common/AnalysisSessionFixture.java A fe/src/test/java/org/apache/impala/common/FrontendFixture.java M fe/src/test/java/org/apache/impala/common/FrontendTestBase.java A fe/src/test/java/org/apache/impala/common/QueryFixture.java M fe/src/test/java/org/apache/impala/planner/CardinalityTest.java A testdata/NullRows/data.csv M testdata/bin/compute-table-stats.sh M testdata/datasets/functional/functional_schema_template.sql M testdata/datasets/functional/schema_constraints.csv 18 files changed, 1,679 insertions(+), 534 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/48/12248/2 -- To view, visit http://gerrit.cloudera.org:8080/12248 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I3da58ee9b0beebeffb170b9430bd36d20dcd2401 Gerrit-Change-Number: 12248 Gerrit-PatchSet: 2 Gerrit-Owner: Paul Rogers <[email protected]> Gerrit-Reviewer: Paul Rogers <[email protected]>
