[Impala-ASF-CR] IMPALA-6228: Control stats extrapolation via tbl prop.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/9139 ) Change subject: IMPALA-6228: Control stats extrapolation via tbl prop. .. Patch Set 5: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/9139 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie49597bf1b93b7572106abc620d91f199cba0cfd Gerrit-Change-Number: 9139 Gerrit-PatchSet: 5 Gerrit-Owner: Alex Behm Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Sat, 03 Feb 2018 22:56:12 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6228: Control stats extrapolation via tbl prop.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/9139 ) Change subject: IMPALA-6228: Control stats extrapolation via tbl prop. .. IMPALA-6228: Control stats extrapolation via tbl prop. Introduces a new TBLPROPERTY for controlling stats extrapolation on a per-table basis: impala.enable.stats.extrapolation=true/false The property key was chosen to be consistent with the impalad startup flag --enable_stats_extrapolation and to indicate that the property was set and is used by Impala. Behavior: - If the property is not set, then the extrapolation behavior is determined by the impalad startup flag. - If the property is set, it overrides the impalad startup flag, i.e., extrapolation can be explicitly enabled or disabled regardless of the startup flag. Testing: - added new unit tests - code/hdfs run passed Change-Id: Ie49597bf1b93b7572106abc620d91f199cba0cfd Reviewed-on: http://gerrit.cloudera.org:8080/9139 Reviewed-by: Alex Behm Tested-by: Impala Public Jenkins --- M fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java M fe/src/main/java/org/apache/impala/service/BackendConfig.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeDDLTest.java M fe/src/test/java/org/apache/impala/planner/StatsExtrapolationTest.java M testdata/workloads/functional-query/queries/QueryTest/stats-extrapolation.test M tests/custom_cluster/test_stats_extrapolation.py A tests/metadata/test_stats_extrapolation.py 9 files changed, 357 insertions(+), 173 deletions(-) Approvals: Alex Behm: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/9139 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ie49597bf1b93b7572106abc620d91f199cba0cfd Gerrit-Change-Number: 9139 Gerrit-PatchSet: 6 Gerrit-Owner: Alex Behm Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vuk Ercegovac
[Impala-ASF-CR] IMPALA-6228: Control stats extrapolation via tbl prop.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/9139 ) Change subject: IMPALA-6228: Control stats extrapolation via tbl prop. .. Patch Set 5: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/1879/ -- To view, visit http://gerrit.cloudera.org:8080/9139 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie49597bf1b93b7572106abc620d91f199cba0cfd Gerrit-Change-Number: 9139 Gerrit-PatchSet: 5 Gerrit-Owner: Alex Behm Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Sat, 03 Feb 2018 19:18:27 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6228: Control stats extrapolation via tbl prop.
Alex Behm has posted comments on this change. ( http://gerrit.cloudera.org:8080/9139 ) Change subject: IMPALA-6228: Control stats extrapolation via tbl prop. .. Patch Set 5: Code-Review+2 Rebase and resolve more conflicts. -- To view, visit http://gerrit.cloudera.org:8080/9139 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie49597bf1b93b7572106abc620d91f199cba0cfd Gerrit-Change-Number: 9139 Gerrit-PatchSet: 5 Gerrit-Owner: Alex Behm Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Sat, 03 Feb 2018 19:17:28 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6228: Control stats extrapolation via tbl prop.
Hello Vuk Ercegovac, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/9139 to look at the new patch set (#5). Change subject: IMPALA-6228: Control stats extrapolation via tbl prop. .. IMPALA-6228: Control stats extrapolation via tbl prop. Introduces a new TBLPROPERTY for controlling stats extrapolation on a per-table basis: impala.enable.stats.extrapolation=true/false The property key was chosen to be consistent with the impalad startup flag --enable_stats_extrapolation and to indicate that the property was set and is used by Impala. Behavior: - If the property is not set, then the extrapolation behavior is determined by the impalad startup flag. - If the property is set, it overrides the impalad startup flag, i.e., extrapolation can be explicitly enabled or disabled regardless of the startup flag. Testing: - added new unit tests - code/hdfs run passed Change-Id: Ie49597bf1b93b7572106abc620d91f199cba0cfd --- M fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java M fe/src/main/java/org/apache/impala/service/BackendConfig.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeDDLTest.java M fe/src/test/java/org/apache/impala/planner/StatsExtrapolationTest.java M testdata/workloads/functional-query/queries/QueryTest/stats-extrapolation.test M tests/custom_cluster/test_stats_extrapolation.py A tests/metadata/test_stats_extrapolation.py 9 files changed, 357 insertions(+), 173 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/39/9139/5 -- To view, visit http://gerrit.cloudera.org:8080/9139 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ie49597bf1b93b7572106abc620d91f199cba0cfd Gerrit-Change-Number: 9139 Gerrit-PatchSet: 5 Gerrit-Owner: Alex Behm Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Vuk Ercegovac
[Impala-ASF-CR] IMPALA-6228: Control stats extrapolation via tbl prop.
Alex Behm has removed a vote on this change. Change subject: IMPALA-6228: Control stats extrapolation via tbl prop. .. Removed Code-Review+2 by Alex Behm -- To view, visit http://gerrit.cloudera.org:8080/9139 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: deleteVote Gerrit-Change-Id: Ie49597bf1b93b7572106abc620d91f199cba0cfd Gerrit-Change-Number: 9139 Gerrit-PatchSet: 4 Gerrit-Owner: Alex Behm Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Vuk Ercegovac
[Impala-ASF-CR] IMPALA-6228: Control stats extrapolation via tbl prop.
Alex Behm has posted comments on this change. ( http://gerrit.cloudera.org:8080/9139 ) Change subject: IMPALA-6228: Control stats extrapolation via tbl prop. .. Patch Set 4: Code-Review+2 Rebase and resolve conflicts -- To view, visit http://gerrit.cloudera.org:8080/9139 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie49597bf1b93b7572106abc620d91f199cba0cfd Gerrit-Change-Number: 9139 Gerrit-PatchSet: 4 Gerrit-Owner: Alex Behm Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Fri, 02 Feb 2018 23:57:25 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6228: Control stats extrapolation via tbl prop.
Hello Vuk Ercegovac, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/9139 to look at the new patch set (#4). Change subject: IMPALA-6228: Control stats extrapolation via tbl prop. .. IMPALA-6228: Control stats extrapolation via tbl prop. Introduces a new TBLPROPERTY for controlling stats extrapolation on a per-table basis: impala.enable.stats.extrapolation=true/false The property key was chosen to be consistent with the impalad startup flag --enable_stats_extrapolation and to indicate that the property was set and is used by Impala. Behavior: - If the property is not set, then the extrapolation behavior is determined by the impalad startup flag. - If the property is set, it overrides the impalad startup flag, i.e., extrapolation can be explicitly enabled or disabled regardless of the startup flag. Testing: - added new unit tests - code/hdfs run passed Change-Id: Ie49597bf1b93b7572106abc620d91f199cba0cfd --- M fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java M fe/src/main/java/org/apache/impala/service/BackendConfig.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeDDLTest.java M fe/src/test/java/org/apache/impala/planner/StatsExtrapolationTest.java M testdata/workloads/functional-query/queries/QueryTest/stats-extrapolation.test M tests/custom_cluster/test_stats_extrapolation.py A tests/metadata/test_stats_extrapolation.py 9 files changed, 329 insertions(+), 278 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/39/9139/4 -- To view, visit http://gerrit.cloudera.org:8080/9139 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ie49597bf1b93b7572106abc620d91f199cba0cfd Gerrit-Change-Number: 9139 Gerrit-PatchSet: 4 Gerrit-Owner: Alex Behm Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Vuk Ercegovac
[Impala-ASF-CR] IMPALA-6228: Control stats extrapolation via tbl prop.
Alex Behm has posted comments on this change. ( http://gerrit.cloudera.org:8080/9139 ) Change subject: IMPALA-6228: Control stats extrapolation via tbl prop. .. Patch Set 3: Code-Review+2 Will wait for http://gerrit.cloudera.org:8080/9133 and rebase before merging -- To view, visit http://gerrit.cloudera.org:8080/9139 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie49597bf1b93b7572106abc620d91f199cba0cfd Gerrit-Change-Number: 9139 Gerrit-PatchSet: 3 Gerrit-Owner: Alex Behm Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Wed, 31 Jan 2018 22:14:58 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6228: Control stats extrapolation via tbl prop.
Vuk Ercegovac has posted comments on this change. ( http://gerrit.cloudera.org:8080/9139 ) Change subject: IMPALA-6228: Control stats extrapolation via tbl prop. .. Patch Set 3: Code-Review+1 (1 comment) http://gerrit.cloudera.org:8080/#/c/9139/1/tests/metadata/test_stats_extrapolation.py File tests/metadata/test_stats_extrapolation.py: http://gerrit.cloudera.org:8080/#/c/9139/1/tests/metadata/test_stats_extrapolation.py@25 PS1, Line 25: class TestStatsExtrapolation(ImpalaTestSuite): > Added a minimal e2e test. thanks! -- To view, visit http://gerrit.cloudera.org:8080/9139 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie49597bf1b93b7572106abc620d91f199cba0cfd Gerrit-Change-Number: 9139 Gerrit-PatchSet: 3 Gerrit-Owner: Alex Behm Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Wed, 31 Jan 2018 22:05:31 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-6228: Control stats extrapolation via tbl prop.
Hello Vuk Ercegovac, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/9139 to look at the new patch set (#3). Change subject: IMPALA-6228: Control stats extrapolation via tbl prop. .. IMPALA-6228: Control stats extrapolation via tbl prop. Introduces a new TBLPROPERTY for controlling stats extrapolation on a per-table basis: impala.enable.stats.extrapolation=true/false The property key was chosen to be consistent with the impalad startup flag --enable_stats_extrapolation and to indicate that the property was set and is used by Impala. Behavior: - If the property is not set, then the extrapolation behavior is determined by the impalad startup flag. - If the property is set, it overrides the impalad startup flag, i.e., extrapolation can be explicitly enabled or disabled regardless of the startup flag. Testing: - added new unit tests - code/hdfs run passed Change-Id: Ie49597bf1b93b7572106abc620d91f199cba0cfd --- M fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java M fe/src/main/java/org/apache/impala/service/BackendConfig.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeDDLTest.java M fe/src/test/java/org/apache/impala/planner/StatsExtrapolationTest.java M testdata/workloads/functional-query/queries/QueryTest/stats-extrapolation.test M tests/custom_cluster/test_stats_extrapolation.py A tests/metadata/test_stats_extrapolation.py 9 files changed, 308 insertions(+), 143 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/39/9139/3 -- To view, visit http://gerrit.cloudera.org:8080/9139 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ie49597bf1b93b7572106abc620d91f199cba0cfd Gerrit-Change-Number: 9139 Gerrit-PatchSet: 3 Gerrit-Owner: Alex Behm Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Vuk Ercegovac
[Impala-ASF-CR] IMPALA-6228: Control stats extrapolation via tbl prop.
Alex Behm has posted comments on this change. ( http://gerrit.cloudera.org:8080/9139 ) Change subject: IMPALA-6228: Control stats extrapolation via tbl prop. .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/9139/1/tests/metadata/test_stats_extrapolation.py File tests/metadata/test_stats_extrapolation.py: http://gerrit.cloudera.org:8080/#/c/9139/1/tests/metadata/test_stats_extrapolation.py@25 PS1, Line 25: class TestStatsExtrapolation(ImpalaTestSuite): > I think it would be good to keep an e2e test with that flag exercised. Added a minimal e2e test. -- To view, visit http://gerrit.cloudera.org:8080/9139 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie49597bf1b93b7572106abc620d91f199cba0cfd Gerrit-Change-Number: 9139 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Behm Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Wed, 31 Jan 2018 21:59:16 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-6228: Control stats extrapolation via tbl prop.
Vuk Ercegovac has posted comments on this change. ( http://gerrit.cloudera.org:8080/9139 ) Change subject: IMPALA-6228: Control stats extrapolation via tbl prop. .. Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/9139/1/tests/metadata/test_stats_extrapolation.py File tests/metadata/test_stats_extrapolation.py: http://gerrit.cloudera.org:8080/#/c/9139/1/tests/metadata/test_stats_extrapolation.py@25 PS1, Line 25: class TestStatsExtrapolation(ImpalaTestSuite): > You are correct, the impalad flag is not tested with an E2E test anymore. I I think it would be good to keep an e2e test with that flag exercised. -- To view, visit http://gerrit.cloudera.org:8080/9139 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie49597bf1b93b7572106abc620d91f199cba0cfd Gerrit-Change-Number: 9139 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Behm Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Wed, 31 Jan 2018 18:00:02 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-6228: Control stats extrapolation via tbl prop.
Hello Vuk Ercegovac, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/9139 to look at the new patch set (#2). Change subject: IMPALA-6228: Control stats extrapolation via tbl prop. .. IMPALA-6228: Control stats extrapolation via tbl prop. Introduces a new TBLPROPERTY for controlling stats extrapolation on a per-table basis: impala.enable.stats.extrapolation=true/false The property key was chosen to be consistent with the impalad startup flag --enable_stats_extrapolation and to indicate that the property was set and is used by Impala. Behavior: - If the property is not set, then the extrapolation behavior is determined by the impalad startup flag. - If the property is set, it overrides the impalad startup flag, i.e., extrapolation can be explicitly enabled or disabled regardless of the startup flag. Testing: - added new unit tests - code/hdfs run passed Change-Id: Ie49597bf1b93b7572106abc620d91f199cba0cfd --- M fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java M fe/src/main/java/org/apache/impala/service/BackendConfig.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeDDLTest.java M fe/src/test/java/org/apache/impala/planner/StatsExtrapolationTest.java M testdata/workloads/functional-query/queries/QueryTest/stats-extrapolation.test R tests/metadata/test_stats_extrapolation.py 8 files changed, 170 insertions(+), 63 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/39/9139/2 -- To view, visit http://gerrit.cloudera.org:8080/9139 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ie49597bf1b93b7572106abc620d91f199cba0cfd Gerrit-Change-Number: 9139 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Behm Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Vuk Ercegovac
[Impala-ASF-CR] IMPALA-6228: Control stats extrapolation via tbl prop.
Alex Behm has posted comments on this change. ( http://gerrit.cloudera.org:8080/9139 ) Change subject: IMPALA-6228: Control stats extrapolation via tbl prop. .. Patch Set 1: (6 comments) http://gerrit.cloudera.org:8080/#/c/9139/1/fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java File fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java: http://gerrit.cloudera.org:8080/#/c/9139/1/fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java@549 PS1, Line 549: this > nit: remove "this" Good point. Done. http://gerrit.cloudera.org:8080/#/c/9139/1/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java File fe/src/main/java/org/apache/impala/catalog/HdfsTable.java: http://gerrit.cloudera.org:8080/#/c/9139/1/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java@130 PS1, Line 130: // setting for a specific table. By default, tables do not have property set and rely > nit: ... the property ... Done http://gerrit.cloudera.org:8080/#/c/9139/1/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java@1971 PS1, Line 1971:* Returns true stats extrapolation is enabled for this table, false otherwise. > nit: if Done http://gerrit.cloudera.org:8080/#/c/9139/1/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java@1975 PS1, Line 1975: enableStatsExtrapolation > Current name sounds like it modifies state instead of tests state. how abou Done http://gerrit.cloudera.org:8080/#/c/9139/1/fe/src/test/java/org/apache/impala/planner/StatsExtrapolationTest.java File fe/src/test/java/org/apache/impala/planner/StatsExtrapolationTest.java: http://gerrit.cloudera.org:8080/#/c/9139/1/fe/src/test/java/org/apache/impala/planner/StatsExtrapolationTest.java@73 PS1, Line 73: actualExrtapNumRows > nit: spelling Done http://gerrit.cloudera.org:8080/#/c/9139/1/tests/metadata/test_stats_extrapolation.py File tests/metadata/test_stats_extrapolation.py: http://gerrit.cloudera.org:8080/#/c/9139/1/tests/metadata/test_stats_extrapolation.py@25 PS1, Line 25: class TestStatsExtrapolation(ImpalaTestSuite): > perhaps I've misread the diff, but is the impalad flag for extrapolation st You are correct, the impalad flag is not tested with an E2E test anymore. I was thinking that might not be necessary, but happy to add a new test if you prefer. -- To view, visit http://gerrit.cloudera.org:8080/9139 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie49597bf1b93b7572106abc620d91f199cba0cfd Gerrit-Change-Number: 9139 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Behm Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Tue, 30 Jan 2018 20:25:01 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-6228: Control stats extrapolation via tbl prop.
Vuk Ercegovac has posted comments on this change. ( http://gerrit.cloudera.org:8080/9139 ) Change subject: IMPALA-6228: Control stats extrapolation via tbl prop. .. Patch Set 1: (6 comments) looks good. mostly nit comments and one clarifying question for the e2e test. http://gerrit.cloudera.org:8080/#/c/9139/1/fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java File fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java: http://gerrit.cloudera.org:8080/#/c/9139/1/fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java@549 PS1, Line 549: this nit: remove "this" http://gerrit.cloudera.org:8080/#/c/9139/1/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java File fe/src/main/java/org/apache/impala/catalog/HdfsTable.java: http://gerrit.cloudera.org:8080/#/c/9139/1/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java@130 PS1, Line 130: // setting for a specific table. By default, tables do not have property set and rely nit: ... the property ... http://gerrit.cloudera.org:8080/#/c/9139/1/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java@1971 PS1, Line 1971:* Returns true stats extrapolation is enabled for this table, false otherwise. nit: if http://gerrit.cloudera.org:8080/#/c/9139/1/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java@1975 PS1, Line 1975: enableStatsExtrapolation Current name sounds like it modifies state instead of tests state. how about: isStatsExtrapolationEnabled() ? http://gerrit.cloudera.org:8080/#/c/9139/1/fe/src/test/java/org/apache/impala/planner/StatsExtrapolationTest.java File fe/src/test/java/org/apache/impala/planner/StatsExtrapolationTest.java: http://gerrit.cloudera.org:8080/#/c/9139/1/fe/src/test/java/org/apache/impala/planner/StatsExtrapolationTest.java@73 PS1, Line 73: actualExrtapNumRows nit: spelling http://gerrit.cloudera.org:8080/#/c/9139/1/tests/metadata/test_stats_extrapolation.py File tests/metadata/test_stats_extrapolation.py: http://gerrit.cloudera.org:8080/#/c/9139/1/tests/metadata/test_stats_extrapolation.py@25 PS1, Line 25: class TestStatsExtrapolation(ImpalaTestSuite): perhaps I've misread the diff, but is the impalad flag for extrapolation still tested with an e2e test with this change? -- To view, visit http://gerrit.cloudera.org:8080/9139 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie49597bf1b93b7572106abc620d91f199cba0cfd Gerrit-Change-Number: 9139 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Behm Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Tue, 30 Jan 2018 18:33:15 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-6228: Control stats extrapolation via tbl prop.
Alex Behm has uploaded this change for review. ( http://gerrit.cloudera.org:8080/9139 Change subject: IMPALA-6228: Control stats extrapolation via tbl prop. .. IMPALA-6228: Control stats extrapolation via tbl prop. Introduces a new TBLPROPERTY for controlling stats extrapolation on a per-table basis: impala.enable.stats.extrapolation=true/false The property key was chosen to be consistent with the impalad startup flag --enable_stats_extrapolation and to indicate that the property was set and is used by Impala. Behavior: - If the property is not set, then the extrapolation behavior is determined by the impalad startup flag. - If the property is set, it overrides the impalad startup flag, i.e., extrapolation can be explicitly enabled or disabled regardless of the startup flag. Testing: - added new unit tests - code/hdfs run passed Change-Id: Ie49597bf1b93b7572106abc620d91f199cba0cfd --- M fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeDDLTest.java M fe/src/test/java/org/apache/impala/planner/StatsExtrapolationTest.java M testdata/workloads/functional-query/queries/QueryTest/stats-extrapolation.test R tests/metadata/test_stats_extrapolation.py 7 files changed, 167 insertions(+), 60 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/39/9139/1 -- To view, visit http://gerrit.cloudera.org:8080/9139 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ie49597bf1b93b7572106abc620d91f199cba0cfd Gerrit-Change-Number: 9139 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Behm