[Impala-ASF-CR] IMPALA-4551: Limit the size of SQL statements
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14012 ) Change subject: IMPALA-4551: Limit the size of SQL statements .. Patch Set 3: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/14012 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5675fb4a08c1dc51ae5bcf467cbb969cc064602c Gerrit-Change-Number: 14012 Gerrit-PatchSet: 3 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Tue, 13 Aug 2019 05:17:06 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8859: fix test global config file for remote clusters
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14053 ) Change subject: IMPALA-8859: fix test_global_config_file for remote clusters .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/4229/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/14053 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I448e5a7dfc0ab6fd53182a593e2fff1a12a10fd7 Gerrit-Change-Number: 14053 Gerrit-PatchSet: 1 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 13 Aug 2019 04:20:32 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8859: fix test global config file for remote clusters
Tim Armstrong has uploaded this change for review. ( http://gerrit.cloudera.org:8080/14053 Change subject: IMPALA-8859: fix test_global_config_file for remote clusters .. IMPALA-8859: fix test_global_config_file for remote clusters I think the bug is that necessary environment variables were not passed in - the environment was clobbered instead of just having the necessary variable added. Change-Id: I448e5a7dfc0ab6fd53182a593e2fff1a12a10fd7 --- M tests/shell/test_shell_commandline.py 1 file changed, 3 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/53/14053/1 -- To view, visit http://gerrit.cloudera.org:8080/14053 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I448e5a7dfc0ab6fd53182a593e2fff1a12a10fd7 Gerrit-Change-Number: 14053 Gerrit-PatchSet: 1 Gerrit-Owner: Tim Armstrong
[Impala-ASF-CR] IMPALA-8847: Ignore add partition events with empty partition list
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14049 ) Change subject: IMPALA-8847: Ignore add partition events with empty partition list .. Patch Set 4: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/4775/ -- To view, visit http://gerrit.cloudera.org:8080/14049 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I877ce6233934e7090cd18e497f748bc6479838cb Gerrit-Change-Number: 14049 Gerrit-PatchSet: 4 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Tue, 13 Aug 2019 02:42:56 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4551: Limit the size of SQL statements
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14012 ) Change subject: IMPALA-4551: Limit the size of SQL statements .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/4228/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/14012 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5675fb4a08c1dc51ae5bcf467cbb969cc064602c Gerrit-Change-Number: 14012 Gerrit-PatchSet: 3 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Tue, 13 Aug 2019 01:49:14 + Gerrit-HasComments: No
[Impala-ASF-CR] Revert "IMPALA-8791: Handle the case where there is no fragment scheduled on"
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14052 ) Change subject: Revert "IMPALA-8791: Handle the case where there is no fragment scheduled on" .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/4227/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/14052 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id20cf3581995f450de6f491e7874cbcf23b52cda Gerrit-Change-Number: 14052 Gerrit-PatchSet: 1 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 13 Aug 2019 01:38:10 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4551: Limit the size of SQL statements
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14012 ) Change subject: IMPALA-4551: Limit the size of SQL statements .. Patch Set 3: (1 comment) http://gerrit.cloudera.org:8080/#/c/14012/3/tests/common/impala_connection.py File tests/common/impala_connection.py: http://gerrit.cloudera.org:8080/#/c/14012/3/tests/common/impala_connection.py@49 PS3, Line 49: def log_sql_stmt(sql_stmt): flake8: E302 expected 2 blank lines, found 0 -- To view, visit http://gerrit.cloudera.org:8080/14012 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5675fb4a08c1dc51ae5bcf467cbb969cc064602c Gerrit-Change-Number: 14012 Gerrit-PatchSet: 3 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Tue, 13 Aug 2019 01:08:04 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-4551: Limit the size of SQL statements
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14012 ) Change subject: IMPALA-4551: Limit the size of SQL statements .. Patch Set 3: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4776/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/14012 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5675fb4a08c1dc51ae5bcf467cbb969cc064602c Gerrit-Change-Number: 14012 Gerrit-PatchSet: 3 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Tue, 13 Aug 2019 01:07:57 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4551: Limit the size of SQL statements
Hello Bharath Vissapragada, Michael Ho, Quanlong Huang, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/14012 to look at the new patch set (#3). Change subject: IMPALA-4551: Limit the size of SQL statements .. IMPALA-4551: Limit the size of SQL statements Various BI tools generate and run SQL. When used incorrectly or misconfigured, the tools can generate extremely large SQLs. Some of these SQL statements reach 10s of megabytes. Large SQL statements impose costs throughout execution, including statement rewrite logic in the frontend and codegen in the backend. The resource usage of these statements can impact the stability of the system or the ability to run other SQL statements. This implements two new query options that provide controls to reject large SQL statements. - The first, MAX_STATEMENT_LENGTH_BYTES is a cap on the total size of the SQL statement (in bytes). It is applied before any parsing or analysis. It uses a default value of 16MB. - The second, STATEMENT_EXPRESSION_LIMIT, is a limit on the total number of expressions in a statement or any views that it references. The limit is applied upon the first round of analysis, but it is not reapplied when statement rewrite rules are applied. Certain expressions such as literals in IN lists or VALUES clauses are not analyzed and do not count towards the limit. It uses a default value of 250,000. The two are complementary. Since enforcing the statement expression limit requires parsing and analyzing the statement, the MAX_STATEMENT_LENGTH_BYTES sets an upper bound on the size of statement that needs to be parsed and analyzed. Testing confirms that even statements approaching 16MB get through the first round of analysis within a few seconds and then are rejected. This also changes the logging in tests/common/impala_connection.py to limit the total SQL size that it will print to 128KB. This is prevents the JUnitXML (which includes this logging) from being too large. Existing tests do not run SQL larger than about 80KB, so this only applies to tests added in this change that run multi-MB SQLs to verify limits. Testing: - This adds frontend tests that verify the low level semantics about how expressions are counted and verifies that the expression limits are enforced. - This adds end-to-end tests that verify both the MAX_STATEMENT_LENGTH_BYTES and STATEMENT_EXPRESSION_LIMIT at their defaults values. - There is also an end-to-end test that runs in exhaustive mode that runs a SQL with close to 250,000 expressions. Change-Id: I5675fb4a08c1dc51ae5bcf467cbb969cc064602c --- M be/src/service/impala-server.cc M be/src/service/query-options-test.cc M be/src/service/query-options.cc M be/src/service/query-options.h M common/thrift/ImpalaInternalService.thrift M common/thrift/ImpalaService.thrift M common/thrift/generate_error_codes.py M fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/analysis/Expr.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeExprsTest.java M tests/common/impala_connection.py M tests/query_test/test_exprs.py 13 files changed, 377 insertions(+), 9 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/12/14012/3 -- To view, visit http://gerrit.cloudera.org:8080/14012 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I5675fb4a08c1dc51ae5bcf467cbb969cc064602c Gerrit-Change-Number: 14012 Gerrit-PatchSet: 3 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Quanlong Huang
[Impala-ASF-CR] IMPALA-4551: Limit the size of SQL statements
Joe McDonnell has posted comments on this change. (
http://gerrit.cloudera.org:8080/14012 )
Change subject: IMPALA-4551: Limit the size of SQL statements
..
Patch Set 2:
(11 comments)
http://gerrit.cloudera.org:8080/#/c/14012/2/be/src/service/query-options.h
File be/src/service/query-options.h:
http://gerrit.cloudera.org:8080/#/c/14012/2/be/src/service/query-options.h@190
PS2, Line 190:
> nit: Do we need spaces here according our code style?
Removed the spaces for these new lines. I don't think the spaces are needed for
any code style thing.
http://gerrit.cloudera.org:8080/#/c/14012/2/be/src/service/query-options.cc
File be/src/service/query-options.cc:
http://gerrit.cloudera.org:8080/#/c/14012/2/be/src/service/query-options.cc@824
PS2, Line 824: Invalid statement expression limit
> nit: can we also print the value?
Added the actual value here and for the max statement length bytes.
http://gerrit.cloudera.org:8080/#/c/14012/2/be/src/service/query-options.cc@825
PS2, Line 825: Value
> typo: Valid
Done
http://gerrit.cloudera.org:8080/#/c/14012/2/be/src/service/query-options.cc@838
PS2, Line 838: value
> nit: values
Done
http://gerrit.cloudera.org:8080/#/c/14012/1/fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java
File fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java:
http://gerrit.cloudera.org:8080/#/c/14012/1/fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java@462
PS1, Line 462: boolean isExplain = analysisResult_.isExplainStmt();
> nit: Probably worth mentioning that we want to enforce this before the rewr
Done
http://gerrit.cloudera.org:8080/#/c/14012/2/fe/src/main/java/org/apache/impala/analysis/Analyzer.java
File fe/src/main/java/org/apache/impala/analysis/Analyzer.java:
http://gerrit.cloudera.org:8080/#/c/14012/2/fe/src/main/java/org/apache/impala/analysis/Analyzer.java@2861
PS2, Line 2861: String errorStr = String.format("Exceeded the statement
expression limit (%s)\n" +
> nit: use "%d" for int.
Switched to %d. I don't know the history, but it makes sense to use %d.
http://gerrit.cloudera.org:8080/#/c/14012/1/fe/src/test/java/org/apache/impala/analysis/AnalyzeExprsTest.java
File fe/src/test/java/org/apache/impala/analysis/AnalyzeExprsTest.java:
http://gerrit.cloudera.org:8080/#/c/14012/1/fe/src/test/java/org/apache/impala/analysis/AnalyzeExprsTest.java@2435
PS1, Line 2435: String repCols20 = getRepeatedColumnReference("int_col",
20, true);
> Verify that analyzer.numStmtExprs_ is accounted properly?
Added asserts for the AnalyzesOk() calls to verify numStmtExprs_ is exactly
what we expect. The AnalysisError() cases already include the numStmtExprs_ in
the error message.
http://gerrit.cloudera.org:8080/#/c/14012/1/fe/src/test/java/org/apache/impala/analysis/AnalyzeExprsTest.java@2493
PS1, Line 2493: StringBuilder inList = new StringBuilder();
> what about constant expressions in the IN lists? foo IN (2*3, 3*4...) ? It
Good point, I added tests for arithmetic expressions 1*2*3*4. This found a bug
where these expressions could be double-counted due to the Exprs being cloned
and reanalyzed. I added a per-Expr variable to track whether the Expr had been
counted. Each Expr will only be counted once. This also required some logic to
skip counting these expressions when in a WITH clause (it gets accounted
separately).
I added a test that has an IN list with an arithmetic expression.
http://gerrit.cloudera.org:8080/#/c/14012/2/fe/src/test/java/org/apache/impala/analysis/AnalyzeExprsTest.java
File fe/src/test/java/org/apache/impala/analysis/AnalyzeExprsTest.java:
http://gerrit.cloudera.org:8080/#/c/14012/2/fe/src/test/java/org/apache/impala/analysis/AnalyzeExprsTest.java@2443
PS2, Line 2443: // WHERE clause
> nit: For coverage, maybe test the expression in the WHERE clause?
The way I'm thinking about this is that the SQL without the WHERE has 20
expressions and adding the WHERE with bool_col adds one expression, so it would
only fail if the WHERE were counted. I added a comment here.
http://gerrit.cloudera.org:8080/#/c/14012/1/tests/query_test/test_exprs.py
File tests/query_test/test_exprs.py:
http://gerrit.cloudera.org:8080/#/c/14012/1/tests/query_test/test_exprs.py@139
PS1, Line 139: # This takes 20+ minutes, so only run it on exhaustive.
> Is the intention to test the default limits here? If not, we can probably s
I wanted this to test as close to the default limits as possible. It is unclear
how much value this test has. In my runs, it doesn't seem to impact the total
runtime of the exhaustive tests by much.
http://gerrit.cloudera.org:8080/#/c/14012/2/tests/query_test/test_exprs.py
File tests/query_test/test_exprs.py:
http://gerrit.cloudera.org:8080/#/c/14012/2/tests/query_test/test_exprs.py@216
PS2, Line 216:
> flake8: E261 at least two spaces before inline comment
Done
--
To view, visit
[Impala-ASF-CR] Clean up stress tests in core
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/14002 ) Change subject: Clean up stress tests in core .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/14002/1/tests/stress/test_mini_stress.py File tests/stress/test_mini_stress.py: http://gerrit.cloudera.org:8080/#/c/14002/1/tests/stress/test_mini_stress.py@a1 PS1, Line 1: > Sorry -- I'm a bit confused. We don't run these tests ever? The code commen All of the individual tests in this file either have an xfail or a strip marker. https://gerrit.cloudera.org/#/c/14002/1/tests/stress/test_ddl_stress.py is still run in exhaustive -- To view, visit http://gerrit.cloudera.org:8080/14002 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icc8d948a3a95bd964a7acbe5722f01891a248f11 Gerrit-Change-Number: 14002 Gerrit-PatchSet: 1 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 13 Aug 2019 01:02:27 + Gerrit-HasComments: Yes
[Impala-ASF-CR] Revert "IMPALA-8791: Handle the case where there is no fragment scheduled on"
Tim Armstrong has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/14052 ) Change subject: Revert "IMPALA-8791: Handle the case where there is no fragment scheduled on" .. Revert "IMPALA-8791: Handle the case where there is no fragment scheduled on" This reverts commit 760169edcbca438c5964380a604b6c271c6bd1a3. Change-Id: Id20cf3581995f450de6f491e7874cbcf23b52cda Reviewed-on: http://gerrit.cloudera.org:8080/14052 Reviewed-by: Tim Armstrong Tested-by: Tim Armstrong --- M be/src/runtime/coordinator-backend-state.cc M be/src/runtime/coordinator-backend-state.h M be/src/runtime/coordinator.cc M be/src/runtime/coordinator.h M be/src/scheduling/admission-controller-test.cc M be/src/scheduling/admission-controller.cc M be/src/scheduling/query-schedule.cc M be/src/scheduling/query-schedule.h M be/src/scheduling/scheduler.cc M be/src/scheduling/scheduler.h M fe/src/main/java/org/apache/impala/planner/Planner.java M testdata/workloads/functional-query/queries/QueryTest/dedicated-coord-mem-estimates.test 12 files changed, 123 insertions(+), 154 deletions(-) Approvals: Tim Armstrong: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/14052 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Id20cf3581995f450de6f491e7874cbcf23b52cda Gerrit-Change-Number: 14052 Gerrit-PatchSet: 2 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] Revert "IMPALA-8791: Handle the case where there is no fragment scheduled on"
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/14052 ) Change subject: Revert "IMPALA-8791: Handle the case where there is no fragment scheduled on" .. Patch Set 1: Verified+1 Code-Review+2 This was the latest commit, so should be safe to revert. -- To view, visit http://gerrit.cloudera.org:8080/14052 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id20cf3581995f450de6f491e7874cbcf23b52cda Gerrit-Change-Number: 14052 Gerrit-PatchSet: 1 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 13 Aug 2019 01:00:23 + Gerrit-HasComments: No
[Impala-ASF-CR] Revert "IMPALA-8791: Handle the case where there is no fragment scheduled on"
Tim Armstrong has uploaded this change for review. ( http://gerrit.cloudera.org:8080/14052 Change subject: Revert "IMPALA-8791: Handle the case where there is no fragment scheduled on" .. Revert "IMPALA-8791: Handle the case where there is no fragment scheduled on" This reverts commit 760169edcbca438c5964380a604b6c271c6bd1a3. Change-Id: Id20cf3581995f450de6f491e7874cbcf23b52cda --- M be/src/runtime/coordinator-backend-state.cc M be/src/runtime/coordinator-backend-state.h M be/src/runtime/coordinator.cc M be/src/runtime/coordinator.h M be/src/scheduling/admission-controller-test.cc M be/src/scheduling/admission-controller.cc M be/src/scheduling/query-schedule.cc M be/src/scheduling/query-schedule.h M be/src/scheduling/scheduler.cc M be/src/scheduling/scheduler.h M fe/src/main/java/org/apache/impala/planner/Planner.java M testdata/workloads/functional-query/queries/QueryTest/dedicated-coord-mem-estimates.test 12 files changed, 123 insertions(+), 154 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/52/14052/1 -- To view, visit http://gerrit.cloudera.org:8080/14052 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Id20cf3581995f450de6f491e7874cbcf23b52cda Gerrit-Change-Number: 14052 Gerrit-PatchSet: 1 Gerrit-Owner: Tim Armstrong
[Impala-ASF-CR] Clean up stress tests in core
David Knupp has posted comments on this change. ( http://gerrit.cloudera.org:8080/14002 ) Change subject: Clean up stress tests in core .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/14002/1/tests/stress/test_mini_stress.py File tests/stress/test_mini_stress.py: http://gerrit.cloudera.org:8080/#/c/14002/1/tests/stress/test_mini_stress.py@a1 PS1, Line 1: Sorry -- I'm a bit confused. We don't run these tests ever? The code comment made it seem like they aren't skipped in exhaustive runs, so I would have assumed this file was still needed. -- To view, visit http://gerrit.cloudera.org:8080/14002 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icc8d948a3a95bd964a7acbe5722f01891a248f11 Gerrit-Change-Number: 14002 Gerrit-PatchSet: 1 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Comment-Date: Tue, 13 Aug 2019 00:47:55 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8584: Add cookie support to the HTTP HS2 server
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13672 ) Change subject: IMPALA-8584: Add cookie support to the HTTP HS2 server .. Patch Set 5: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/4226/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13672 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I647c06f94ef91aa3b6413e91576c4ec506ed57f4 Gerrit-Change-Number: 13672 Gerrit-PatchSet: 5 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Tue, 13 Aug 2019 00:43:47 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8766: Undo hadoop-cloud-storage + HWX Nexus
Vihang Karajgaonkar has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/14024 ) Change subject: IMPALA-8766: Undo hadoop-cloud-storage + HWX Nexus .. IMPALA-8766: Undo hadoop-cloud-storage + HWX Nexus Previous commits for IMPALA-8766 attempted to use hadoop-cloud-storage to satisfy Impala's cloud dependencies (e.g. hadoop-aws, hadoop-azure, etc). On builds with USE_CDP_HIVE=true, this adds Knox gateway-cloud-bindings. However, the entry for hadoop-cloud-storage artifact in the impala.cdp.repo maven repository introduces dependencies that are external to that repository. This requires the HWX Nexus repository to resolve those dangling dependencies. Unfortunately, HWX Nexus ages out old jars, including the ones we need. This stops using hadoop-cloud-storage, and instead adds a direct dependency to Knox for USE_CDP_HIVE=true. It disables the HWX Nexus repository and leaves a tombstone explaining why. Testing: - Deleted my .m2 directory and rebuilt Impala with USE_CDP_HIVE=true - Verified the CLASSPATH still contains the right jars on USE_CDP_HIVE=true Change-Id: I79a0c2575fc50bbc3b393c150c0bce22258ea1bd Reviewed-on: http://gerrit.cloudera.org:8080/14024 Tested-by: Impala Public Jenkins Reviewed-by: Vihang Karajgaonkar --- M bin/impala-config.sh M fe/pom.xml M impala-parent/pom.xml 3 files changed, 48 insertions(+), 49 deletions(-) Approvals: Impala Public Jenkins: Verified Vihang Karajgaonkar: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/14024 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I79a0c2575fc50bbc3b393c150c0bce22258ea1bd Gerrit-Change-Number: 14024 Gerrit-PatchSet: 4 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Vihang Karajgaonkar
[Impala-ASF-CR] IMPALA-8584: Add cookie support to the HTTP HS2 server
Hello Andrew Sherman, Todd Lipcon, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13672 to look at the new patch set (#5). Change subject: IMPALA-8584: Add cookie support to the HTTP HS2 server .. IMPALA-8584: Add cookie support to the HTTP HS2 server This patch modifies the HTTP HS2 server to accept cookies for authentication in order to avoid having to authenticate every request through LDAP or Kerberos. It adds a flag, --max_cookie_lifetime_s, that determines how long generated cookies are valid for. Setting the flag to 0 disables cookie support. The cookies include a SHA256 HMAC signature that it used to verify them. They also have a timestamp that is used to determine if they have expired. If a cookie is successfully verified and hasn't expired, the username contained in the cookie is set on the connection. Each impalad uses its own key to generate the signature, so clients that reconnect to a different impalad will have to reauthenticate. On a single impalad cookies are valid across sessions and connections. A new cookie is generated and sent back with the Set-Cookie header on each request that was authenticated without using a cookie. Cookies are of the form: impala.hs2.auth=;HttpOnly;MaxAge= where: cookie = &&& and 'signature' is the SHA256 HMAC of the rest of the cookie The 'Secure' flag, which indicates to clients that the cookie should only be sent over secure connections, is omitted if '--ldap_passwords_in_clear_ok' is true. This is intended only for testing. It also adds the metrics: impala.thrift-server.hiveserver2-http-frontend.total-cookie-auth-success impala.thrift-server.hiveserver2-http-frontend.total-cookie-auth-failure Testing: - Added tests to the FE LDAP tests that use the metrics to verify successful and failed cookie attempts. Change-Id: I647c06f94ef91aa3b6413e91576c4ec506ed57f4 --- M be/src/rpc/CMakeLists.txt M be/src/rpc/auth-provider.h M be/src/rpc/authentication.cc A be/src/rpc/cookie-util.cc A be/src/rpc/cookie-util.h M be/src/transport/THttpServer.cpp M be/src/transport/THttpServer.h M be/src/util/openssl-util.cc M be/src/util/openssl-util.h M common/thrift/metrics.json M fe/src/test/java/org/apache/impala/customcluster/LdapHS2Test.java M fe/src/test/java/org/apache/impala/customcluster/LdapJdbcTest.java M fe/src/test/java/org/apache/impala/service/JdbcTestBase.java 13 files changed, 502 insertions(+), 103 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/72/13672/5 -- To view, visit http://gerrit.cloudera.org:8080/13672 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I647c06f94ef91aa3b6413e91576c4ec506ed57f4 Gerrit-Change-Number: 13672 Gerrit-PatchSet: 5 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Todd Lipcon
[Impala-ASF-CR] [DOCS] Put impala date doc in the right place in the alphabetical order
Alex Rodoni has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/14051 ) Change subject: [DOCS] Put impala_date doc in the right place in the alphabetical order .. [DOCS] Put impala_date doc in the right place in the alphabetical order Change-Id: Ic89535c8948f5476782371f8e7f9848ee2b1a14d Reviewed-on: http://gerrit.cloudera.org:8080/14051 Reviewed-by: Alex Rodoni Tested-by: Impala Public Jenkins --- M docs/impala.ditamap 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: Alex Rodoni: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/14051 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ic89535c8948f5476782371f8e7f9848ee2b1a14d Gerrit-Change-Number: 14051 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] [DOCS] Put impala date doc in the right place in the alphabetical order
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14051 ) Change subject: [DOCS] Put impala_date doc in the right place in the alphabetical order .. Patch Set 1: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/447/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/14051 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic89535c8948f5476782371f8e7f9848ee2b1a14d Gerrit-Change-Number: 14051 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 12 Aug 2019 22:41:59 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8847: Ignore add partition events with empty partition list
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14049 ) Change subject: IMPALA-8847: Ignore add partition events with empty partition list .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/4225/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/14049 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I877ce6233934e7090cd18e497f748bc6479838cb Gerrit-Change-Number: 14049 Gerrit-PatchSet: 4 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Mon, 12 Aug 2019 22:36:34 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8847: Ignore add partition events with empty partition list
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14049 ) Change subject: IMPALA-8847: Ignore add partition events with empty partition list .. Patch Set 4: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4775/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/14049 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I877ce6233934e7090cd18e497f748bc6479838cb Gerrit-Change-Number: 14049 Gerrit-PatchSet: 4 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Mon, 12 Aug 2019 22:35:44 + Gerrit-HasComments: No
[Impala-ASF-CR] [DOCS] Put impala date doc in the right place in the alphabetical order
Alex Rodoni has posted comments on this change. ( http://gerrit.cloudera.org:8080/14051 ) Change subject: [DOCS] Put impala_date doc in the right place in the alphabetical order .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/14051 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic89535c8948f5476782371f8e7f9848ee2b1a14d Gerrit-Change-Number: 14051 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 12 Aug 2019 22:24:06 + Gerrit-HasComments: No
[Impala-ASF-CR] [DOCS] Put impala date doc in the right place in the alphabetical order
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14051 ) Change subject: [DOCS] Put impala_date doc in the right place in the alphabetical order .. Patch Set 1: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/447/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/14051 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic89535c8948f5476782371f8e7f9848ee2b1a14d Gerrit-Change-Number: 14051 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 12 Aug 2019 22:23:36 + Gerrit-HasComments: No
[Impala-ASF-CR] [DOCS] Put impala date doc in the right place in the alphabetical order
Alex Rodoni has uploaded this change for review. ( http://gerrit.cloudera.org:8080/14051 Change subject: [DOCS] Put impala_date doc in the right place in the alphabetical order .. [DOCS] Put impala_date doc in the right place in the alphabetical order Change-Id: Ic89535c8948f5476782371f8e7f9848ee2b1a14d --- M docs/impala.ditamap 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/51/14051/1 -- To view, visit http://gerrit.cloudera.org:8080/14051 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ic89535c8948f5476782371f8e7f9848ee2b1a14d Gerrit-Change-Number: 14051 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni
[Impala-ASF-CR] IMPALA-8847: Ignore add partition events with empty partition list
Anurag Mantripragada has posted comments on this change. ( http://gerrit.cloudera.org:8080/14049 ) Change subject: IMPALA-8847: Ignore add partition events with empty partition list .. Patch Set 4: Code-Review+1 Looks good to me. -- To view, visit http://gerrit.cloudera.org:8080/14049 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I877ce6233934e7090cd18e497f748bc6479838cb Gerrit-Change-Number: 14049 Gerrit-PatchSet: 4 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Mon, 12 Aug 2019 22:17:40 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8198: DATE: Read from avro.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/13944 ) Change subject: IMPALA-8198: DATE: Read from avro. .. Patch Set 1: (2 comments) http://gerrit.cloudera.org:8080/#/c/13944/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/13944/1//COMMIT_MSG@27 PS1, Line 27: Note that Hive 3.1 switched to proleptic Gregorian calendar too, so I didn't see anything in the tests that expects different results for Hive 2 and 3. We support running tests with both Hive 2 and 3, so I would have expected to see something like that. http://gerrit.cloudera.org:8080/#/c/13944/1/be/src/exec/hdfs-avro-scanner.cc File be/src/exec/hdfs-avro-scanner.cc: http://gerrit.cloudera.org:8080/#/c/13944/1/be/src/exec/hdfs-avro-scanner.cc@1025 PS1, Line 1025: read_field_fn = codegen->GetFunction(IRFunction::READ_AVRO_DATE, false); Why is the branch condition here different from the interpreted MaterializeTuple() function? In that function we were branching on both the schema type and the slot type. -- To view, visit http://gerrit.cloudera.org:8080/13944 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7a9d5b93a22cf3a00244037e187f8c145cacc959 Gerrit-Change-Number: 13944 Gerrit-PatchSet: 1 Gerrit-Owner: Attila Jeges Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Mon, 12 Aug 2019 22:06:53 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8791: Handle the case where there is no fragment scheduled on the coordinator
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/13992 ) Change subject: IMPALA-8791: Handle the case where there is no fragment scheduled on the coordinator .. IMPALA-8791: Handle the case where there is no fragment scheduled on the coordinator This patch fixes a bug where if an insert or CTAS query has no fragments scheduled on the coordinator and a mem limit is to be enforced on the query (either through query option or automatically through estimates) then the same limit is also applied to the coordinator backend even though it does not execute anything. Highlights: - coord_backend_mem_to_admit_/mem_limit will always refer to the memory to admit/limit for the coordinator regardless of which fragments are scheduled on it. - There will always be a BackendExecParams added for the coordinator because coordinator always spawns a QueryState object with a mem_tracker for tracking runtime filter mem and the result set cache. For the case where this BackendExecParams is empty (no instances scheduled) it would ensure that some minimal amount of memory is accounted for by the admission controller and the right mem limit is applied to the QueryState spawned by the coordinator - added changes to Coordinator and Coordinator::BackendState classes to handle an empty BackendExecParams object Testing: The following cases need to be tested where the kind of fragments schduled on the coordinator backend are: 1. Coordinator fragment + other exec fragments 2. Coordinator fragment only 3. other exec fragments only (eg. insert into values OR insert into select 1) 4. No fragments, but coordinator still creates a QueryState Case 1 is covered by tests working with non-dedicated coordinators. Rest are covered by test_mem_limit_dedicated_coordinator in test_admission_controller.py Change-Id: If5631fa1490d9612ffac3c4c4715348de47d6df2 Reviewed-on: http://gerrit.cloudera.org:8080/13992 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M be/src/runtime/coordinator-backend-state.cc M be/src/runtime/coordinator-backend-state.h M be/src/runtime/coordinator.cc M be/src/runtime/coordinator.h M be/src/scheduling/admission-controller-test.cc M be/src/scheduling/admission-controller.cc M be/src/scheduling/query-schedule.cc M be/src/scheduling/query-schedule.h M be/src/scheduling/scheduler.cc M be/src/scheduling/scheduler.h M fe/src/main/java/org/apache/impala/planner/Planner.java M testdata/workloads/functional-query/queries/QueryTest/dedicated-coord-mem-estimates.test 12 files changed, 154 insertions(+), 123 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/13992 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: If5631fa1490d9612ffac3c4c4715348de47d6df2 Gerrit-Change-Number: 13992 Gerrit-PatchSet: 5 Gerrit-Owner: Bikramjeet Vig Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-8847: Ignore add partition events with empty partition list
Vihang Karajgaonkar has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/14049 ) Change subject: IMPALA-8847: Ignore add partition events with empty partition list .. IMPALA-8847: Ignore add partition events with empty partition list Certain Hive queries like "alter table add if not exists partition ()" generate a add_partition event even if the partition did not really exists. Such events have a empty partition list in the event message which trips on the Precondition check in the AddPartitionEvent. This causes event processor to go into error state. The only way to recover is to issue invalidate metadata in such a case. The patch adds logic to ignore such events. Testing: 1. Added a test case which reproduces the issue. The test case works after the patch is applied. Change-Id: I877ce6233934e7090cd18e497f748bc6479838cb --- M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java M tests/custom_cluster/test_event_processing.py A tests/util/event_processor_utils.py 3 files changed, 162 insertions(+), 10 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/49/14049/4 -- To view, visit http://gerrit.cloudera.org:8080/14049 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I877ce6233934e7090cd18e497f748bc6479838cb Gerrit-Change-Number: 14049 Gerrit-PatchSet: 4 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar
[Impala-ASF-CR] IMPALA-8791: Handle the case where there is no fragment scheduled on the coordinator
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13992 ) Change subject: IMPALA-8791: Handle the case where there is no fragment scheduled on the coordinator .. Patch Set 4: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/13992 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If5631fa1490d9612ffac3c4c4715348de47d6df2 Gerrit-Change-Number: 13992 Gerrit-PatchSet: 4 Gerrit-Owner: Bikramjeet Vig Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Mon, 12 Aug 2019 21:55:53 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8847: Ignore add partition events with empty partition list
Vihang Karajgaonkar has posted comments on this change. ( http://gerrit.cloudera.org:8080/14049 ) Change subject: IMPALA-8847: Ignore add partition events with empty partition list .. Patch Set 3: (7 comments) http://gerrit.cloudera.org:8080/#/c/14049/3/tests/custom_cluster/test_event_processing.py File tests/custom_cluster/test_event_processing.py: http://gerrit.cloudera.org:8080/#/c/14049/3/tests/custom_cluster/test_event_processing.py@181 PS3, Line 181: def wait_for_insert_event_processing(self, previous_event_id): > I like that you moved out EventProcessorUtils. However, this file heavily u yes, I agree. I am planning to clean it up as part of IMPALA-8795. This method had some custom logic in place where it was waiting for 2 events so didn't want to touch that to keep the scope minimum. http://gerrit.cloudera.org:8080/#/c/14049/3/tests/custom_cluster/test_event_processing.py@200 PS3, Line 200: get_event_processor_metrics > Same as my above comment, this function can be removed here and modify this I am planning to clean up all the event polling tests together in IMPALA-8795 http://gerrit.cloudera.org:8080/#/c/14049/3/tests/custom_cluster/test_event_processing.py@215 PS3, Line 215: get_last_synced_event_id > Same as previous comment. I am planning to clean up all the event polling tests together in IMPALA-8795 http://gerrit.cloudera.org:8080/#/c/14049/3/tests/util/event_processor_utils.py File tests/util/event_processor_utils.py: http://gerrit.cloudera.org:8080/#/c/14049/3/tests/util/event_processor_utils.py@27 PS3, Line 27: class EventProcessorUtils(object): > flake8: E302 expected 2 blank lines, found 1 Done http://gerrit.cloudera.org:8080/#/c/14049/3/tests/util/event_processor_utils.py@30 PS3, Line 30: D > flake8: E303 too many blank lines (2) Done http://gerrit.cloudera.org:8080/#/c/14049/3/tests/util/event_processor_utils.py@50 PS3, Line 50: . > flake8: E131 continuation line unaligned for hanging indent Done http://gerrit.cloudera.org:8080/#/c/14049/3/tests/util/event_processor_utils.py@56 PS3, Line 56: @ > flake8: E303 too many blank lines (2) Done -- To view, visit http://gerrit.cloudera.org:8080/14049 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I877ce6233934e7090cd18e497f748bc6479838cb Gerrit-Change-Number: 14049 Gerrit-PatchSet: 3 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Mon, 12 Aug 2019 21:55:44 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8847: Ignore add partition events with empty partition list
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14049 ) Change subject: IMPALA-8847: Ignore add partition events with empty partition list .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/4224/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/14049 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I877ce6233934e7090cd18e497f748bc6479838cb Gerrit-Change-Number: 14049 Gerrit-PatchSet: 3 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Mon, 12 Aug 2019 21:51:45 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8847: Ignore add partition events with empty partition list
Anurag Mantripragada has posted comments on this change. ( http://gerrit.cloudera.org:8080/14049 ) Change subject: IMPALA-8847: Ignore add partition events with empty partition list .. Patch Set 3: Code-Review+1 +1 to the change in general. -- To view, visit http://gerrit.cloudera.org:8080/14049 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I877ce6233934e7090cd18e497f748bc6479838cb Gerrit-Change-Number: 14049 Gerrit-PatchSet: 3 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Mon, 12 Aug 2019 21:30:43 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8847: Ignore add partition events with empty partition list
Anurag Mantripragada has posted comments on this change. ( http://gerrit.cloudera.org:8080/14049 ) Change subject: IMPALA-8847: Ignore add partition events with empty partition list .. Patch Set 3: (3 comments) Overall, looks good to me. I like moving EventProcessorUtils out. If not in this change, would be nice to create a JIRA to clean up test_event_processing.py. We can use EventProcessorUtils for insert events too. http://gerrit.cloudera.org:8080/#/c/14049/3/tests/custom_cluster/test_event_processing.py File tests/custom_cluster/test_event_processing.py: http://gerrit.cloudera.org:8080/#/c/14049/3/tests/custom_cluster/test_event_processing.py@181 PS3, Line 181: def wait_for_insert_event_processing(self, previous_event_id): I like that you moved out EventProcessorUtils. However, this file heavily uses the same utils. Maybe we can create another JIRA to clean up this file. Especially, tests for insert events can be modified to use EventProcessorUtils functions. http://gerrit.cloudera.org:8080/#/c/14049/3/tests/custom_cluster/test_event_processing.py@200 PS3, Line 200: get_event_processor_metrics Same as my above comment, this function can be removed here and modify this file to use EventProcessorUtils. http://gerrit.cloudera.org:8080/#/c/14049/3/tests/custom_cluster/test_event_processing.py@215 PS3, Line 215: get_last_synced_event_id Same as previous comment. -- To view, visit http://gerrit.cloudera.org:8080/14049 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I877ce6233934e7090cd18e497f748bc6479838cb Gerrit-Change-Number: 14049 Gerrit-PatchSet: 3 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Mon, 12 Aug 2019 21:30:03 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8685,IMPALA-8677: Use consistent scheduling for small clusters
Joe McDonnell has posted comments on this change. (
http://gerrit.cloudera.org:8080/14026 )
Change subject: IMPALA-8685,IMPALA-8677: Use consistent scheduling for small
clusters
..
Patch Set 3:
(3 comments)
Working on a new upload
http://gerrit.cloudera.org:8080/#/c/14026/3/be/src/scheduling/scheduler.cc
File be/src/scheduling/scheduler.cc:
http://gerrit.cloudera.org:8080/#/c/14026/3/be/src/scheduling/scheduler.cc@796
PS3, Line 796: P((1/3)^(n-1))
> Is there a typo here ? Not sure what (P(1/3^(n-1)) means ?
Definitely a typo
http://gerrit.cloudera.org:8080/#/c/14026/3/be/src/scheduling/scheduler.cc@811
PS3, Line 811: if (candidates_it == remote_executor_candidates->end()) {
> This is now O(n^2), right? Is there a bound on num_executors and if so, sho
Yes, this is O(n^2).
We limit the num_remote_executor_candidates to be at most 16 via the query
option setting code. We also limit it to be the number of nodes if that is
smaller. The default is 3 and some systems are going to use 2. I doubt we are
going to set it higher than 3, so we could cut the maximum allowed value to 8
without any real problem.
I haven't benchmarked this.
http://gerrit.cloudera.org:8080/#/c/14026/3/be/src/scheduling/scheduler.cc@811
PS3, Line 811: if (candidates_it == remote_executor_candidates->end()) {
> Or we can consider using an unordered_set to track the candidates found so
My thinking was that when n is small, maintaining one structure (even though it
is O(n^2)) might still be better than maintaining two.
It is easy to go back to using the set. I would just put the IpAddrs directly
in the vector rather than iterating over the set at the end.
--
To view, visit http://gerrit.cloudera.org:8080/14026
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Icfdb2cc53d7206e316ea8a1cc28ad443f246f741
Gerrit-Change-Number: 14026
Gerrit-PatchSet: 3
Gerrit-Owner: Joe McDonnell
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Joe McDonnell
Gerrit-Reviewer: Lars Volker
Gerrit-Reviewer: Michael Ho
Gerrit-Comment-Date: Mon, 12 Aug 2019 21:27:42 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8752: Added Jaro-Winkler edit distance and similarity built-in function
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13870 ) Change subject: IMPALA-8752: Added Jaro-Winkler edit distance and similarity built-in function .. Patch Set 8: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/4773/ -- To view, visit http://gerrit.cloudera.org:8080/13870 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I64d7f461516c5e66cc27d62612bc8cc0e8f0178c Gerrit-Change-Number: 13870 Gerrit-PatchSet: 8 Gerrit-Owner: Norbert Luksa Gerrit-Reviewer: Greg Rahn Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Norbert Luksa Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Mon, 12 Aug 2019 21:19:13 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8847: Ignore add partition events with empty partition list
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14049 ) Change subject: IMPALA-8847: Ignore add partition events with empty partition list .. Patch Set 3: (4 comments) http://gerrit.cloudera.org:8080/#/c/14049/3/tests/util/event_processor_utils.py File tests/util/event_processor_utils.py: http://gerrit.cloudera.org:8080/#/c/14049/3/tests/util/event_processor_utils.py@27 PS3, Line 27: class EventProcessorUtils(object): flake8: E302 expected 2 blank lines, found 1 http://gerrit.cloudera.org:8080/#/c/14049/3/tests/util/event_processor_utils.py@30 PS3, Line 30: D flake8: E303 too many blank lines (2) http://gerrit.cloudera.org:8080/#/c/14049/3/tests/util/event_processor_utils.py@50 PS3, Line 50: . flake8: E131 continuation line unaligned for hanging indent http://gerrit.cloudera.org:8080/#/c/14049/3/tests/util/event_processor_utils.py@56 PS3, Line 56: @ flake8: E303 too many blank lines (2) -- To view, visit http://gerrit.cloudera.org:8080/14049 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I877ce6233934e7090cd18e497f748bc6479838cb Gerrit-Change-Number: 14049 Gerrit-PatchSet: 3 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Mon, 12 Aug 2019 21:11:36 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8847: Ignore add partition events with empty partition list
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14049 ) Change subject: IMPALA-8847: Ignore add partition events with empty partition list .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/4223/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/14049 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I877ce6233934e7090cd18e497f748bc6479838cb Gerrit-Change-Number: 14049 Gerrit-PatchSet: 2 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 12 Aug 2019 21:10:41 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8847: Ignore add partition events with empty partition list
Vihang Karajgaonkar has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/14049 ) Change subject: IMPALA-8847: Ignore add partition events with empty partition list .. IMPALA-8847: Ignore add partition events with empty partition list Certain Hive queries like "alter table add if not exists partition ()" generate a add_partition event even if the partition did not really exists. Such events have a empty partition list in the event message which trips on the Precondition check in the AddPartitionEvent. This causes event processor to go into error state. The only way to recover is to issue invalidate metadata in such a case. The patch adds logic to ignore such events. Testing: 1. Added a test case which reproduces the issue. The test case works after the patch is applied. Change-Id: I877ce6233934e7090cd18e497f748bc6479838cb --- M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java M tests/custom_cluster/test_event_processing.py A tests/util/event_processor_utils.py 3 files changed, 163 insertions(+), 10 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/49/14049/3 -- To view, visit http://gerrit.cloudera.org:8080/14049 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I877ce6233934e7090cd18e497f748bc6479838cb Gerrit-Change-Number: 14049 Gerrit-PatchSet: 3 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar
[Impala-ASF-CR] IMPALA-8847: Ignore add partition events with empty partition list
Vihang Karajgaonkar has posted comments on this change. ( http://gerrit.cloudera.org:8080/14049 ) Change subject: IMPALA-8847: Ignore add partition events with empty partition list .. Patch Set 2: (8 comments) http://gerrit.cloudera.org:8080/#/c/14049/2/tests/custom_cluster/test_event_processing.py File tests/custom_cluster/test_event_processing.py: http://gerrit.cloudera.org:8080/#/c/14049/2/tests/custom_cluster/test_event_processing.py@156 PS2, Line 156: > line has trailing whitespace Done http://gerrit.cloudera.org:8080/#/c/14049/2/tests/util/event_processor_utils.py File tests/util/event_processor_utils.py: http://gerrit.cloudera.org:8080/#/c/14049/2/tests/util/event_processor_utils.py@22 PS2, Line 22: import logging > flake8: F401 'logging' imported but unused Done http://gerrit.cloudera.org:8080/#/c/14049/2/tests/util/event_processor_utils.py@28 PS2, Line 28: class EventProcessorUtils(object): > flake8: E302 expected 2 blank lines, found 1 Done http://gerrit.cloudera.org:8080/#/c/14049/2/tests/util/event_processor_utils.py@31 PS2, Line 31: @ > flake8: E301 expected 1 blank line, found 0 Done http://gerrit.cloudera.org:8080/#/c/14049/2/tests/util/event_processor_utils.py@47 PS2, Line 47: T > flake8: F821 undefined name 'TimeoutError' Done http://gerrit.cloudera.org:8080/#/c/14049/2/tests/util/event_processor_utils.py@49 PS2, Line 49: . > flake8: E131 continuation line unaligned for hanging indent Done http://gerrit.cloudera.org:8080/#/c/14049/2/tests/util/event_processor_utils.py@53 PS2, Line 53: > line has trailing whitespace Done http://gerrit.cloudera.org:8080/#/c/14049/2/tests/util/event_processor_utils.py@62 PS2, Line 62: > line has trailing whitespace Done -- To view, visit http://gerrit.cloudera.org:8080/14049 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I877ce6233934e7090cd18e497f748bc6479838cb Gerrit-Change-Number: 14049 Gerrit-PatchSet: 2 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Mon, 12 Aug 2019 21:10:49 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8847: Ignore add partition events with empty partition list
Vihang Karajgaonkar has uploaded this change for review. ( http://gerrit.cloudera.org:8080/14049 Change subject: IMPALA-8847: Ignore add partition events with empty partition list .. IMPALA-8847: Ignore add partition events with empty partition list Certain Hive queries like "alter table add if not exists partition ()" generate a add_partition event even if the partition did not really exists. Such events have a empty partition list in the event message which trips on the Precondition check in the AddPartitionEvent. This causes event processor to go into error state. The only way to recover is to issue invalidate metadata in such a case. The patch adds logic to ignore such events. Testing: 1. Added a test case which reproduces the issue. The test case works after the patch is applied. Change-Id: I877ce6233934e7090cd18e497f748bc6479838cb --- M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java M tests/custom_cluster/test_event_processing.py A tests/util/event_processor_utils.py 3 files changed, 162 insertions(+), 10 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/49/14049/2 -- To view, visit http://gerrit.cloudera.org:8080/14049 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I877ce6233934e7090cd18e497f748bc6479838cb Gerrit-Change-Number: 14049 Gerrit-PatchSet: 2 Gerrit-Owner: Vihang Karajgaonkar
[Impala-ASF-CR] IMPALA-8839: Remove COLUMN STATS ACCURATE from properties
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14037 ) Change subject: IMPALA-8839: Remove COLUMN_STATS_ACCURATE from properties .. Patch Set 6: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/4222/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/14037 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I13f4a77022a7112e10a07314359f927eae083deb Gerrit-Change-Number: 14037 Gerrit-PatchSet: 6 Gerrit-Owner: Yongzhi Chen Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Yongzhi Chen Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Mon, 12 Aug 2019 19:24:22 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8160: [DOCS] Document CAST (...FORMAT..) function
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14044 ) Change subject: IMPALA-8160: [DOCS] Document CAST (...FORMAT..) function .. Patch Set 2: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/446/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/14044 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6235ffd03ac56e648552058ff02491a55289c092 Gerrit-Change-Number: 14044 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 12 Aug 2019 18:59:35 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8839: Remove COLUMN STATS ACCURATE from properties
Yongzhi Chen has posted comments on this change. (
http://gerrit.cloudera.org:8080/14037 )
Change subject: IMPALA-8839: Remove COLUMN_STATS_ACCURATE from properties
..
Patch Set 5:
(2 comments)
submit patch set 6
http://gerrit.cloudera.org:8080/#/c/14037/3/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
File fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java:
http://gerrit.cloudera.org:8080/#/c/14037/3/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@3762
PS3, Line 3762: table.getDb().getName(), table.getName());
: }
: }
> You could get the write id of INSERT the same way as we get the transaction
I got the conclusion of the same writeid by testing as the following:
This list the write ID in partitions table after 1. Hive did compute stats for
the whole table (analyze table insertonly_part_colstats compute statistics for
columns;) . 2. Impala insert a row with column stats accurate remove.
3. Hive compute stats for the whole table again. You can see the write ID in
partitions increased only 1 each time which means no waste of write id number.
And it consists with hive's insert statement. See the last select on
partitions(it is after hive insert a row to the 2010-01-01 partiton:
HMS_home_yongzhi_Impala_cdp=> select * from "PARTITIONS" where "TBL_ID"=3274;
PART_ID | CREATE_TIME | LAST_ACCESS_TIME | PART_NAME | SD_ID | TBL_ID |
WRITE_ID
-+-+--+---+---++--
15991 | 1565289739 |0 | ds=2010-01-01 | 19250 | 3274 |
18
15992 | 1565289739 |0 | ds=2010-01-02 | 19251 | 3274 |
18
(2 rows)
HMS_home_yongzhi_Impala_cdp=> select * from "PARTITIONS" where "TBL_ID"=3274;
PART_ID | CREATE_TIME | LAST_ACCESS_TIME | PART_NAME | SD_ID | TBL_ID |
WRITE_ID
-+-+--+---+---++--
15991 | 1565289739 |0 | ds=2010-01-01 | 19250 | 3274 |
19
15992 | 1565289739 |0 | ds=2010-01-02 | 19251 | 3274 |
18
(2 rows)
HMS_home_yongzhi_Impala_cdp=> select * from "PARTITION_PARAMS" where
"PART_ID"=15991;
PART_ID | PARAM_KEY | PARAM_VALUE
-+---+-
15991 | transient_lastDdlTime | 1565633500
15991 | numFiles | 4
15991 | totalSize | 8
15991 | numRows | 4
15991 | rawDataSize | 4
(5 rows)
HMS_home_yongzhi_Impala_cdp=> select * from "TBLS" where
"TBL_NAME"='insertonly_part_colstats';
HMS_home_yongzhi_Impala_cdp=> select * from "PARTITIONS" where "TBL_ID"=3274;
PART_ID | CREATE_TIME | LAST_ACCESS_TIME | PART_NAME | SD_ID | TBL_ID |
WRITE_ID
-+-+--+---+---++--
15991 | 1565289739 |0 | ds=2010-01-01 | 19250 | 3274 |
20
15992 | 1565289739 |0 | ds=2010-01-02 | 19251 | 3274 |
20
(2 rows)
HMS_home_yongzhi_Impala_cdp=> select * from "PARTITIONS" where "TBL_ID"=3274;
PART_ID | CREATE_TIME | LAST_ACCESS_TIME | PART_NAME | SD_ID | TBL_ID |
WRITE_ID
-+-+--+---+---++--
15991 | 1565289739 |0 | ds=2010-01-01 | 19250 | 3274 |
21
15992 | 1565289739 |0 | ds=2010-01-02 | 19251 | 3274 |
20
(2 rows)
http://gerrit.cloudera.org:8080/#/c/14037/5/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
File fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java:
http://gerrit.cloudera.org:8080/#/c/14037/5/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@3718
PS5, Line 3718: if (update.isSetTransaction_id()) {
: transactionId = update.getTransaction_id();
: }
> nit: fits single line
Done
--
To view, visit http://gerrit.cloudera.org:8080/14037
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I13f4a77022a7112e10a07314359f927eae083deb
Gerrit-Change-Number: 14037
Gerrit-PatchSet: 5
Gerrit-Owner: Yongzhi Chen
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Yongzhi Chen
Gerrit-Reviewer: Zoltan Borok-Nagy
Gerrit-Comment-Date: Mon, 12 Aug 2019 18:43:52 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8839: Remove COLUMN STATS ACCURATE from properties
Hello Zoltan Borok-Nagy, Csaba Ringhofer, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/14037 to look at the new patch set (#6). Change subject: IMPALA-8839: Remove COLUMN_STATS_ACCURATE from properties .. IMPALA-8839: Remove COLUMN_STATS_ACCURATE from properties Hive depends on property COLUMN_STATS_ACCURATE to tell if the stored statistics accurate. After Impala inserts data, it does not set statistics values up-to-date(for example numRows). Impala should unset COLUMN_STATS_ACCURATE to tell Hive the stored stats are no longer accurate. The patch impletes: After Impala insert data, Remove COLUMN_STATS_ACCURATE from table properties if it exists Remove COLUMN_STATS_ACCURATE from partition params if it exists Add helper methods to handle alter table/partition for acid tables. Implements the stats changes above for both acid/non-acid tables. Tests: Manual tests. Run core tests. Add ee tests to test interop with Hive for acid/external tables. Change-Id: I13f4a77022a7112e10a07314359f927eae083deb --- M fe/src/compat-hive-2/java/org/apache/impala/compat/MetastoreShim.java M fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java A testdata/workloads/functional-query/queries/QueryTest/acid-clear-statsaccurate.test A testdata/workloads/functional-query/queries/QueryTest/clear-statsaccurate.test M tests/query_test/test_acid.py 6 files changed, 340 insertions(+), 5 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/37/14037/6 -- To view, visit http://gerrit.cloudera.org:8080/14037 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I13f4a77022a7112e10a07314359f927eae083deb Gerrit-Change-Number: 14037 Gerrit-PatchSet: 6 Gerrit-Owner: Yongzhi Chen Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Yongzhi Chen Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR] IMPALA-8846: Undefined behaviour in RleEncoder::Put
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/14042 ) Change subject: IMPALA-8846: Undefined behaviour in RleEncoder::Put .. IMPALA-8846: Undefined behaviour in RleEncoder::Put To test for overflow, we used 'repeat_count_ <= std::numeric_limits::max()', but this is always true as repeat_count_ is an int. This could have lead to undefined behaviour because we increment repeat_count_ afterwards. Changed the comparison not to allow equality. Change-Id: I269443d1f1680e672fde7dd88eab5fcb56c65613 Reviewed-on: http://gerrit.cloudera.org:8080/14042 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M be/src/util/rle-encoding.h 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/14042 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I269443d1f1680e672fde7dd88eab5fcb56c65613 Gerrit-Change-Number: 14042 Gerrit-PatchSet: 3 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-8846: Undefined behaviour in RleEncoder::Put
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14042 ) Change subject: IMPALA-8846: Undefined behaviour in RleEncoder::Put .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/14042 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I269443d1f1680e672fde7dd88eab5fcb56c65613 Gerrit-Change-Number: 14042 Gerrit-PatchSet: 2 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 12 Aug 2019 18:33:25 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8160: [DOCS] Document CAST (...FORMAT..) function
Hello Gabor Kaszab, Attila Jeges, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/14044 to look at the new patch set (#2). Change subject: IMPALA-8160: [DOCS] Document CAST (...FORMAT..) function .. IMPALA-8160: [DOCS] Document CAST (...FORMAT..) function - Added the Date and Timestamp patterns supported for the new CAST signature. Change-Id: I6235ffd03ac56e648552058ff02491a55289c092 --- M docs/topics/impala_conversion_functions.xml 1 file changed, 667 insertions(+), 65 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/44/14044/2 -- To view, visit http://gerrit.cloudera.org:8080/14044 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I6235ffd03ac56e648552058ff02491a55289c092 Gerrit-Change-Number: 14044 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-8160: [DOCS] Document CAST (...FORMAT..) function
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14044 ) Change subject: IMPALA-8160: [DOCS] Document CAST (...FORMAT..) function .. Patch Set 2: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/446/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/14044 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6235ffd03ac56e648552058ff02491a55289c092 Gerrit-Change-Number: 14044 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 12 Aug 2019 18:30:40 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8160: [DOCS] Document CAST (...FORMAT..) function
Alex Rodoni has posted comments on this change. ( http://gerrit.cloudera.org:8080/14044 ) Change subject: IMPALA-8160: [DOCS] Document CAST (...FORMAT..) function .. Patch Set 1: (4 comments) http://gerrit.cloudera.org:8080/#/c/14044/1/docs/topics/impala_conversion_functions.xml File docs/topics/impala_conversion_functions.xml: http://gerrit.cloudera.org:8080/#/c/14044/1/docs/topics/impala_conversion_functions.xml@84 PS1, Line 84: If the expression value is of a type that cannot be converted to : the target type, e.g. from TIMESTAMP to : DECIMAL, the function returns NULL. > DECIMAL and DATE types are exceptions to this rule: they return an error in BOOLEAN returned an error. Noted those 3 types. http://gerrit.cloudera.org:8080/#/c/14044/1/docs/topics/impala_conversion_functions.xml@175 PS1, Line 175: TIMESTAMP > This should be DATE too, if in L176 the expected result is of DATE type. Done http://gerrit.cloudera.org:8080/#/c/14044/1/docs/topics/impala_conversion_functions.xml@528 PS1, Line 528: -15 and 15 > I'm not sure if this is the valid range. Please check with Gabor Kaszab. Gabor? http://gerrit.cloudera.org:8080/#/c/14044/1/docs/topics/impala_conversion_functions.xml@555 PS1, Line 555: 0 and 59 > I'm not sure if this is the valid range. Please check with Gabor Kaszab. Gabor? -- To view, visit http://gerrit.cloudera.org:8080/14044 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6235ffd03ac56e648552058ff02491a55289c092 Gerrit-Change-Number: 14044 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 12 Aug 2019 18:29:30 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8755: Frontend support for Z-ordering
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13955 ) Change subject: IMPALA-8755: Frontend support for Z-ordering .. Patch Set 7: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/4221/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13955 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie122002ca8f52ca2c1e1ec8ff1d476ae1f4f875d Gerrit-Change-Number: 13955 Gerrit-PatchSet: 7 Gerrit-Owner: Norbert Luksa Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Norbert Luksa Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Mon, 12 Aug 2019 17:48:22 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7770: SPLIT PART to support negative indexes
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13880 ) Change subject: IMPALA-7770: SPLIT_PART to support negative indexes .. Patch Set 5: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/4220/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13880 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2db762989a90bd95661a59eb9c11a29eb2edfafb Gerrit-Change-Number: 13880 Gerrit-PatchSet: 5 Gerrit-Owner: Norbert Luksa Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Norbert Luksa Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Mon, 12 Aug 2019 17:48:04 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8791: Handle the case where there is no fragment scheduled on the coordinator
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13992 ) Change subject: IMPALA-8791: Handle the case where there is no fragment scheduled on the coordinator .. Patch Set 4: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/13992 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If5631fa1490d9612ffac3c4c4715348de47d6df2 Gerrit-Change-Number: 13992 Gerrit-PatchSet: 4 Gerrit-Owner: Bikramjeet Vig Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Mon, 12 Aug 2019 17:47:34 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8791: Handle the case where there is no fragment scheduled on the coordinator
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13992 ) Change subject: IMPALA-8791: Handle the case where there is no fragment scheduled on the coordinator .. Patch Set 4: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4774/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/13992 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If5631fa1490d9612ffac3c4c4715348de47d6df2 Gerrit-Change-Number: 13992 Gerrit-PatchSet: 4 Gerrit-Owner: Bikramjeet Vig Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Mon, 12 Aug 2019 17:47:35 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8837: [DOCS] HTTP support for proxy/delegation connection
Alex Rodoni has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/14033 ) Change subject: IMPALA-8837: [DOCS] HTTP support for proxy/delegation connection .. IMPALA-8837: [DOCS] HTTP support for proxy/delegation connection - Added a line on Knox support. Change-Id: I591e0fd736ea114aa52a999acf41806a94e49382 Reviewed-on: http://gerrit.cloudera.org:8080/14033 Tested-by: Impala Public Jenkins Reviewed-by: Thomas Tauber-Marshall --- M docs/topics/impala_authentication.xml M docs/topics/impala_delegation.xml 2 files changed, 14 insertions(+), 6 deletions(-) Approvals: Impala Public Jenkins: Verified Thomas Tauber-Marshall: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/14033 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I591e0fd736ea114aa52a999acf41806a94e49382 Gerrit-Change-Number: 14033 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall
[Impala-ASF-CR] IMPALA-7374: [DOCS] Document the new DATE data type in Impala
Alex Rodoni has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/13983 ) Change subject: IMPALA-7374: [DOCS] Document the new DATE data type in Impala .. IMPALA-7374: [DOCS] Document the new DATE data type in Impala Change-Id: I0c28361c7f0d225708eaf4b955c6704520eaaa68 Reviewed-on: http://gerrit.cloudera.org:8080/13983 Tested-by: Impala Public Jenkins Reviewed-by: Attila Jeges --- M docs/impala.ditamap M docs/topics/impala_date.xml M docs/topics/impala_literals.xml 3 files changed, 329 insertions(+), 189 deletions(-) Approvals: Impala Public Jenkins: Verified Attila Jeges: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/13983 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I0c28361c7f0d225708eaf4b955c6704520eaaa68 Gerrit-Change-Number: 13983 Gerrit-PatchSet: 6 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-8837: [DOCS] HTTP support for proxy/delegation connection
Thomas Tauber-Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/14033 ) Change subject: IMPALA-8837: [DOCS] HTTP support for proxy/delegation connection .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/14033 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I591e0fd736ea114aa52a999acf41806a94e49382 Gerrit-Change-Number: 14033 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Mon, 12 Aug 2019 17:34:43 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8818: Replace deque with spillable queue in BufferedPRS
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14039 ) Change subject: IMPALA-8818: Replace deque with spillable queue in BufferedPRS .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/4219/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/14039 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I10f9e72374cdf9501c0e5e2c5b39c13688ae65a9 Gerrit-Change-Number: 14039 Gerrit-PatchSet: 4 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Mon, 12 Aug 2019 17:22:57 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8766: Undo hadoop-cloud-storage + HWX Nexus
Vihang Karajgaonkar has posted comments on this change. ( http://gerrit.cloudera.org:8080/14024 ) Change subject: IMPALA-8766: Undo hadoop-cloud-storage + HWX Nexus .. Patch Set 3: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/14024 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I79a0c2575fc50bbc3b393c150c0bce22258ea1bd Gerrit-Change-Number: 14024 Gerrit-PatchSet: 3 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Mon, 12 Aug 2019 17:14:48 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8752: Added Jaro-Winkler edit distance and similarity built-in function
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13870 ) Change subject: IMPALA-8752: Added Jaro-Winkler edit distance and similarity built-in function .. Patch Set 8: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/13870 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I64d7f461516c5e66cc27d62612bc8cc0e8f0178c Gerrit-Change-Number: 13870 Gerrit-PatchSet: 8 Gerrit-Owner: Norbert Luksa Gerrit-Reviewer: Greg Rahn Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Norbert Luksa Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Mon, 12 Aug 2019 17:09:53 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8752: Added Jaro-Winkler edit distance and similarity built-in function
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13870 ) Change subject: IMPALA-8752: Added Jaro-Winkler edit distance and similarity built-in function .. Patch Set 8: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4773/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/13870 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I64d7f461516c5e66cc27d62612bc8cc0e8f0178c Gerrit-Change-Number: 13870 Gerrit-PatchSet: 8 Gerrit-Owner: Norbert Luksa Gerrit-Reviewer: Greg Rahn Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Norbert Luksa Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Mon, 12 Aug 2019 17:09:54 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8752: Added Jaro-Winkler edit distance and similarity built-in function
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/13870 ) Change subject: IMPALA-8752: Added Jaro-Winkler edit distance and similarity built-in function .. Patch Set 7: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/13870 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I64d7f461516c5e66cc27d62612bc8cc0e8f0178c Gerrit-Change-Number: 13870 Gerrit-PatchSet: 7 Gerrit-Owner: Norbert Luksa Gerrit-Reviewer: Greg Rahn Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Norbert Luksa Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Mon, 12 Aug 2019 17:09:27 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7770: SPLIT PART to support negative indexes
Norbert Luksa has uploaded a new patch set (#5). ( http://gerrit.cloudera.org:8080/13880 ) Change subject: IMPALA-7770: SPLIT_PART to support negative indexes .. IMPALA-7770: SPLIT_PART to support negative indexes Third parameter of SPLIT_PART (nth field) accepts now negative values, and searches the string backwards. Testing: * Added unint tests to expr-test.cc Change-Id: I2db762989a90bd95661a59eb9c11a29eb2edfafb --- M be/src/exprs/expr-test.cc M be/src/exprs/string-functions-ir.cc 2 files changed, 37 insertions(+), 12 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/80/13880/5 -- To view, visit http://gerrit.cloudera.org:8080/13880 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I2db762989a90bd95661a59eb9c11a29eb2edfafb Gerrit-Change-Number: 13880 Gerrit-PatchSet: 5 Gerrit-Owner: Norbert Luksa Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Norbert Luksa Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR] IMPALA-8839: Remove COLUMN STATS ACCURATE from properties
Zoltan Borok-Nagy has posted comments on this change. (
http://gerrit.cloudera.org:8080/14037 )
Change subject: IMPALA-8839: Remove COLUMN_STATS_ACCURATE from properties
..
Patch Set 5:
(2 comments)
Did a quick initial pass over it. Looks good to me in overall but I'm planning
to do another pass tomorrow.
http://gerrit.cloudera.org:8080/#/c/14037/3/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
File fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java:
http://gerrit.cloudera.org:8080/#/c/14037/3/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@3762
PS3, Line 3762: table.getDb().getName(), table.getName());
: }
: }
> From my test, it seems the same value. How to get Insert statement's writeI
You could get the write id of INSERT the same way as we get the transaction id,
i.e. putting it in the relevant thrift object and transfer it from the
coordinator.
But since allocateTableWriteId() returns the same write id I think it's not a
problem to get it this way. It's just one extra round-trip to HMS.
http://gerrit.cloudera.org:8080/#/c/14037/5/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
File fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java:
http://gerrit.cloudera.org:8080/#/c/14037/5/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@3718
PS5, Line 3718: if (update.isSetTransaction_id()) {
: transactionId = update.getTransaction_id();
: }
nit: fits single line
--
To view, visit http://gerrit.cloudera.org:8080/14037
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I13f4a77022a7112e10a07314359f927eae083deb
Gerrit-Change-Number: 14037
Gerrit-PatchSet: 5
Gerrit-Owner: Yongzhi Chen
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Yongzhi Chen
Gerrit-Reviewer: Zoltan Borok-Nagy
Gerrit-Comment-Date: Mon, 12 Aug 2019 17:08:15 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8755: Frontend support for Z-ordering
Norbert Luksa has posted comments on this change. ( http://gerrit.cloudera.org:8080/13955 ) Change subject: IMPALA-8755: Frontend support for Z-ordering .. Patch Set 6: (4 comments) http://gerrit.cloudera.org:8080/#/c/13955/6/common/thrift/DataSinks.thrift File common/thrift/DataSinks.thrift: http://gerrit.cloudera.org:8080/#/c/13955/6/common/thrift/DataSinks.thrift@83 PS6, Line 83: // Sorting algorithm. If not lexical, the backend should not populate the : // RowGroup::sorting_columns list in parquet files. : 7: required Types.TSortingAlgorithm sorting_algorithm > I think it's rather an 'order' than an 'algorithm'. Algorithm is e.g. quick You are right, replaced 'algorithm' with 'order'. The Parquet stuff is relevant to the sort_columns parameter above, and I would leave it there for clarification. http://gerrit.cloudera.org:8080/#/c/13955/6/fe/src/main/cup/sql-parser.cup File fe/src/main/cup/sql-parser.cup: http://gerrit.cloudera.org:8080/#/c/13955/6/fe/src/main/cup/sql-parser.cup@1399 PS6, Line 1399: RESULT = new CreateTableLikeStmt(tbl_def.getTblName(), > nit: whitespaces at line end Done http://gerrit.cloudera.org:8080/#/c/13955/6/fe/src/test/java/org/apache/impala/analysis/AnalyzeDDLTest.java File fe/src/test/java/org/apache/impala/analysis/AnalyzeDDLTest.java: http://gerrit.cloudera.org:8080/#/c/13955/6/fe/src/test/java/org/apache/impala/analysis/AnalyzeDDLTest.java@2570 PS6, Line 2570: > nit: missing words Done http://gerrit.cloudera.org:8080/#/c/13955/6/testdata/workloads/functional-query/queries/QueryTest/create-table-like-table-zorder.test File testdata/workloads/functional-query/queries/QueryTest/create-table-like-table-zorder.test: http://gerrit.cloudera.org:8080/#/c/13955/6/testdata/workloads/functional-query/queries/QueryTest/create-table-like-table-zorder.test@24 PS6, Line 24: string_col > We shouldn't allow string cols to be sorted by ZORDER. It's better to prohi Done -- To view, visit http://gerrit.cloudera.org:8080/13955 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie122002ca8f52ca2c1e1ec8ff1d476ae1f4f875d Gerrit-Change-Number: 13955 Gerrit-PatchSet: 6 Gerrit-Owner: Norbert Luksa Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Norbert Luksa Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Mon, 12 Aug 2019 16:52:15 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8818: Replace deque with spillable queue in BufferedPRS
Hello Michael Ho, Tim Armstrong, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/14039 to look at the new patch set (#4). Change subject: IMPALA-8818: Replace deque with spillable queue in BufferedPRS .. IMPALA-8818: Replace deque with spillable queue in BufferedPRS Replaces DequeRowBatchQueue with SpillableRowBatchQueue in BufferedPlanRootSink. A few changes to BufferedPlanRootSink were necessary for it to work with the spillable queue, however, all the synchronization logic is the same. SpillableRowBatchQueue is a wrapper around a BufferedTupleStream and a ReservationManager. It takes in a TBackendResourceProfile that specifies the max / min memory reservation the BufferedTupleStream can use to buffer rows. The 'max_unpinned_bytes' parameter limits the max number of bytes that can be unpinned in the BufferedTupleStream. The limit is a 'soft' limit because calls to AddBatch may push the amount of unpinned memory over the limit. The queue is non-blocking and not thread safe. It provides AddBatch and GetBatch methods. Calls to AddBatch spill if the BufferedTupleStream does not have enough reservation to fit the entire RowBatch. Adds two new query options: 'MAX_PINNED_RESULT_SPOOLING_MEMORY' and 'MAX_UNPINNED_RESULT_SPOOLING_MEMORY', which bound the amount of pinned and unpinned memory that a query can use for spooling, respectively. MAX_PINNED_RESULT_SPOOLING_MEMORY must be <= MAX_UNPINNED_RESULT_SPOOLING_MEMORY in order to allow all the pinned data in the BufferedTupleStream to be unpinned. This is enforced in a new method in QueryOptions called 'ValidateQueryOptions'. Planner Changes: PlanRootSink.java now computes a full ResourceProfile if result spooling is enabled. The min mem reservation is bounded by the size of the read and write pages used by the BufferedTupleStream. The max mem reservation is bounded by 'MAX_PINNED_RESULT_SPOOLING_MEMORY'. The mem estimate is computed by estimating the size of the result set using stats. BufferedTupleStream Re-Factoring: For the most part, using a BufferedTupleStream outside an ExecNode works properly. However, some changes were necessary: * The message for the MAX_ROW_SIZE error is ExecNode specific. In order to fix this, this patch introduces the concept of an ExecNode 'label' which is a more generic version of an ExecNode 'id'. * The definition of TBackendResourceProfile lived in PlanNodes.thrift, it was moved to its own file so it can be used by DataSinks.thrift. * Modified BufferedTupleStream so it internally tracks how many bytes are unpinned (necessary for 'MAX_UNPINNED_RESULT_SPOOLING_MEMORY'). Metrics: * Added a few of the metrics mentioned in IMPALA-8825 to BufferedPlanRootSink. Specifically, added timers to track how much time is spent waiting in the BufferedPlanRootSink 'Send' and 'GetNext' methods. * The BufferedTupleStream in the SpillableRowBatchQueue exposes several BufferPool metrics such as number of reserved and unpinned bytes. Bug Fixes: * Fixed a bug in BufferedPlanRootSink where the MemPool used by the expression evaluators was not being cleared incrementally. * Fixed a bug where the inactive timer was not being properly updated in BufferedPlanRootSink. * Fixed a bug where RowBatch memory was not freed if BufferedPlanRootSink::GetNext terminated early because it could not handle requests where num_results < BATCH_SIZE. Testing: * Added new tests to test_result_spooling.py. * Updated errors thrown in spilling-large-rows.test. * Ran exhaustive tests. Change-Id: I10f9e72374cdf9501c0e5e2c5b39c13688ae65a9 --- M be/generated-sources/gen-cpp/CMakeLists.txt M be/src/exec/analytic-eval-node.cc M be/src/exec/blocking-plan-root-sink.cc M be/src/exec/blocking-plan-root-sink.h M be/src/exec/buffered-plan-root-sink.cc M be/src/exec/buffered-plan-root-sink.h M be/src/exec/data-sink.cc M be/src/exec/exec-node.cc M be/src/exec/exec-node.h M be/src/exec/grouping-aggregator-partition.cc M be/src/exec/grouping-aggregator.cc M be/src/exec/partial-sort-node.cc M be/src/exec/partitioned-hash-join-builder.cc M be/src/exec/partitioned-hash-join-builder.h M be/src/exec/partitioned-hash-join-node.cc M be/src/exec/plan-root-sink.h M be/src/exec/sort-node.cc M be/src/runtime/CMakeLists.txt M be/src/runtime/buffered-tuple-stream-test.cc M be/src/runtime/buffered-tuple-stream.cc M be/src/runtime/buffered-tuple-stream.h M be/src/runtime/sorter.cc M be/src/runtime/sorter.h A be/src/runtime/spillable-row-batch-queue.cc A be/src/runtime/spillable-row-batch-queue.h M be/src/service/impala-beeswax-server.cc M be/src/service/impala-server.cc M be/src/service/query-options-test.cc M be/src/service/query-options.cc M be/src/service/query-options.h M common/thrift/CMakeLists.txt M common/thrift/DataSinks.thrift M common/thrift/ImpalaInternalService.thrift M common/thrift/ImpalaService.thrift M common/thrift/PlanNodes.thrift A
[Impala-ASF-CR] IMPALA-8160: [DOCS] Document CAST (...FORMAT..) function
Attila Jeges has posted comments on this change. (
http://gerrit.cloudera.org:8080/14044 )
Change subject: IMPALA-8160: [DOCS] Document CAST (...FORMAT..) function
..
Patch Set 1:
(4 comments)
http://gerrit.cloudera.org:8080/#/c/14044/1/docs/topics/impala_conversion_functions.xml
File docs/topics/impala_conversion_functions.xml:
http://gerrit.cloudera.org:8080/#/c/14044/1/docs/topics/impala_conversion_functions.xml@84
PS1, Line 84: If the expression value is of a type that
cannot be converted to
: the target type, e.g. from
TIMESTAMP to
: DECIMAL, the function returns
NULL.
DECIMAL and DATE types are exceptions to this rule: they return an error
instead.
impala> select cast('123x' as decimal);
ERROR: UDF ERROR: String to Decimal parse failed
http://gerrit.cloudera.org:8080/#/c/14044/1/docs/topics/impala_conversion_functions.xml@175
PS1, Line 175: TIMESTAMP
This should be DATE too, if in L176 the expected result is of DATE type.
http://gerrit.cloudera.org:8080/#/c/14044/1/docs/topics/impala_conversion_functions.xml@528
PS1, Line 528: -15 and 15
I'm not sure if this is the valid range. Please check with Gabor Kaszab.
http://gerrit.cloudera.org:8080/#/c/14044/1/docs/topics/impala_conversion_functions.xml@555
PS1, Line 555: 0 and 59
I'm not sure if this is the valid range. Please check with Gabor Kaszab.
--
To view, visit http://gerrit.cloudera.org:8080/14044
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I6235ffd03ac56e648552058ff02491a55289c092
Gerrit-Change-Number: 14044
Gerrit-PatchSet: 1
Gerrit-Owner: Alex Rodoni
Gerrit-Reviewer: Attila Jeges
Gerrit-Reviewer: Gabor Kaszab
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Comment-Date: Mon, 12 Aug 2019 16:27:30 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8685,IMPALA-8677: Use consistent scheduling for small clusters
Lars Volker has posted comments on this change. (
http://gerrit.cloudera.org:8080/14026 )
Change subject: IMPALA-8685,IMPALA-8677: Use consistent scheduling for small
clusters
..
Patch Set 3:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/14026/3/be/src/scheduling/scheduler.cc
File be/src/scheduling/scheduler.cc:
http://gerrit.cloudera.org:8080/#/c/14026/3/be/src/scheduling/scheduler.cc@545
PS3, Line 545: int num_remote_executor_candidates =
query_options.num_remote_executor_candidates;
: if (executor_group.NumExecutors() <
num_remote_executor_candidates) {
: num_remote_executor_candidates =
executor_group.NumExecutors();
: }
nit: use std::min
http://gerrit.cloudera.org:8080/#/c/14026/3/be/src/scheduling/scheduler.cc@811
PS3, Line 811: if (candidates_it == remote_executor_candidates->end()) {
This is now O(n^2), right? Is there a bound on num_executors and if so, should
we add a DCHECK to make sure it's not large? Have you benchmarked this to see
if it changes the runtime significantly?
--
To view, visit http://gerrit.cloudera.org:8080/14026
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Icfdb2cc53d7206e316ea8a1cc28ad443f246f741
Gerrit-Change-Number: 14026
Gerrit-PatchSet: 3
Gerrit-Owner: Joe McDonnell
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Joe McDonnell
Gerrit-Reviewer: Lars Volker
Gerrit-Reviewer: Michael Ho
Gerrit-Comment-Date: Mon, 12 Aug 2019 16:15:11 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7984: Port runtime filter from Thrift RPC to KRPC
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13882 ) Change subject: IMPALA-7984: Port runtime filter from Thrift RPC to KRPC .. Patch Set 14: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/4218/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13882 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6b394796d250286510e157ae326882bfc01d387a Gerrit-Change-Number: 13882 Gerrit-PatchSet: 14 Gerrit-Owner: Fang-Yu Rao Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Mon, 12 Aug 2019 15:39:49 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7984: Port runtime filter from Thrift RPC to KRPC
Fang-Yu Rao has uploaded a new patch set (#14). ( http://gerrit.cloudera.org:8080/13882 ) Change subject: IMPALA-7984: Port runtime filter from Thrift RPC to KRPC .. IMPALA-7984: Port runtime filter from Thrift RPC to KRPC Previously the aggregation and propagation of a runtime filter in Impala is implemented using Thrift RPC, which suffers from a disadvantage that the number of connections in a cluster grows with both the number of queries and cluster size. This patch ports the functions that implement the aggregation and propagation of a runtime filter, i.e., UpdateFilter() and PublishFilter(), respctively, to KRPC, which requires only one connection per direction between every pair of hosts, thus reducing the number of connections in a cluster. In addition, this patch also incorporates KRPC sidecar when the runtime filter is a Bloom filter. KRPC sidecar eliminates the need for an extra copy of the Bloom filter contents when a Bloom filter is serialized to be transmitted and hence reduces the serialization overhead. Two related BE tests bloom-filter-test.cc and bloom-filter-benchmark.cc are also modified accordingly. TODO: To remove unnecessary code related to ImpalaInternalService. Testing: This patch has passed the exhaustive tests. Change-Id: I6b394796d250286510e157ae326882bfc01d387a --- M be/src/benchmarks/bloom-filter-benchmark.cc M be/src/rpc/thrift-server-test.cc M be/src/runtime/backend-client.h M be/src/runtime/client-cache.cc M be/src/runtime/coordinator-backend-state.cc M be/src/runtime/coordinator-backend-state.h M be/src/runtime/coordinator-filter-state.h M be/src/runtime/coordinator.cc M be/src/runtime/coordinator.h M be/src/runtime/data-stream-test.cc M be/src/runtime/decimal-value.h M be/src/runtime/decimal-value.inline.h M be/src/runtime/exec-env.cc M be/src/runtime/fragment-instance-state.cc M be/src/runtime/fragment-instance-state.h M be/src/runtime/krpc-data-stream-recvr.cc M be/src/runtime/query-state.cc M be/src/runtime/query-state.h M be/src/runtime/runtime-filter-bank.cc M be/src/runtime/runtime-filter-bank.h M be/src/runtime/timestamp-value.h M be/src/scheduling/request-pool-service.h M be/src/service/client-request-state.cc M be/src/service/client-request-state.h M be/src/service/data-stream-service.cc M be/src/service/data-stream-service.h M be/src/service/frontend.h M be/src/service/impala-internal-service.cc M be/src/service/impala-internal-service.h M be/src/service/impala-server.cc M be/src/service/impala-server.h M be/src/util/bloom-filter-test.cc M be/src/util/bloom-filter.cc M be/src/util/bloom-filter.h M be/src/util/min-max-filter-test.cc M be/src/util/min-max-filter.cc M be/src/util/min-max-filter.h M common/protobuf/common.proto M common/protobuf/data_stream_service.proto M common/thrift/ImpalaInternalService.thrift 40 files changed, 1,063 insertions(+), 728 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/82/13882/14 -- To view, visit http://gerrit.cloudera.org:8080/13882 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I6b394796d250286510e157ae326882bfc01d387a Gerrit-Change-Number: 13882 Gerrit-PatchSet: 14 Gerrit-Owner: Fang-Yu Rao Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Thomas Tauber-Marshall
[Impala-ASF-CR] IMPALA-8846: Undefined behaviour in RleEncoder::Put
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14042 ) Change subject: IMPALA-8846: Undefined behaviour in RleEncoder::Put .. Patch Set 2: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4772/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/14042 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I269443d1f1680e672fde7dd88eab5fcb56c65613 Gerrit-Change-Number: 14042 Gerrit-PatchSet: 2 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 12 Aug 2019 14:22:58 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8846: Undefined behaviour in RleEncoder::Put
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14042 ) Change subject: IMPALA-8846: Undefined behaviour in RleEncoder::Put .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/14042 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I269443d1f1680e672fde7dd88eab5fcb56c65613 Gerrit-Change-Number: 14042 Gerrit-PatchSet: 2 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 12 Aug 2019 14:22:58 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8703: ISO:SQL:2016 datetime patterns - Milestone 1
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13722 ) Change subject: IMPALA-8703: ISO:SQL:2016 datetime patterns - Milestone 1 .. Patch Set 13: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4771/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/13722 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I19d8d097a45ae6f103b6cd1b2d81aad38dfd9e23 Gerrit-Change-Number: 13722 Gerrit-PatchSet: 13 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 12 Aug 2019 08:47:21 + Gerrit-HasComments: No
