[Impala-ASF-CR](asf-site) Added changelog for 3.1.0
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/12011 ) Change subject: Added changelog for 3.1.0 .. Patch Set 1: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/12011 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: asf-site Gerrit-MessageType: comment Gerrit-Change-Id: I244b68a8b2ef4ac373e857c73326da5425ab66d3 Gerrit-Change-Number: 12011 Gerrit-PatchSet: 1 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Thu, 06 Dec 2018 11:10:30 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7853: Add support to read int64 NANO timestamps from Parquet
Gabor Kaszab has posted comments on this change. ( http://gerrit.cloudera.org:8080/11984 ) Change subject: IMPALA-7853: Add support to read int64 NANO timestamps from Parquet .. Patch Set 5: Code-Review+2 (1 comment) http://gerrit.cloudera.org:8080/#/c/11984/2/be/src/exec/parquet-common.cc File be/src/exec/parquet-common.cc: http://gerrit.cloudera.org:8080/#/c/11984/2/be/src/exec/parquet-common.cc@148 PS2, Line 148: > I would vote for returning to the last solution. Sure, I'm convinced. Sorry for the extra few rounds :) -- To view, visit http://gerrit.cloudera.org:8080/11984 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I932396d8646f43c0b9ca4a6359f164c4d8349d8f Gerrit-Change-Number: 11984 Gerrit-PatchSet: 5 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Thu, 06 Dec 2018 19:00:08 + Gerrit-HasComments: Yes
[Impala-ASF-CR](asf-site) [DOCS] Impala 3.1 Docs to be published
Alex Rodoni has posted comments on this change. ( http://gerrit.cloudera.org:8080/12046 ) Change subject: [DOCS] Impala 3.1 Docs to be published .. Patch Set 1: > (1 comment) Yes, some docs commits didn't make it into branch-3.1.0, but there is nothing 3.2-specific in docs yet, so 5189300b2f9b12f147c8dfd39bd5e30613fcfbfeis fine. -- To view, visit http://gerrit.cloudera.org:8080/12046 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: asf-site Gerrit-MessageType: comment Gerrit-Change-Id: Icdfa4df8f75090b4783912713e2820eacbdfa904 Gerrit-Change-Number: 12046 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Michael Brown Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Thu, 06 Dec 2018 21:08:19 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7821: Refactor ExprRewriteRulesTest for easier debugging
Bharath Vissapragada has posted comments on this change. ( http://gerrit.cloudera.org:8080/11890 ) Change subject: IMPALA-7821: Refactor ExprRewriteRulesTest for easier debugging .. Patch Set 4: I did a quick pass on the child builds and everything seems to have finished. It is unclear where it is hung. I'm doing force kill for now (unless someone knows if we should collect any diagnostics before we do that). Pipeline] [AllTests] build (Building ubuntu-16.04-from-scratch) 07:48:25 [AllTests] Scheduling project: ubuntu-16.04-from-scratch [Pipeline] [UBSAN] build (Building ubuntu-16.04-from-scratch) 07:48:25 [UBSAN] Scheduling project: ubuntu-16.04-from-scratch [Pipeline] [BuildOptions] build (Building all-build-options-ub1604) 07:48:25 [BuildOptions] Scheduling project: all-build-options-ub1604 [Pipeline] [Python26Compatibility] build (Building python26-incompatibility-check) 07:48:25 [Python26Compatibility] Scheduling project: python26-incompatibility-check [Pipeline] [TidyAndBuildOnlyAndRat] build (Building clang-tidy-ub1604) 07:48:25 [TidyAndBuildOnlyAndRat] Scheduling project: clang-tidy-ub1604 07:50:53 [Python26Compatibility] Starting building: python26-incompatibility-check #1057 [Pipeline] [Python26Compatibility] } 07:53:00 [AllTests] Starting building: ubuntu-16.04-from-scratch #3793 [Pipeline] [AllTests] } 17:48:25 Cancelling nested steps due to timeout 17:49:25 Body did not finish within grace period; terminating with extreme prejudice -- To view, visit http://gerrit.cloudera.org:8080/11890 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I99a28b170bc0132424041e51f61ebe5c848c4083 Gerrit-Change-Number: 11890 Gerrit-PatchSet: 4 Gerrit-Owner: Paul Rogers Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Paul Rogers Gerrit-Comment-Date: Thu, 06 Dec 2018 19:59:48 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7659: Populate NULL count while computing column stats
Bharath Vissapragada has uploaded a new patch set (#9) to the change originally created by piotr.findei...@gmail.com. ( http://gerrit.cloudera.org:8080/11565 ) Change subject: IMPALA-7659: Populate NULL count while computing column stats .. IMPALA-7659: Populate NULL count while computing column stats It was disabled for performance reasons (IMPALA-1003) and this patch re-enables it since a lot of codegen improvements have happened since then. This patch switches the aggregation to use the CASE conditional instead of IF since the former has proper codegen support (IMPALA-7655). Tests: = - Updated the affected tests to include the null counts. - Added unit tests that verify IS [NOT] NULL predicates' cardinality estimation. Perf note: = I reran the compute stats child query with null counts included on the store_sales table from 1000 SF (1TB) tpcds dataset. The table had 22 non-partitioned columns (on which null counts were computed) and ~2.8B rows. This experiment showed around 7-8% perf drop compared to the same child query without null counts for these columns. Change-Id: Ic68f8b4c3756eb1980ce299a602a7d56db1e507a --- M fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java M fe/src/test/java/org/apache/impala/catalog/PartialCatalogInfoTest.java M fe/src/test/java/org/apache/impala/catalog/local/LocalCatalogTest.java M fe/src/test/java/org/apache/impala/planner/CardinalityTest.java M testdata/workloads/functional-planner/queries/PlannerTest/subquery-rewrite.test M testdata/workloads/functional-query/queries/QueryTest/alter-table-set-column-stats.test M testdata/workloads/functional-query/queries/QueryTest/alter-table.test M testdata/workloads/functional-query/queries/QueryTest/compute-stats-decimal.test M testdata/workloads/functional-query/queries/QueryTest/compute-stats-incremental.test M testdata/workloads/functional-query/queries/QueryTest/compute-stats-keywords.test M testdata/workloads/functional-query/queries/QueryTest/compute-stats.test M testdata/workloads/functional-query/queries/QueryTest/hbase-compute-stats-incremental.test M testdata/workloads/functional-query/queries/QueryTest/hbase-compute-stats.test M testdata/workloads/functional-query/queries/QueryTest/show-stats.test 14 files changed, 1,295 insertions(+), 1,304 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/65/11565/9 -- To view, visit http://gerrit.cloudera.org:8080/11565 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ic68f8b4c3756eb1980ce299a602a7d56db1e507a Gerrit-Change-Number: 11565 Gerrit-PatchSet: 9 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Paul Rogers Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vuk Ercegovac
[Impala-ASF-CR] IMPALA-7821: Refactor ExprRewriteRulesTest for easier debugging
Bharath Vissapragada has posted comments on this change. ( http://gerrit.cloudera.org:8080/11890 ) Change subject: IMPALA-7821: Refactor ExprRewriteRulesTest for easier debugging .. Patch Set 4: Oh wait, I just saw the thread dump option and I can see the following. It seems to be waiting for new executors. Wondering if it has something to do with the maintenance email from Lars. Thread #6 at DSL.parallel(Native Method) at WorkflowScript.run(WorkflowScript:8) at DSL.timeout(killer task reported done) at WorkflowScript.run(WorkflowScript:6) at DSL.ansiColor(Native Method) at WorkflowScript.run(WorkflowScript:2) at DSL.timestamps(Native Method) at WorkflowScript.run(WorkflowScript:1) Thread #13 at DSL.build(waiting to schedule ubuntu-16.04-from-scratch; blocked: Waiting for next available executor on ‘ubuntu-16.04’) at WorkflowScript.run(WorkflowScript:18) Thread #14 at DSL.build(waiting to schedule all-build-options-ub1604; blocked: Waiting for next available executor on ‘ubuntu-16.04’) at WorkflowScript.run(WorkflowScript:33) Thread #16 at DSL.build(waiting to schedule clang-tidy-ub1604; blocked: Waiting for next available executor on ‘ubuntu-16.04’) at WorkflowScript.run(WorkflowScript:50) I could also confirm that from https://jenkins.impala.io/job/parallel-all-tests/4671/flowGraphTable/ For ex: https://jenkins.impala.io/job/parallel-all-tests/4671/execution/node/20/log/ -- To view, visit http://gerrit.cloudera.org:8080/11890 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I99a28b170bc0132424041e51f61ebe5c848c4083 Gerrit-Change-Number: 11890 Gerrit-PatchSet: 4 Gerrit-Owner: Paul Rogers Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Paul Rogers Gerrit-Comment-Date: Thu, 06 Dec 2018 20:04:13 + Gerrit-HasComments: No
[Impala-ASF-CR](asf-site) [DOCS] Impala 3.1 Docs to be published
Alex Rodoni has uploaded this change for review. ( http://gerrit.cloudera.org:8080/12046 Change subject: [DOCS] Impala 3.1 Docs to be published .. [DOCS] Impala 3.1 Docs to be published Add Impala docs from branch master, commit hash 5189300b2f9b12f147c8dfd39bd5e30613fcfbfe Change-Id: Icdfa4df8f75090b4783912713e2820eacbdfa904 --- M docs/build/html/commonltr.css M docs/build/html/commonrtl.css M docs/build/html/index.html M docs/build/html/topics/impala_abort_on_error.html M docs/build/html/topics/impala_adls.html M docs/build/html/topics/impala_admin.html M docs/build/html/topics/impala_admission.html M docs/build/html/topics/impala_aggregate_functions.html M docs/build/html/topics/impala_aliases.html A docs/build/html/topics/impala_allow_erasure_coded_files.html M docs/build/html/topics/impala_allow_unsupported_formats.html A docs/build/html/topics/impala_alter_database.html M docs/build/html/topics/impala_alter_table.html M docs/build/html/topics/impala_alter_view.html M docs/build/html/topics/impala_analytic_functions.html M docs/build/html/topics/impala_appx_count_distinct.html M docs/build/html/topics/impala_appx_median.html M docs/build/html/topics/impala_array.html M docs/build/html/topics/impala_auditing.html M docs/build/html/topics/impala_authentication.html M docs/build/html/topics/impala_authorization.html M docs/build/html/topics/impala_avg.html M docs/build/html/topics/impala_avro.html M docs/build/html/topics/impala_batch_size.html M docs/build/html/topics/impala_bigint.html M docs/build/html/topics/impala_bit_functions.html M docs/build/html/topics/impala_boolean.html M docs/build/html/topics/impala_breakpad.html M docs/build/html/topics/impala_buffer_pool_limit.html M docs/build/html/topics/impala_char.html A docs/build/html/topics/impala_comment.html M docs/build/html/topics/impala_comments.html M docs/build/html/topics/impala_complex_types.html M docs/build/html/topics/impala_components.html M docs/build/html/topics/impala_compression_codec.html M docs/build/html/topics/impala_compute_stats.html M docs/build/html/topics/impala_compute_stats_min_sample_size.html M docs/build/html/topics/impala_concepts.html M docs/build/html/topics/impala_conditional_functions.html M docs/build/html/topics/impala_config.html M docs/build/html/topics/impala_config_options.html M docs/build/html/topics/impala_config_performance.html M docs/build/html/topics/impala_connecting.html M docs/build/html/topics/impala_conversion_functions.html M docs/build/html/topics/impala_count.html M docs/build/html/topics/impala_create_database.html M docs/build/html/topics/impala_create_function.html M docs/build/html/topics/impala_create_role.html M docs/build/html/topics/impala_create_table.html M docs/build/html/topics/impala_create_view.html A docs/build/html/topics/impala_custom_timezones.html M docs/build/html/topics/impala_databases.html M docs/build/html/topics/impala_datatypes.html M docs/build/html/topics/impala_datetime_functions.html M docs/build/html/topics/impala_ddl.html M docs/build/html/topics/impala_debug_action.html M docs/build/html/topics/impala_decimal.html M docs/build/html/topics/impala_decimal_v2.html A docs/build/html/topics/impala_dedicated_coordinator.html M docs/build/html/topics/impala_default_join_distribution_mode.html M docs/build/html/topics/impala_default_spillable_buffer_size.html M docs/build/html/topics/impala_delegation.html M docs/build/html/topics/impala_delete.html M docs/build/html/topics/impala_describe.html M docs/build/html/topics/impala_development.html M docs/build/html/topics/impala_disable_codegen.html A docs/build/html/topics/impala_disable_codegen_rows_threshold.html M docs/build/html/topics/impala_disable_row_runtime_filtering.html M docs/build/html/topics/impala_disable_streaming_preaggregations.html M docs/build/html/topics/impala_disable_unsafe_spills.html M docs/build/html/topics/impala_disk_space.html M docs/build/html/topics/impala_distinct.html M docs/build/html/topics/impala_dml.html M docs/build/html/topics/impala_double.html M docs/build/html/topics/impala_drop_database.html M docs/build/html/topics/impala_drop_function.html M docs/build/html/topics/impala_drop_role.html M docs/build/html/topics/impala_drop_stats.html M docs/build/html/topics/impala_drop_table.html M docs/build/html/topics/impala_drop_view.html A docs/build/html/topics/impala_enable_expr_rewrites.html M docs/build/html/topics/impala_exec_single_node_rows_threshold.html M docs/build/html/topics/impala_exec_time_limit_s.html M docs/build/html/topics/impala_explain.html M docs/build/html/topics/impala_explain_level.html M docs/build/html/topics/impala_explain_plan.html M docs/build/html/topics/impala_faq.html M docs/build/html/topics/impala_file_formats.html M docs/build/html/topics/impala_fixed_issues.html M docs/build/html/topics/impala_float.html M docs/build/html/topics/impala_functions.html M
[Impala-ASF-CR] IMPALA-7821: Refactor ExprRewriteRulesTest for easier debugging
Bharath Vissapragada has posted comments on this change. ( http://gerrit.cloudera.org:8080/11890 ) Change subject: IMPALA-7821: Refactor ExprRewriteRulesTest for easier debugging .. Patch Set 4: The build seems to be hung for 11hrs. -- To view, visit http://gerrit.cloudera.org:8080/11890 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I99a28b170bc0132424041e51f61ebe5c848c4083 Gerrit-Change-Number: 11890 Gerrit-PatchSet: 4 Gerrit-Owner: Paul Rogers Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Paul Rogers Gerrit-Comment-Date: Thu, 06 Dec 2018 19:54:12 + Gerrit-HasComments: No
[Impala-ASF-CR](asf-site) [DOCS] Impala 3.1 Docs to be published
Michael Brown has posted comments on this change. ( http://gerrit.cloudera.org:8080/12046 ) Change subject: [DOCS] Impala 3.1 Docs to be published .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/12046/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/12046/1//COMMIT_MSG@9 PS1, Line 9: Add Impala docs from branch master, commit hash : 5189300b2f9b12f147c8dfd39bd5e30613fcfbfe Is this the correct hash to use? $ git fetch apache $ git diff apache/branch-3.1.0 5189300b2f9b12f147c8dfd39bd5e30613fcfbfe -- docs shows ~600 lines of differences. Did one or more doc commits not make it into 3.1.0? -- To view, visit http://gerrit.cloudera.org:8080/12046 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: asf-site Gerrit-MessageType: comment Gerrit-Change-Id: Icdfa4df8f75090b4783912713e2820eacbdfa904 Gerrit-Change-Number: 12046 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Michael Brown Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Thu, 06 Dec 2018 20:41:02 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-4475: part 1, reduce size of TExecQueryFInstancesParams
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/9251 ) Change subject: IMPALA-4475: part 1, reduce size of TExecQueryFInstancesParams .. Patch Set 1: I think it's better if we keep it abandoned until someone adopts it and moves it forward though, it gets confusing with all these open code reviews that aren't meant to be reviewed. -- To view, visit http://gerrit.cloudera.org:8080/9251 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I195c59efc73e5fd4c310ccfc96b480d2209bde09 Gerrit-Change-Number: 9251 Gerrit-PatchSet: 1 Gerrit-Owner: Vuk Ercegovac Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 06 Dec 2018 19:10:18 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7926: Fix flakiness in test reconnect
Thomas Marshall has uploaded this change for review. ( http://gerrit.cloudera.org:8080/12045 Change subject: IMPALA-7926: Fix flakiness in test_reconnect .. IMPALA-7926: Fix flakiness in test_reconnect test_reconnect launches a shell that connects to one impalad in the minicluster then reconnects to a different impalad while checking that the impalad's open session metric changes accordingly. To do this, the test gets the number of open sessions at the start of the test and then expects that the number of sessions will have increased by 1 on the impalad that the shell is currently connected to. This can be a problem if there is a session left over from another test that is still active when test_reconnect starts but exits while it's running. test_reconnect is already marked to run serially, so there shouldn't be any other sessions open while it runs anyways. The solution is to wait at the start of the test until any sessions left over from other tests have exited. Testing: - Ran the test in an environment where the timing was previously causing it to fail almost deterministically and it now passes. Change-Id: I3017ca3bf7b4e33440cffb80e9a48a63bec14434 --- M tests/shell/test_shell_interactive.py 1 file changed, 11 insertions(+), 5 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/45/12045/1 -- To view, visit http://gerrit.cloudera.org:8080/12045 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I3017ca3bf7b4e33440cffb80e9a48a63bec14434 Gerrit-Change-Number: 12045 Gerrit-PatchSet: 1 Gerrit-Owner: Thomas Marshall
[Impala-ASF-CR] IMPALA-5031: Extra large shifts are undefined behavior
Jim Apple has posted comments on this change. ( http://gerrit.cloudera.org:8080/11951 ) Change subject: IMPALA-5031: Extra large shifts are undefined behavior .. Patch Set 2: Code-Review+2 Carry Tim's -- To view, visit http://gerrit.cloudera.org:8080/11951 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie9cec6b2a887e62da594975bc971d38c1d38b0a0 Gerrit-Change-Number: 11951 Gerrit-PatchSet: 2 Gerrit-Owner: Jim Apple Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 06 Dec 2018 19:40:37 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6591: Fix test ssl flaky test
Fredy Wijaya has uploaded this change for review. ( http://gerrit.cloudera.org:8080/12047 Change subject: IMPALA-6591: Fix test_ssl flaky test .. IMPALA-6591: Fix test_ssl flaky test test_ssl has a logic that waits for the number of in-flight queries to be 1. However, the logic for wait_for_num_in_flight_queries(1) only waits for the condition to be true for a period of time and does not throw an exception when the time has elapsed and the condition is not met. In other words, the logic that loops while the number of in-flight queries is 1 never gets executed. I was able to simulate this issue by making Impala shell start much longer. Prior to this patch, in the event that Impala shell took much longer to start, the test started sending the commands to Impala shell even when Impala shell was not ready to receive commands. The patch fixes the issue by waiting for 3 seconds to make sure Impala shell fully starts since starting Impala with SSL enabled can take a bit of time. Although I am not 100% certain on the root cause, the patch seems to fix the issue to unblock the flakiness for now. We should investigate more on this issue, or perhaps update the test to be non-flaky. Testing: - Ran core and exhaustive tests on CentOS 6 Change-Id: I9805269d8b806aecf5d744c219967649a041d49f --- M tests/custom_cluster/test_client_ssl.py 1 file changed, 2 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/47/12047/1 -- To view, visit http://gerrit.cloudera.org:8080/12047 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I9805269d8b806aecf5d744c219967649a041d49f Gerrit-Change-Number: 12047 Gerrit-PatchSet: 1 Gerrit-Owner: Fredy Wijaya
[Impala-ASF-CR] IMPALA-6591: Fix test ssl flaky test
Fredy Wijaya has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/12047 ) Change subject: IMPALA-6591: Fix test_ssl flaky test .. IMPALA-6591: Fix test_ssl flaky test test_ssl has a logic that waits for the number of in-flight queries to be 1. However, the logic for wait_for_num_in_flight_queries(1) only waits for the condition to be true for a period of time and does not throw an exception when the time has elapsed and the condition is not met. In other words, the logic that loops while the number of in-flight queries is 1 never gets executed. I was able to simulate this issue by making Impala shell start much longer. Prior to this patch, in the event that Impala shell took much longer to start, the test started sending the commands to Impala shell even when Impala shell was not ready to receive commands. The patch fixes the issue by waiting for 3 seconds to make sure Impala shell fully starts since starting Impala with SSL enabled can take a bit of time. Although I am not 100% certain on the root cause, the patch seems to fix the issue to unblock the test flakiness for now. We should perhaps investigate more on this issue and update the test to be non-flaky. Testing: - Ran core and exhaustive tests several times on CentOS 6 without any issue Change-Id: I9805269d8b806aecf5d744c219967649a041d49f --- M tests/custom_cluster/test_client_ssl.py 1 file changed, 2 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/47/12047/2 -- To view, visit http://gerrit.cloudera.org:8080/12047 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I9805269d8b806aecf5d744c219967649a041d49f Gerrit-Change-Number: 12047 Gerrit-PatchSet: 2 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Lars Volker
[Impala-ASF-CR] IMPALA-7659: Populate NULL count while computing column stats
Bharath Vissapragada has uploaded a new patch set (#8) to the change originally created by piotr.findei...@gmail.com. ( http://gerrit.cloudera.org:8080/11565 ) Change subject: IMPALA-7659: Populate NULL count while computing column stats .. IMPALA-7659: Populate NULL count while computing column stats It was disabled for performance reasons (IMPALA-1003) and this patch re-enables it since a lot of codegen improvements have happened since then. This patch switches the aggregation to use the CASE conditional instead of IF since the former has proper codegen support (IMPALA-7655). Tests: Updated the affected tests to include the null counts. Change-Id: Ic68f8b4c3756eb1980ce299a602a7d56db1e507a --- M fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java M fe/src/test/java/org/apache/impala/catalog/PartialCatalogInfoTest.java M fe/src/test/java/org/apache/impala/catalog/local/LocalCatalogTest.java M fe/src/test/java/org/apache/impala/planner/CardinalityTest.java M testdata/workloads/functional-planner/queries/PlannerTest/subquery-rewrite.test M testdata/workloads/functional-query/queries/QueryTest/alter-table-set-column-stats.test M testdata/workloads/functional-query/queries/QueryTest/alter-table.test M testdata/workloads/functional-query/queries/QueryTest/compute-stats-decimal.test M testdata/workloads/functional-query/queries/QueryTest/compute-stats-incremental.test M testdata/workloads/functional-query/queries/QueryTest/compute-stats-keywords.test M testdata/workloads/functional-query/queries/QueryTest/compute-stats.test M testdata/workloads/functional-query/queries/QueryTest/hbase-compute-stats-incremental.test M testdata/workloads/functional-query/queries/QueryTest/hbase-compute-stats.test M testdata/workloads/functional-query/queries/QueryTest/show-stats.test 14 files changed, 1,295 insertions(+), 1,304 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/65/11565/8 -- To view, visit http://gerrit.cloudera.org:8080/11565 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ic68f8b4c3756eb1980ce299a602a7d56db1e507a Gerrit-Change-Number: 11565 Gerrit-PatchSet: 8 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Paul Rogers Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vuk Ercegovac
[Impala-ASF-CR] IMPALA-7659: Populate NULL count while computing column stats
Bharath Vissapragada has posted comments on this change. ( http://gerrit.cloudera.org:8080/11565 ) Change subject: IMPALA-7659: Populate NULL count while computing column stats .. Patch Set 9: Tim/Paul: Mind taking another quick look at the updated test and commit message? Thanks. -- To view, visit http://gerrit.cloudera.org:8080/11565 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic68f8b4c3756eb1980ce299a602a7d56db1e507a Gerrit-Change-Number: 11565 Gerrit-PatchSet: 9 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Paul Rogers Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 06 Dec 2018 21:24:35 + Gerrit-HasComments: No
[Impala-ASF-CR] Print query id when timing out waiting for state
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12039 ) Change subject: Print query id when timing out waiting for state .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/12039 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ided03536b8031b8699ba65e1bf81a473b762dac7 Gerrit-Change-Number: 12039 Gerrit-PatchSet: 2 Gerrit-Owner: Lars Volker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 06 Dec 2018 22:50:13 + Gerrit-HasComments: No
[Impala-ASF-CR] Print query id when timing out waiting for state
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12039 ) Change subject: Print query id when timing out waiting for state .. Patch Set 2: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3531/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/12039 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ided03536b8031b8699ba65e1bf81a473b762dac7 Gerrit-Change-Number: 12039 Gerrit-PatchSet: 2 Gerrit-Owner: Lars Volker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 06 Dec 2018 22:50:14 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4555: Make QueryState's status reporting more robust
Thomas Marshall has uploaded this change for review. ( http://gerrit.cloudera.org:8080/12049 Change subject: IMPALA-4555: Make QueryState's status reporting more robust .. IMPALA-4555: Make QueryState's status reporting more robust QueryState periodically collects runtime profiles from all of its fragment instances and sends them to the coordinator. Previously, each time this happens, if the rpc fails, QueryState will retry twice after a configurable timeout and then cancel the fragment instances under the assumption that the coordinator no longer exists. We've found in real clusters that this logic is too sensitive to failed rpcs and can result in fragment instances being cancelled even in cases where the coordinator is still running. This patch makes a few improvements to this logic: - When an intermediate report is generated, if the first attempt to send it to the coordinator fails, the report is discarded. Only the final report, when all of the fragments instances have completed, is retried. - Exponential backoff is used, both for the time between generating intermediate reports (controlled by FLAG_status_report_interval_ms) and for the time between retries for the final report (controlled by FLAG_report_status_retry_interval_ms) such that for a period between retries of 't', on try 'n' the actual timeout will be t * n. Testing: - Added a test which results in a large number of failed intermediate status reports but still succeeds. Change-Id: Ib6007013fc2c9e8eeba11b752ee58fb3038da971 --- M be/src/runtime/coordinator-backend-state.cc M be/src/runtime/fragment-instance-state.cc M be/src/runtime/fragment-instance-state.h M be/src/runtime/query-state.cc M be/src/runtime/query-state.h M be/src/runtime/runtime-state.cc M be/src/runtime/runtime-state.h M tests/custom_cluster/test_rpc_timeout.py 8 files changed, 61 insertions(+), 15 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/49/12049/1 -- To view, visit http://gerrit.cloudera.org:8080/12049 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ib6007013fc2c9e8eeba11b752ee58fb3038da971 Gerrit-Change-Number: 12049 Gerrit-PatchSet: 1 Gerrit-Owner: Thomas Marshall
[Impala-ASF-CR](asf-site) [DOCS] Impala 3.1 Docs to be published
Alex Rodoni has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/12046 ) Change subject: [DOCS] Impala 3.1 Docs to be published .. [DOCS] Impala 3.1 Docs to be published Add Impala docs from branch master, commit hash 5189300b2f9b12f147c8dfd39bd5e30613fcfbfe Change-Id: Icdfa4df8f75090b4783912713e2820eacbdfa904 Reviewed-on: http://gerrit.cloudera.org:8080/12046 Reviewed-by: Michael Brown Tested-by: Alex Rodoni --- M docs/build/html/commonltr.css M docs/build/html/commonrtl.css M docs/build/html/index.html D docs/build/html/topics/impala_abort_on_default_limit_exceeded.html M docs/build/html/topics/impala_abort_on_error.html M docs/build/html/topics/impala_adls.html M docs/build/html/topics/impala_admin.html M docs/build/html/topics/impala_admission.html M docs/build/html/topics/impala_aggregate_functions.html M docs/build/html/topics/impala_aliases.html A docs/build/html/topics/impala_allow_erasure_coded_files.html M docs/build/html/topics/impala_allow_unsupported_formats.html A docs/build/html/topics/impala_alter_database.html M docs/build/html/topics/impala_alter_table.html M docs/build/html/topics/impala_alter_view.html M docs/build/html/topics/impala_analytic_functions.html M docs/build/html/topics/impala_appx_count_distinct.html M docs/build/html/topics/impala_appx_median.html M docs/build/html/topics/impala_array.html M docs/build/html/topics/impala_auditing.html M docs/build/html/topics/impala_authentication.html M docs/build/html/topics/impala_authorization.html M docs/build/html/topics/impala_avg.html M docs/build/html/topics/impala_avro.html M docs/build/html/topics/impala_batch_size.html M docs/build/html/topics/impala_bigint.html M docs/build/html/topics/impala_bit_functions.html M docs/build/html/topics/impala_boolean.html M docs/build/html/topics/impala_breakpad.html M docs/build/html/topics/impala_buffer_pool_limit.html M docs/build/html/topics/impala_char.html A docs/build/html/topics/impala_comment.html M docs/build/html/topics/impala_comments.html M docs/build/html/topics/impala_complex_types.html M docs/build/html/topics/impala_components.html M docs/build/html/topics/impala_compression_codec.html M docs/build/html/topics/impala_compute_stats.html M docs/build/html/topics/impala_compute_stats_min_sample_size.html M docs/build/html/topics/impala_concepts.html M docs/build/html/topics/impala_conditional_functions.html M docs/build/html/topics/impala_config.html M docs/build/html/topics/impala_config_options.html M docs/build/html/topics/impala_config_performance.html M docs/build/html/topics/impala_connecting.html M docs/build/html/topics/impala_conversion_functions.html M docs/build/html/topics/impala_count.html M docs/build/html/topics/impala_create_database.html M docs/build/html/topics/impala_create_function.html M docs/build/html/topics/impala_create_role.html M docs/build/html/topics/impala_create_table.html M docs/build/html/topics/impala_create_view.html A docs/build/html/topics/impala_custom_timezones.html M docs/build/html/topics/impala_databases.html M docs/build/html/topics/impala_datatypes.html M docs/build/html/topics/impala_datetime_functions.html M docs/build/html/topics/impala_ddl.html M docs/build/html/topics/impala_debug_action.html M docs/build/html/topics/impala_decimal.html M docs/build/html/topics/impala_decimal_v2.html A docs/build/html/topics/impala_dedicated_coordinator.html M docs/build/html/topics/impala_default_join_distribution_mode.html D docs/build/html/topics/impala_default_order_by_limit.html M docs/build/html/topics/impala_default_spillable_buffer_size.html M docs/build/html/topics/impala_delegation.html M docs/build/html/topics/impala_delete.html M docs/build/html/topics/impala_describe.html M docs/build/html/topics/impala_development.html M docs/build/html/topics/impala_disable_codegen.html A docs/build/html/topics/impala_disable_codegen_rows_threshold.html M docs/build/html/topics/impala_disable_row_runtime_filtering.html M docs/build/html/topics/impala_disable_streaming_preaggregations.html M docs/build/html/topics/impala_disable_unsafe_spills.html M docs/build/html/topics/impala_disk_space.html M docs/build/html/topics/impala_distinct.html M docs/build/html/topics/impala_dml.html M docs/build/html/topics/impala_double.html M docs/build/html/topics/impala_drop_database.html M docs/build/html/topics/impala_drop_function.html M docs/build/html/topics/impala_drop_role.html M docs/build/html/topics/impala_drop_stats.html M docs/build/html/topics/impala_drop_table.html M docs/build/html/topics/impala_drop_view.html A docs/build/html/topics/impala_enable_expr_rewrites.html M docs/build/html/topics/impala_exec_single_node_rows_threshold.html M docs/build/html/topics/impala_exec_time_limit_s.html M docs/build/html/topics/impala_explain.html M docs/build/html/topics/impala_explain_level.html M docs/build/html/topics/impala_explain_plan.html M
[Impala-ASF-CR] IMPALA-7821: Refactor ExprRewriteRulesTest for easier debugging
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11890 ) Change subject: IMPALA-7821: Refactor ExprRewriteRulesTest for easier debugging .. Patch Set 5: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3533/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/11890 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I99a28b170bc0132424041e51f61ebe5c848c4083 Gerrit-Change-Number: 11890 Gerrit-PatchSet: 5 Gerrit-Owner: Paul Rogers Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Paul Rogers Gerrit-Comment-Date: Thu, 06 Dec 2018 23:35:38 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6591: Fix test ssl flaky test
Lars Volker has posted comments on this change. ( http://gerrit.cloudera.org:8080/12047 ) Change subject: IMPALA-6591: Fix test_ssl flaky test .. Patch Set 2: (3 comments) http://gerrit.cloudera.org:8080/#/c/12047/2//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/12047/2//COMMIT_MSG@13 PS2, Line 13: met. In other words, the logic that loops while the number of in-flight can you point to where that logic is? ImpaladService.wait_for_num_in_flight_queries() looks ok to me. http://gerrit.cloudera.org:8080/#/c/12047/2/tests/custom_cluster/test_client_ssl.py File tests/custom_cluster/test_client_ssl.py: http://gerrit.cloudera.org:8080/#/c/12047/2/tests/custom_cluster/test_client_ssl.py@88 PS2, Line 88: p = ImpalaShell(args="--ssl") Can we change the constructor to return only when the shell is ready to accept commands? http://gerrit.cloudera.org:8080/#/c/12047/2/tests/custom_cluster/test_client_ssl.py@93 PS2, Line 93: impalad.wait_for_num_in_flight_queries(1) This method actually returns a boolean on whether it reached the required number of queries. We should assert that it returns true. -- To view, visit http://gerrit.cloudera.org:8080/12047 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9805269d8b806aecf5d744c219967649a041d49f Gerrit-Change-Number: 12047 Gerrit-PatchSet: 2 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Lars Volker Gerrit-Comment-Date: Thu, 06 Dec 2018 23:40:51 + Gerrit-HasComments: Yes
[Impala-ASF-CR](asf-site) [DOCS] Impala 3.1 Docs to be published
Hello Lars Volker, Michael Brown, Jim Apple, Zoltan Borok-Nagy, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/12046 to look at the new patch set (#2). Change subject: [DOCS] Impala 3.1 Docs to be published .. [DOCS] Impala 3.1 Docs to be published Add Impala docs from branch master, commit hash 5189300b2f9b12f147c8dfd39bd5e30613fcfbfe Change-Id: Icdfa4df8f75090b4783912713e2820eacbdfa904 --- M docs/build/html/commonltr.css M docs/build/html/commonrtl.css M docs/build/html/index.html D docs/build/html/topics/impala_abort_on_default_limit_exceeded.html M docs/build/html/topics/impala_abort_on_error.html M docs/build/html/topics/impala_adls.html M docs/build/html/topics/impala_admin.html M docs/build/html/topics/impala_admission.html M docs/build/html/topics/impala_aggregate_functions.html M docs/build/html/topics/impala_aliases.html A docs/build/html/topics/impala_allow_erasure_coded_files.html M docs/build/html/topics/impala_allow_unsupported_formats.html A docs/build/html/topics/impala_alter_database.html M docs/build/html/topics/impala_alter_table.html M docs/build/html/topics/impala_alter_view.html M docs/build/html/topics/impala_analytic_functions.html M docs/build/html/topics/impala_appx_count_distinct.html M docs/build/html/topics/impala_appx_median.html M docs/build/html/topics/impala_array.html M docs/build/html/topics/impala_auditing.html M docs/build/html/topics/impala_authentication.html M docs/build/html/topics/impala_authorization.html M docs/build/html/topics/impala_avg.html M docs/build/html/topics/impala_avro.html M docs/build/html/topics/impala_batch_size.html M docs/build/html/topics/impala_bigint.html M docs/build/html/topics/impala_bit_functions.html M docs/build/html/topics/impala_boolean.html M docs/build/html/topics/impala_breakpad.html M docs/build/html/topics/impala_buffer_pool_limit.html M docs/build/html/topics/impala_char.html A docs/build/html/topics/impala_comment.html M docs/build/html/topics/impala_comments.html M docs/build/html/topics/impala_complex_types.html M docs/build/html/topics/impala_components.html M docs/build/html/topics/impala_compression_codec.html M docs/build/html/topics/impala_compute_stats.html M docs/build/html/topics/impala_compute_stats_min_sample_size.html M docs/build/html/topics/impala_concepts.html M docs/build/html/topics/impala_conditional_functions.html M docs/build/html/topics/impala_config.html M docs/build/html/topics/impala_config_options.html M docs/build/html/topics/impala_config_performance.html M docs/build/html/topics/impala_connecting.html M docs/build/html/topics/impala_conversion_functions.html M docs/build/html/topics/impala_count.html M docs/build/html/topics/impala_create_database.html M docs/build/html/topics/impala_create_function.html M docs/build/html/topics/impala_create_role.html M docs/build/html/topics/impala_create_table.html M docs/build/html/topics/impala_create_view.html A docs/build/html/topics/impala_custom_timezones.html M docs/build/html/topics/impala_databases.html M docs/build/html/topics/impala_datatypes.html M docs/build/html/topics/impala_datetime_functions.html M docs/build/html/topics/impala_ddl.html M docs/build/html/topics/impala_debug_action.html M docs/build/html/topics/impala_decimal.html M docs/build/html/topics/impala_decimal_v2.html A docs/build/html/topics/impala_dedicated_coordinator.html M docs/build/html/topics/impala_default_join_distribution_mode.html D docs/build/html/topics/impala_default_order_by_limit.html M docs/build/html/topics/impala_default_spillable_buffer_size.html M docs/build/html/topics/impala_delegation.html M docs/build/html/topics/impala_delete.html M docs/build/html/topics/impala_describe.html M docs/build/html/topics/impala_development.html M docs/build/html/topics/impala_disable_codegen.html A docs/build/html/topics/impala_disable_codegen_rows_threshold.html M docs/build/html/topics/impala_disable_row_runtime_filtering.html M docs/build/html/topics/impala_disable_streaming_preaggregations.html M docs/build/html/topics/impala_disable_unsafe_spills.html M docs/build/html/topics/impala_disk_space.html M docs/build/html/topics/impala_distinct.html M docs/build/html/topics/impala_dml.html M docs/build/html/topics/impala_double.html M docs/build/html/topics/impala_drop_database.html M docs/build/html/topics/impala_drop_function.html M docs/build/html/topics/impala_drop_role.html M docs/build/html/topics/impala_drop_stats.html M docs/build/html/topics/impala_drop_table.html M docs/build/html/topics/impala_drop_view.html A docs/build/html/topics/impala_enable_expr_rewrites.html M docs/build/html/topics/impala_exec_single_node_rows_threshold.html M docs/build/html/topics/impala_exec_time_limit_s.html M docs/build/html/topics/impala_explain.html M docs/build/html/topics/impala_explain_level.html M docs/build/html/topics/impala_explain_plan.html M
[Impala-ASF-CR](asf-site) [DOCS] Impala 3.1 Docs to be published
Michael Brown has posted comments on this change. ( http://gerrit.cloudera.org:8080/12046 ) Change subject: [DOCS] Impala 3.1 Docs to be published .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/12046 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: asf-site Gerrit-MessageType: comment Gerrit-Change-Id: Icdfa4df8f75090b4783912713e2820eacbdfa904 Gerrit-Change-Number: 12046 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Michael Brown Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Thu, 06 Dec 2018 22:49:20 + Gerrit-HasComments: No
[Impala-ASF-CR](asf-site) [DOCS] Impala 3.1 Docs to be published
Alex Rodoni has posted comments on this change. ( http://gerrit.cloudera.org:8080/12046 ) Change subject: [DOCS] Impala 3.1 Docs to be published .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/12046 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: asf-site Gerrit-MessageType: comment Gerrit-Change-Id: Icdfa4df8f75090b4783912713e2820eacbdfa904 Gerrit-Change-Number: 12046 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Michael Brown Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Thu, 06 Dec 2018 23:05:20 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7659: Populate NULL count while computing column stats
Paul Rogers has posted comments on this change. ( http://gerrit.cloudera.org:8080/11565 ) Change subject: IMPALA-7659: Populate NULL count while computing column stats .. Patch Set 9: Code-Review+1 LGTM -- To view, visit http://gerrit.cloudera.org:8080/11565 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic68f8b4c3756eb1980ce299a602a7d56db1e507a Gerrit-Change-Number: 11565 Gerrit-PatchSet: 9 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Paul Rogers Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Fri, 07 Dec 2018 00:14:57 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4555: Make QueryState's status reporting more robust
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12049 ) Change subject: IMPALA-4555: Make QueryState's status reporting more robust .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/1555/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12049 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib6007013fc2c9e8eeba11b752ee58fb3038da971 Gerrit-Change-Number: 12049 Gerrit-PatchSet: 1 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Comment-Date: Thu, 06 Dec 2018 23:39:34 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7926: Fix flakiness in test reconnect
Lars Volker has posted comments on this change. ( http://gerrit.cloudera.org:8080/12045 ) Change subject: IMPALA-7926: Fix flakiness in test_reconnect .. Patch Set 1: (3 comments) http://gerrit.cloudera.org:8080/#/c/12045/1/tests/shell/test_shell_interactive.py File tests/shell/test_shell_interactive.py: http://gerrit.cloudera.org:8080/#/c/12045/1/tests/shell/test_shell_interactive.py@235 PS1, Line 235: initial_impala_service.wait_for_metric_value( The default value for wait_for_metric_value() is 10 seconds. Will that be enough? http://gerrit.cloudera.org:8080/#/c/12045/1/tests/shell/test_shell_interactive.py@242 PS1, Line 242: assert get_num_open_sessions(initial_impala_service) == 0 If you change get_num_open_sessions to call wait_for_metric_value(), you can just assert here without waiting above. http://gerrit.cloudera.org:8080/#/c/12045/1/tests/shell/test_shell_interactive.py@253 PS1, Line 253: sleep(5) while you're here remove this and change get_num_open_sessions to call wait_for_metric_value instead of get_metric_value()? -- To view, visit http://gerrit.cloudera.org:8080/12045 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3017ca3bf7b4e33440cffb80e9a48a63bec14434 Gerrit-Change-Number: 12045 Gerrit-PatchSet: 1 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Lars Volker Gerrit-Comment-Date: Thu, 06 Dec 2018 23:48:52 + Gerrit-HasComments: Yes
[Impala-ASF-CR](asf-site) [DOCS] Impala 3.1 Docs to be published
Michael Brown has posted comments on this change. ( http://gerrit.cloudera.org:8080/12046 ) Change subject: [DOCS] Impala 3.1 Docs to be published .. Patch Set 1: I think as part of this patch you also need to "git rm impala_abort_on_default_limit_exceeded.html impala_default_order_by_limit.html impala_max_io_buffers.html impala_reservation_request_timeout.html impala_scan_node_codegen_threshold.html impala_v_cpu_cores.html", because building docs from 5189300b2f9b12f147c8dfd39bd5e30613fcfbfe does not produce these files. Sorry that we don't have tooling that does this or anything to catch it. Maybe before 3.2.0 we can whip something up. -- To view, visit http://gerrit.cloudera.org:8080/12046 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: asf-site Gerrit-MessageType: comment Gerrit-Change-Id: Icdfa4df8f75090b4783912713e2820eacbdfa904 Gerrit-Change-Number: 12046 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Michael Brown Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Thu, 06 Dec 2018 22:14:26 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7738: Implement timeouts for HDFS open calls
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11874 ) Change subject: IMPALA-7738: Implement timeouts for HDFS open calls .. Patch Set 12: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3532/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/11874 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia14403ca5f3f19c6d5f61b9ab2306b0ad3267454 Gerrit-Change-Number: 11874 Gerrit-PatchSet: 12 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 06 Dec 2018 23:03:20 + Gerrit-HasComments: No
[Impala-ASF-CR](asf-site) [DOCS] Impala 3.1 Docs to be published
Alex Rodoni has posted comments on this change. ( http://gerrit.cloudera.org:8080/12046 ) Change subject: [DOCS] Impala 3.1 Docs to be published .. Patch Set 1: > I think as part of this patch you also need to "git rm > impala_abort_on_default_limit_exceeded.html > impala_default_order_by_limit.html > impala_max_io_buffers.html impala_reservation_request_timeout.html > impala_scan_node_codegen_threshold.html impala_v_cpu_cores.html", > because building docs from 5189300b2f9b12f147c8dfd39bd5e30613fcfbfe > does not produce these files. > > Sorry that we don't have tooling that does this or anything to > catch it. Maybe before 3.2.0 we can whip something up. Done. -- To view, visit http://gerrit.cloudera.org:8080/12046 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: asf-site Gerrit-MessageType: comment Gerrit-Change-Id: Icdfa4df8f75090b4783912713e2820eacbdfa904 Gerrit-Change-Number: 12046 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Michael Brown Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Thu, 06 Dec 2018 22:47:14 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7807: Analysis test fixture
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11881 ) Change subject: IMPALA-7807: Analysis test fixture .. Patch Set 9: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3534/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/11881 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id69c99b284960f16394f61072d38dd00269bc10c Gerrit-Change-Number: 11881 Gerrit-PatchSet: 9 Gerrit-Owner: Paul Rogers Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Paul Rogers Gerrit-Comment-Date: Thu, 06 Dec 2018 23:36:18 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5031: Extra large shifts are undefined behavior
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11951 ) Change subject: IMPALA-5031: Extra large shifts are undefined behavior .. Patch Set 2: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3535/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/11951 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie9cec6b2a887e62da594975bc971d38c1d38b0a0 Gerrit-Change-Number: 11951 Gerrit-PatchSet: 2 Gerrit-Owner: Jim Apple Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 07 Dec 2018 00:11:31 + Gerrit-HasComments: No
[Impala-ASF-CR](asf-site) Download 3.1.0
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/12010 ) Change subject: Download 3.1.0 .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/12010 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: asf-site Gerrit-MessageType: comment Gerrit-Change-Id: Id2d0dd660e077858f40a7cf78ad4a479626c1f46 Gerrit-Change-Number: 12010 Gerrit-PatchSet: 2 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Thu, 06 Dec 2018 11:10:03 + Gerrit-HasComments: No
[Impala-ASF-CR](asf-site) Download 3.1.0
Zoltan Borok-Nagy has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/12010 ) Change subject: Download 3.1.0 .. Download 3.1.0 Updated download links for release 3.1.0 Change-Id: Id2d0dd660e077858f40a7cf78ad4a479626c1f46 Reviewed-on: http://gerrit.cloudera.org:8080/12010 Reviewed-by: Jim Apple Tested-by: Zoltan Borok-Nagy --- M downloads.html 1 file changed, 19 insertions(+), 9 deletions(-) Approvals: Jim Apple: Looks good to me, approved Zoltan Borok-Nagy: Verified -- To view, visit http://gerrit.cloudera.org:8080/12010 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: asf-site Gerrit-MessageType: merged Gerrit-Change-Id: Id2d0dd660e077858f40a7cf78ad4a479626c1f46 Gerrit-Change-Number: 12010 Gerrit-PatchSet: 3 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR](asf-site) Added changelog for 3.1.0
Zoltan Borok-Nagy has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/12011 ) Change subject: Added changelog for 3.1.0 .. Added changelog for 3.1.0 Change-Id: I244b68a8b2ef4ac373e857c73326da5425ab66d3 Reviewed-on: http://gerrit.cloudera.org:8080/12011 Reviewed-by: Jim Apple Tested-by: Zoltan Borok-Nagy --- A docs/changelog-3.1.html 1 file changed, 1,138 insertions(+), 0 deletions(-) Approvals: Jim Apple: Looks good to me, approved Zoltan Borok-Nagy: Verified -- To view, visit http://gerrit.cloudera.org:8080/12011 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: asf-site Gerrit-MessageType: merged Gerrit-Change-Id: I244b68a8b2ef4ac373e857c73326da5425ab66d3 Gerrit-Change-Number: 12011 Gerrit-PatchSet: 2 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR](asf-site) Added changelog for 3.1.0
Zoltan Borok-Nagy has removed a vote on this change. Change subject: Added changelog for 3.1.0 .. Removed Verified-1 by Impala Public Jenkins -- To view, visit http://gerrit.cloudera.org:8080/12011 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: asf-site Gerrit-MessageType: deleteVote Gerrit-Change-Id: I244b68a8b2ef4ac373e857c73326da5425ab66d3 Gerrit-Change-Number: 12011 Gerrit-PatchSet: 1 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jim Apple
[Impala-ASF-CR] IMPALA-7853: Add support to read int64 NANO timestamps from Parquet
Gabor Kaszab has posted comments on this change. ( http://gerrit.cloudera.org:8080/11984 ) Change subject: IMPALA-7853: Add support to read int64 NANO timestamps from Parquet .. Patch Set 5: (1 comment) http://gerrit.cloudera.org:8080/#/c/11984/2/be/src/exec/parquet-common.cc File be/src/exec/parquet-common.cc: http://gerrit.cloudera.org:8080/#/c/11984/2/be/src/exec/parquet-common.cc@148 PS2, Line 148: > I have added convert_int96_timestamps as a parameter to GetTimestampInfoFro I see your problem now as this looks suboptimal in my opinion too. I don't want to hold you up with this nitpicking, but what if you saved the incoming convert_int96_timestamps constructor parameter to a member of ParquetTimestampDecoder (defaulting to false)? Then you don't have to pass it as a parameter to GetTimestampInfoFromSchema() and this part of the code would look cleaner. What do you think? -- To view, visit http://gerrit.cloudera.org:8080/11984 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I932396d8646f43c0b9ca4a6359f164c4d8349d8f Gerrit-Change-Number: 11984 Gerrit-PatchSet: 5 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Thu, 06 Dec 2018 11:10:41 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7943: Bump the default client timeout set on impala-shell
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12051 ) Change subject: IMPALA-7943: Bump the default client timeout set on impala-shell .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/12051 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ifc40069e86cbf93634320804efba003fb5551afe Gerrit-Change-Number: 12051 Gerrit-PatchSet: 2 Gerrit-Owner: Anuj Phadke Gerrit-Reviewer: Anuj Phadke Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Michael Ho Gerrit-Comment-Date: Fri, 07 Dec 2018 03:02:04 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7943: Bump the default client timeout set on impala-shell
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12051 ) Change subject: IMPALA-7943: Bump the default client timeout set on impala-shell .. Patch Set 2: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3536/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/12051 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ifc40069e86cbf93634320804efba003fb5551afe Gerrit-Change-Number: 12051 Gerrit-PatchSet: 2 Gerrit-Owner: Anuj Phadke Gerrit-Reviewer: Anuj Phadke Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Michael Ho Gerrit-Comment-Date: Fri, 07 Dec 2018 03:02:05 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7943: Bump the default client timeout set on impala-shell
Lars Volker has posted comments on this change. ( http://gerrit.cloudera.org:8080/12051 ) Change subject: IMPALA-7943: Bump the default client timeout set on impala-shell .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/12051 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ifc40069e86cbf93634320804efba003fb5551afe Gerrit-Change-Number: 12051 Gerrit-PatchSet: 2 Gerrit-Owner: Anuj Phadke Gerrit-Reviewer: Anuj Phadke Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Michael Ho Gerrit-Comment-Date: Fri, 07 Dec 2018 03:04:32 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7821: Refactor ExprRewriteRulesTest for easier debugging
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11890 ) Change subject: IMPALA-7821: Refactor ExprRewriteRulesTest for easier debugging .. Patch Set 5: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/11890 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I99a28b170bc0132424041e51f61ebe5c848c4083 Gerrit-Change-Number: 11890 Gerrit-PatchSet: 5 Gerrit-Owner: Paul Rogers Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Paul Rogers Gerrit-Comment-Date: Fri, 07 Dec 2018 03:27:45 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7807: Analysis test fixture
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11881 ) Change subject: IMPALA-7807: Analysis test fixture .. Patch Set 9: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/3534/ -- To view, visit http://gerrit.cloudera.org:8080/11881 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id69c99b284960f16394f61072d38dd00269bc10c Gerrit-Change-Number: 11881 Gerrit-PatchSet: 9 Gerrit-Owner: Paul Rogers Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Paul Rogers Gerrit-Comment-Date: Fri, 07 Dec 2018 03:40:44 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7795: Implement REFRESH AUTHORIZATION statement
Paul Rogers has posted comments on this change. (
http://gerrit.cloudera.org:8080/11888 )
Change subject: IMPALA-7795: Implement REFRESH AUTHORIZATION statement
..
Patch Set 11: Code-Review+1
(1 comment)
LGTM. The parser/AST changes all look good, especially after moving to the
action enum. Can't speak to the Python tests or actual authorization code.
http://gerrit.cloudera.org:8080/#/c/11888/9/fe/src/main/jflex/sql-scanner.flex
File fe/src/main/jflex/sql-scanner.flex:
http://gerrit.cloudera.org:8080/#/c/11888/9/fe/src/main/jflex/sql-scanner.flex@78
PS9, Line 78: keywordMap.put("authorization",
SqlParserSymbols.KW_AUTHORIZATION);
> I believe authorization is a reserved keyword as specified in L321 of the o
Ah, missed the list of reserved words which are not keywords. Then there is no
regression risk, and the proposed syntax is fine.
--
To view, visit http://gerrit.cloudera.org:8080/11888
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5459e1c97b12dee307e0cf85b94a9f66fd9d9a8c
Gerrit-Change-Number: 11888
Gerrit-PatchSet: 11
Gerrit-Owner: Fredy Wijaya
Gerrit-Reviewer: Bharath Vissapragada
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Fredy Wijaya
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Paul Rogers
Gerrit-Reviewer: Philip Zeyliger
Gerrit-Comment-Date: Fri, 07 Dec 2018 01:12:12 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] Print query id when timing out waiting for state
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12039 ) Change subject: Print query id when timing out waiting for state .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/12039 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ided03536b8031b8699ba65e1bf81a473b762dac7 Gerrit-Change-Number: 12039 Gerrit-PatchSet: 2 Gerrit-Owner: Lars Volker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 07 Dec 2018 02:26:17 + Gerrit-HasComments: No
[Impala-ASF-CR] Print query id when timing out waiting for state
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/12039 ) Change subject: Print query id when timing out waiting for state .. Print query id when timing out waiting for state This makes it easier to find queries that caused failures in the logs. Testing: I triggered this by reducing the timeout in test_restart_services and observing that the query id was printed. Change-Id: Ided03536b8031b8699ba65e1bf81a473b762dac7 Reviewed-on: http://gerrit.cloudera.org:8080/12039 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M tests/common/impala_service.py 1 file changed, 2 insertions(+), 2 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/12039 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ided03536b8031b8699ba65e1bf81a473b762dac7 Gerrit-Change-Number: 12039 Gerrit-PatchSet: 3 Gerrit-Owner: Lars Volker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-1048: show sinks in exec summary
Hello Michael Ho, Lars Volker, Philip Zeyliger, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11967 to look at the new patch set (#14). Change subject: IMPALA-1048: show sinks in exec summary .. IMPALA-1048: show sinks in exec summary The exec summary now includes the total time taken and memory consumed by the data sink at the root of each fragment. Previously the exec summary could hide where time and memory went while executing a query. The high-level changes are: * Generalising logic in the exec summary and runtime profile to handle data sinks, not just plan nodes, including adding richer metadata to runtime profile nodes. * Threading through metadata about the data sinks, like names and estimates, so that it can appear in the exec summary. The major potential downside is that the new timings reported for data stream sender can overlap with the receiver's time and potentially cause confusion. [localhost:21000] default> select count(distinct l_comment) from tpch_parquet.lineitem; summary; Query: select count(distinct l_comment) from tpch_parquet.lineitem Query submitted at: 2018-11-20 16:47:03 (Coordinator: http://tarmstrong-box:25000) Query progress can be monitored at: http://tarmstrong-box:25000/query_plan?query_id=f5464383a3bb6878:54b5252b +---+ | count(distinct l_comment) | +---+ | 4580667 | +---+ Fetched 1 row(s) in 4.53s +-++--+--+---++---+---+---+ | Operator| #Hosts | Avg Time | Max Time | #Rows | Est. #Rows | Peak Mem | Est. Peak Mem | Detail| +-++--+--+---++---+---+---+ | F02:ROOT| 1 | 50.56us | 50.56us | || 0 B | 0 B | | | 06:AGGREGATE| 1 | 275.89us | 275.89us | 1 | 1 | 16.00 KB | 10.00 MB | FINALIZE | | 05:EXCHANGE | 1 | 49.08us | 49.08us | 3 | 1 | 32.00 KB | 16.00 KB | UNPARTITIONED | | F01:EXCHANGE SENDER | 3 | 100.06us | 113.49us | || 16.00 KB | 0 B | | | 02:AGGREGATE| 3 | 19.32ms | 19.57ms | 3 | 1 | 16.00 KB | 10.00 MB | | | 04:AGGREGATE| 3 | 1.29s| 1.43s| 4.58M | 4.65M | 98.02 MB | 62.63 MB | | | 03:EXCHANGE | 3 | 241.64ms | 246.54ms | 5.01M | 4.65M | 9.05 MB | 10.12 MB | HASH(l_comment) | | F00:EXCHANGE SENDER | 3 | 2.43s| 2.58s| || 337.53 KB | 0 B | | | 01:AGGREGATE| 3 | 1.26s| 1.46s| 5.01M | 4.65M | 97.20 MB | 121.17 MB | STREAMING | | 00:SCAN HDFS| 3 | 39.87ms | 41.36ms | 6.00M | 6.00M | 27.87 MB | 80.00 MB | tpch_parquet.lineitem | +-++--+--+---++---+---+---+ Testing: Added a basic observability test. Change-Id: I3fdf7bacae8ff597b255da65af453e174ba53544 --- M be/src/exec/data-sink.cc M be/src/exec/data-sink.h M be/src/exec/exec-node.cc M be/src/exec/exec-node.h M be/src/exec/hbase-table-sink.cc M be/src/exec/hbase-table-sink.h M be/src/exec/hdfs-table-sink.cc M be/src/exec/hdfs-table-sink.h M be/src/exec/kudu-table-sink.cc M be/src/exec/kudu-table-sink.h M be/src/exec/nested-loop-join-builder.cc M be/src/exec/partitioned-hash-join-builder.cc M be/src/exec/plan-root-sink.cc M be/src/exec/plan-root-sink.h M be/src/runtime/coordinator-backend-state.cc M be/src/runtime/coordinator.cc M be/src/runtime/coordinator.h M be/src/runtime/data-stream-test.cc M be/src/runtime/krpc-data-stream-sender.cc M be/src/runtime/krpc-data-stream-sender.h M be/src/util/runtime-profile.cc M be/src/util/runtime-profile.h M be/src/util/summary-util.cc M common/thrift/DataSinks.thrift M common/thrift/ExecStats.thrift M common/thrift/RuntimeProfile.thrift M common/thrift/Types.thrift M fe/src/main/java/org/apache/impala/planner/DataSink.java M fe/src/main/java/org/apache/impala/planner/DataStreamSink.java M fe/src/main/java/org/apache/impala/planner/HBaseTableSink.java M fe/src/main/java/org/apache/impala/planner/HdfsTableSink.java M fe/src/main/java/org/apache/impala/planner/JoinBuildSink.java M fe/src/main/java/org/apache/impala/planner/KuduTableSink.java M fe/src/main/java/org/apache/impala/planner/PlanRootSink.java M shell/impala_client.py M tests/beeswax/impala_beeswax.py M tests/query_test/test_observability.py 37 files changed, 348
[Impala-ASF-CR] IMPALA-1048: show sinks in exec summary
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/11967 ) Change subject: IMPALA-1048: show sinks in exec summary .. Patch Set 14: Code-Review+2 (1 comment) Carry http://gerrit.cloudera.org:8080/#/c/11967/13/be/src/exec/data-sink.h File be/src/exec/data-sink.h: http://gerrit.cloudera.org:8080/#/c/11967/13/be/src/exec/data-sink.h@57 PS13, Line 57: If this is the sink at the root of a fragment, 'sink_id' must be a unique ID fo > duplicate text? Done -- To view, visit http://gerrit.cloudera.org:8080/11967 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3fdf7bacae8ff597b255da65af453e174ba53544 Gerrit-Change-Number: 11967 Gerrit-PatchSet: 14 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 07 Dec 2018 03:37:24 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-1048: show sinks in exec summary
Lars Volker has posted comments on this change. ( http://gerrit.cloudera.org:8080/11967 ) Change subject: IMPALA-1048: show sinks in exec summary .. Patch Set 13: Code-Review+2 (1 comment) Had a nit, otherwise LGTM http://gerrit.cloudera.org:8080/#/c/11967/13/be/src/exec/data-sink.h File be/src/exec/data-sink.h: http://gerrit.cloudera.org:8080/#/c/11967/13/be/src/exec/data-sink.h@57 PS13, Line 57: If this is the sink at the root of a fragment, If this sink is the root sink of duplicate text? -- To view, visit http://gerrit.cloudera.org:8080/11967 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3fdf7bacae8ff597b255da65af453e174ba53544 Gerrit-Change-Number: 11967 Gerrit-PatchSet: 13 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 07 Dec 2018 02:37:45 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7943: Bump the default client timeout set on impala-shell
Anuj Phadke has uploaded this change for review. ( http://gerrit.cloudera.org:8080/12051 Change subject: IMPALA-7943: Bump the default client timeout set on impala-shell .. IMPALA-7943: Bump the default client timeout set on impala-shell As part of IMPALA-7555, we added a default socket timeout of 5 seconds when connecting to an impalad. Under heavy load with kerberos and SSL enabled, we could hit this default timeout. This change bumps up the timeout to 60 secs to make the impala-shell more robust. Change-Id: Ifc40069e86cbf93634320804efba003fb5551afe --- M shell/impala_client.py M shell/impala_shell_config_defaults.py 2 files changed, 2 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/51/12051/1 -- To view, visit http://gerrit.cloudera.org:8080/12051 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ifc40069e86cbf93634320804efba003fb5551afe Gerrit-Change-Number: 12051 Gerrit-PatchSet: 1 Gerrit-Owner: Anuj Phadke
[Impala-ASF-CR] IMPALA-7943: Bump the default client timeout set on impala-shell
Michael Ho has posted comments on this change. ( http://gerrit.cloudera.org:8080/12051 ) Change subject: IMPALA-7943: Bump the default client timeout set on impala-shell .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/12051 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ifc40069e86cbf93634320804efba003fb5551afe Gerrit-Change-Number: 12051 Gerrit-PatchSet: 1 Gerrit-Owner: Anuj Phadke Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Comment-Date: Fri, 07 Dec 2018 02:58:19 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7943: Bump the default client timeout set on impala-shell
Anuj Phadke has posted comments on this change. ( http://gerrit.cloudera.org:8080/12051 ) Change subject: IMPALA-7943: Bump the default client timeout set on impala-shell .. Patch Set 1: 1. opened a listening socket in one terminal #anuj@anuj-OptiPlex-9020:~/Impala$ netcat -l 12345 2. Ran the shell in an another window - anuj@anuj-OptiPlex-9020:~/Impala$ time ./bin/impala-shell.sh --ssl -i localhost:12345 -q "show tables" Starting Impala Shell without Kerberos authentication SSL is enabled. Impala server certificates will NOT be verified (set --ca_cert to change) /home/anuj/Impala/toolchain/thrift-0.9.3-p4/python/lib/python2.7/site-packages/thrift/transport/TSSLSocket.py:80: DeprecationWarning: 3th positional argument is deprecated. Use keyward argument insteand. DeprecationWarning) /home/anuj/Impala/toolchain/thrift-0.9.3-p4/python/lib/python2.7/site-packages/thrift/transport/TSSLSocket.py:80: DeprecationWarning: 4th positional argument is deprecated. Use keyward argument insteand. DeprecationWarning) /home/anuj/Impala/toolchain/thrift-0.9.3-p4/python/lib/python2.7/site-packages/thrift/transport/TSSLSocket.py:80: DeprecationWarning: 5th positional argument is deprecated. Use keyward argument insteand. DeprecationWarning) /home/anuj/Impala/toolchain/thrift-0.9.3-p4/python/lib/python2.7/site-packages/thrift/transport/TSSLSocket.py:216: DeprecationWarning: validate is deprecated. Use cert_reqs=ssl.CERT_NONE instead DeprecationWarning) No handlers could be found for logger "thrift.transport.TSSLSocket" Error connecting: TTransportException, Could not connect to localhost:12345: _ssl.c:495: The handshake operation timed out Not connected to Impala, could not execute queries. real1m0.228s user0m0.132s sys 0m0.040s -- To view, visit http://gerrit.cloudera.org:8080/12051 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ifc40069e86cbf93634320804efba003fb5551afe Gerrit-Change-Number: 12051 Gerrit-PatchSet: 1 Gerrit-Owner: Anuj Phadke Gerrit-Reviewer: Anuj Phadke Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Michael Ho Gerrit-Comment-Date: Fri, 07 Dec 2018 03:01:44 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7821: Refactor ExprRewriteRulesTest for easier debugging
Bharath Vissapragada has posted comments on this change. ( http://gerrit.cloudera.org:8080/11890 ) Change subject: IMPALA-7821: Refactor ExprRewriteRulesTest for easier debugging .. Patch Set 5: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/11890 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I99a28b170bc0132424041e51f61ebe5c848c4083 Gerrit-Change-Number: 11890 Gerrit-PatchSet: 5 Gerrit-Owner: Paul Rogers Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Paul Rogers Gerrit-Comment-Date: Fri, 07 Dec 2018 03:52:58 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7738: Implement timeouts for HDFS open calls
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11874 ) Change subject: IMPALA-7738: Implement timeouts for HDFS open calls .. Patch Set 12: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/11874 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia14403ca5f3f19c6d5f61b9ab2306b0ad3267454 Gerrit-Change-Number: 11874 Gerrit-PatchSet: 12 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 07 Dec 2018 03:12:02 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7943: Bump the default client timeout set on impala-shell
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12051 ) Change subject: IMPALA-7943: Bump the default client timeout set on impala-shell .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/1556/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12051 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ifc40069e86cbf93634320804efba003fb5551afe Gerrit-Change-Number: 12051 Gerrit-PatchSet: 1 Gerrit-Owner: Anuj Phadke Gerrit-Reviewer: Anuj Phadke Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Michael Ho Gerrit-Comment-Date: Fri, 07 Dec 2018 03:29:31 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-1048: show sinks in exec summary
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11967 ) Change subject: IMPALA-1048: show sinks in exec summary .. Patch Set 15: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/11967 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3fdf7bacae8ff597b255da65af453e174ba53544 Gerrit-Change-Number: 11967 Gerrit-PatchSet: 15 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 07 Dec 2018 03:37:45 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7738: Implement timeouts for HDFS open calls
Joe McDonnell has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/11874 ) Change subject: IMPALA-7738: Implement timeouts for HDFS open calls .. IMPALA-7738: Implement timeouts for HDFS open calls This is part 1 of a push to add timeouts for all HDFS operations. It adds timeouts for opening an HDFS file handle. It introduces a new SynchronousThreadPool, which executes an operation in a thread pool and waits up to a specified timeout for the operation to complete. This type of thread pool can accept any subclass of SynchronousWorkItem, and a single thread pool can process different types of work items. It is tested by a new test case in thread-pool-test. This also introduces a new HdfsMonitor which implements timeouts for HDFS operations, currently limited to hdfsOpenFile(). This is implemented using a SynchronousThreadPool. The timeout for hdfs operations is specified by hdfs_operation_timeout_sec, which defaults to 5 minutes. Testing: 1. Added a test to thread-pool-test for the new SynchronousThreadPool. 2. Core tests 3. Added a custom cluster test that does "kill -STOP" for the NameNode and verifies that a subsequent hdfsOpenFile operation times out. Change-Id: Ia14403ca5f3f19c6d5f61b9ab2306b0ad3267454 Reviewed-on: http://gerrit.cloudera.org:8080/11874 Reviewed-by: Joe McDonnell Tested-by: Impala Public Jenkins --- M be/src/common/status.h M be/src/exec/base-sequence-scanner.cc M be/src/runtime/io/CMakeLists.txt M be/src/runtime/io/disk-io-mgr.cc M be/src/runtime/io/disk-io-mgr.h M be/src/runtime/io/handle-cache.h M be/src/runtime/io/handle-cache.inline.h M be/src/runtime/io/hdfs-file-reader.cc M be/src/runtime/io/hdfs-file-reader.h A be/src/runtime/io/hdfs-monitored-ops.cc A be/src/runtime/io/hdfs-monitored-ops.h M be/src/runtime/runtime-state.cc M be/src/util/thread-pool-test.cc M be/src/util/thread-pool.h M common/thrift/generate_error_codes.py A tests/custom_cluster/test_hdfs_timeout.py 16 files changed, 582 insertions(+), 86 deletions(-) Approvals: Joe McDonnell: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/11874 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ia14403ca5f3f19c6d5f61b9ab2306b0ad3267454 Gerrit-Change-Number: 11874 Gerrit-PatchSet: 13 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-5031: Extra large shifts are undefined behavior
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11951 ) Change subject: IMPALA-5031: Extra large shifts are undefined behavior .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/11951 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie9cec6b2a887e62da594975bc971d38c1d38b0a0 Gerrit-Change-Number: 11951 Gerrit-PatchSet: 2 Gerrit-Owner: Jim Apple Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 07 Dec 2018 03:55:10 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5031: Extra large shifts are undefined behavior
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/11951 ) Change subject: IMPALA-5031: Extra large shifts are undefined behavior .. IMPALA-5031: Extra large shifts are undefined behavior According to [expr.shift] in the C++14 standard, an expression in which the shift value is "greater than or equal to the length in bits of the promoted left operand" has undefined behavior. Change-Id: Ie9cec6b2a887e62da594975bc971d38c1d38b0a0 Reviewed-on: http://gerrit.cloudera.org:8080/11951 Reviewed-by: Jim Apple Tested-by: Impala Public Jenkins --- M be/src/util/bit-packing-test.cc 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: Jim Apple: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/11951 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ie9cec6b2a887e62da594975bc971d38c1d38b0a0 Gerrit-Change-Number: 11951 Gerrit-PatchSet: 3 Gerrit-Owner: Jim Apple Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-7903: Fix DCHECK failure in RawValue::PrintValue()
Gabor Kaszab has posted comments on this change. ( http://gerrit.cloudera.org:8080/12021 ) Change subject: IMPALA-7903: Fix DCHECK failure in RawValue::PrintValue() .. Patch Set 2: I did some research on the GVO failure and apparently the new test makes a number of other tests fail. Removing the test and leaving the code change makes the build green again. Investigating the issue. -- To view, visit http://gerrit.cloudera.org:8080/12021 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I22fe7cbbadd0c7eb6d0bd2d50bf500f0528e6af0 Gerrit-Change-Number: 12021 Gerrit-PatchSet: 2 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 06 Dec 2018 13:41:32 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-1048: show sinks in exec summary
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11967 ) Change subject: IMPALA-1048: show sinks in exec summary .. Patch Set 14: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/1557/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11967 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3fdf7bacae8ff597b255da65af453e174ba53544 Gerrit-Change-Number: 11967 Gerrit-PatchSet: 14 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 07 Dec 2018 04:07:18 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7939: Fix issue where CTE is categorized as DML statement
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12052 ) Change subject: IMPALA-7939: Fix issue where CTE is categorized as DML statement .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/1559/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12052 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I011b8e73a0477ac6b2357725452458f972785ae7 Gerrit-Change-Number: 12052 Gerrit-PatchSet: 3 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 07 Dec 2018 05:09:11 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7807: Analysis test fixture
Bharath Vissapragada has posted comments on this change. ( http://gerrit.cloudera.org:8080/11881 ) Change subject: IMPALA-7807: Analysis test fixture .. Patch Set 9: It looks like there are a few genuine test failures here. Paul, mind taking a look at the output here https://jenkins.impala.io/job/gerrit-verify-dryrun/3534/consoleFull -- To view, visit http://gerrit.cloudera.org:8080/11881 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id69c99b284960f16394f61072d38dd00269bc10c Gerrit-Change-Number: 11881 Gerrit-PatchSet: 9 Gerrit-Owner: Paul Rogers Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Paul Rogers Gerrit-Comment-Date: Fri, 07 Dec 2018 06:49:56 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7659: Populate NULL count while computing column stats
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11565 ) Change subject: IMPALA-7659: Populate NULL count while computing column stats .. Patch Set 10: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3538/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/11565 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic68f8b4c3756eb1980ce299a602a7d56db1e507a Gerrit-Change-Number: 11565 Gerrit-PatchSet: 10 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Paul Rogers Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Fri, 07 Dec 2018 06:50:47 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7659: Populate NULL count while computing column stats
Bharath Vissapragada has posted comments on this change. ( http://gerrit.cloudera.org:8080/11565 ) Change subject: IMPALA-7659: Populate NULL count while computing column stats .. Patch Set 9: Code-Review+2 Carrying +2. -- To view, visit http://gerrit.cloudera.org:8080/11565 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic68f8b4c3756eb1980ce299a602a7d56db1e507a Gerrit-Change-Number: 11565 Gerrit-PatchSet: 9 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Paul Rogers Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Fri, 07 Dec 2018 06:50:31 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7943: Bump the default client timeout set on impala-shell
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12051 ) Change subject: IMPALA-7943: Bump the default client timeout set on impala-shell .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/12051 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ifc40069e86cbf93634320804efba003fb5551afe Gerrit-Change-Number: 12051 Gerrit-PatchSet: 2 Gerrit-Owner: Anuj Phadke Gerrit-Reviewer: Anuj Phadke Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Michael Ho Gerrit-Comment-Date: Fri, 07 Dec 2018 06:57:23 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7943: Bump the default client timeout set on impala-shell
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/12051 ) Change subject: IMPALA-7943: Bump the default client timeout set on impala-shell .. IMPALA-7943: Bump the default client timeout set on impala-shell As part of IMPALA-7555, we added a default socket timeout of 5 seconds when connecting to an impalad. Under heavy load with kerberos and SSL enabled, we could hit this default timeout. This change bumps up the timeout to 60 secs to make the impala-shell more robust. Change-Id: Ifc40069e86cbf93634320804efba003fb5551afe Reviewed-on: http://gerrit.cloudera.org:8080/12051 Reviewed-by: Impala Public Jenkins Reviewed-by: Lars Volker Tested-by: Impala Public Jenkins --- M shell/impala_client.py M shell/impala_shell_config_defaults.py 2 files changed, 2 insertions(+), 2 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified Lars Volker: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/12051 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ifc40069e86cbf93634320804efba003fb5551afe Gerrit-Change-Number: 12051 Gerrit-PatchSet: 3 Gerrit-Owner: Anuj Phadke Gerrit-Reviewer: Anuj Phadke Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Michael Ho
[Impala-ASF-CR] IMPALA-7939: Fix issue where CTE is categorized as DML statement
Fredy Wijaya has uploaded this change for review. ( http://gerrit.cloudera.org:8080/12052 Change subject: IMPALA-7939: Fix issue where CTE is categorized as DML statement .. IMPALA-7939: Fix issue where CTE is categorized as DML statement The logic that checks whether a CTE is DML or SELECT uses shlex that splits the statement into tokens and check if any of the tokens matches the DML regular expression. Before this patch, the shlex was set to posix=True, which means the quotes are stripped from the token, e.g. select a from foo where a = 'update' becomes ['select', 'a', 'from', 'foo', 'where', 'a', '=', 'update']. This patch fixes the issue by setting posix=False in shlex that preserves the quotes. ['select', 'a', 'from', 'foo', 'where', 'a', '=', '"update"'] Testing: - Added a new shell test - Ran all shell tests Change-Id: I011b8e73a0477ac6b2357725452458f972785ae7 --- M shell/impala_shell.py M tests/shell/test_shell_interactive.py 2 files changed, 11 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/52/12052/2 -- To view, visit http://gerrit.cloudera.org:8080/12052 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I011b8e73a0477ac6b2357725452458f972785ae7 Gerrit-Change-Number: 12052 Gerrit-PatchSet: 2 Gerrit-Owner: Fredy Wijaya
[Impala-ASF-CR] IMPALA-7659: Populate NULL count while computing column stats
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11565 ) Change subject: IMPALA-7659: Populate NULL count while computing column stats .. Patch Set 10: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/11565 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic68f8b4c3756eb1980ce299a602a7d56db1e507a Gerrit-Change-Number: 11565 Gerrit-PatchSet: 10 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Paul Rogers Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Fri, 07 Dec 2018 06:50:45 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-1048: show sinks in exec summary
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11967 ) Change subject: IMPALA-1048: show sinks in exec summary .. Patch Set 15: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/11967 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3fdf7bacae8ff597b255da65af453e174ba53544 Gerrit-Change-Number: 11967 Gerrit-PatchSet: 15 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 07 Dec 2018 07:33:00 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7939: Fix issue where CTE is categorized as DML statement
Fredy Wijaya has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/12052 ) Change subject: IMPALA-7939: Fix issue where CTE is categorized as DML statement .. IMPALA-7939: Fix issue where CTE is categorized as DML statement The logic that checks whether a CTE is DML or SELECT uses shlex that splits the statement into tokens and check if any of the tokens matches the DML regular expression. Before this patch, the shlex was set to posix=True, which means the quotes are stripped from the token, e.g. select a from foo where a = 'update' becomes ['select', 'a', 'from', 'foo', 'where', 'a', '=', 'update']. As a result, any token that contains "insert", "delete", "upsert", and "update" in will be categorized as DML even though the token is part of string literal value. This patch fixes the issue by setting posix=False in shlex that preserves the quotes. For example: ['select', 'a', 'from', 'foo', 'where', 'a', '=', '"update"'] Testing: - Added a new shell test - Ran all shell tests Change-Id: I011b8e73a0477ac6b2357725452458f972785ae7 --- M shell/impala_shell.py M tests/shell/test_shell_interactive.py 2 files changed, 12 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/52/12052/3 -- To view, visit http://gerrit.cloudera.org:8080/12052 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I011b8e73a0477ac6b2357725452458f972785ae7 Gerrit-Change-Number: 12052 Gerrit-PatchSet: 3 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-7939: Fix issue where CTE is categorized as DML statement
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12052 ) Change subject: IMPALA-7939: Fix issue where CTE is categorized as DML statement .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/1558/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12052 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I011b8e73a0477ac6b2357725452458f972785ae7 Gerrit-Change-Number: 12052 Gerrit-PatchSet: 2 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 07 Dec 2018 04:49:22 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7795: Implement REFRESH AUTHORIZATION statement
Bharath Vissapragada has posted comments on this change. (
http://gerrit.cloudera.org:8080/11888 )
Change subject: IMPALA-7795: Implement REFRESH AUTHORIZATION statement
..
Patch Set 11:
(7 comments)
The code looks fine to me. I still have some concerns around correctness.
http://gerrit.cloudera.org:8080/#/c/11888/11/fe/src/main/java/org/apache/impala/analysis/ResetMetadataStmt.java
File fe/src/main/java/org/apache/impala/analysis/ResetMetadataStmt.java:
http://gerrit.cloudera.org:8080/#/c/11888/11/fe/src/main/java/org/apache/impala/analysis/ResetMetadataStmt.java@138
PS11, Line 138: String dbName = analyzer.getTargetDbName(tableName_);
Preconditions.checkNotNull(tableName_) between L137-138
http://gerrit.cloudera.org:8080/#/c/11888/8/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java
File fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java:
http://gerrit.cloudera.org:8080/#/c/11888/8/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java@1140
PS8, Line 1140: blic void refreshAuthorizat
> Actually we set sentryProxy_ to null in https://github.com/apache/impala/bl
Can we reverse it then. if (sentryProxy == null) return;
http://gerrit.cloudera.org:8080/#/c/11888/8/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
File fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java:
http://gerrit.cloudera.org:8080/#/c/11888/8/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@3451
PS8, Line 3451: catalog_.refreshAuthorization(false);
> We actually increment the catalog version in L3452. I did a quick experimen
Actually thinking a bit more about this, I still am not convinced. I think
there are a few subtle problems here.
1. catalog_.getCatalogVersion() could potentially return a value > what we need
here. Reason being that since you are not locking the versions globally, there
could potentially be concurrent operations that increment versions between
L3451 and L3452.
2. I still am not convinced by the correctness of this. If you see the
coordinator side synchronous wait loop for this, it looks like follows,
Status ImpalaServer::ProcessCatalogUpdateResult(
const TCatalogUpdateResult& catalog_update_result, bool
wait_for_all_subscribers) {
const TUniqueId& catalog_service_id =
catalog_update_result.catalog_service_id;
if (!catalog_update_result.__isset.updated_catalog_objects &&
!catalog_update_result.__isset.removed_catalog_objects) {
// Operation with no result set. Use the version specified in
// 'catalog_update_result' to determine when the effects of this operation
// have been applied to the local catalog cache.
if (catalog_update_result.is_invalidate) {
WaitForMinCatalogUpdate(catalog_update_result.version,
catalog_service_id);
} else {
WaitForCatalogUpdate(catalog_update_result.version, catalog_service_id);
}
In this case since you are setting the version, you'd block on
WaitForCatalogUpdate(catalog_update_result.version, catalog_service_id).
However, since SS does not guarantee broadcast in the order of versions, there
is a chance that the loop in WaitForCatalogUpdate() might break early.
If you see invalidate metadata (which uses a similar logic), you see it calls
WaitForMinCatalogUpdate(). Notice the *Min* part.
3. I'm afraid that if we take the invalidate approach, this becomes
incompatible with Catalog V2 (which does not include an easy way of finding min
versions on the coordinator).
Please let me know if I'm wrong in my understanding.
http://gerrit.cloudera.org:8080/#/c/11888/8/tests/authorization/test_authorization.py
File tests/authorization/test_authorization.py:
http://gerrit.cloudera.org:8080/#/c/11888/8/tests/authorization/test_authorization.py@546
PS8, Line 546:
Can we also do execute_query_expect_failure(select from functional.alltypes)
here.
http://gerrit.cloudera.org:8080/#/c/11888/8/tests/authorization/test_authorization.py@555
PS8, Line 555: finally:
execute_query_expect_success()
http://gerrit.cloudera.org:8080/#/c/11888/8/tests/authorization/test_grant_revoke.py
File tests/authorization/test_grant_revoke.py:
http://gerrit.cloudera.org:8080/#/c/11888/8/tests/authorization/test_grant_revoke.py@211
PS8, Line 211: """Tests that explicit grant/revoke within Impala and external
privilege update should
: produce correct privileges in Impala after a Sentry
refresh."""
: group_name = grp.getgrnam(getuser()).gr_name
: try:
: self.client.execute("create role %s" % unique_role)
: self.client.execute("grant role %s to group `%s`" %
(unique_role, group_name))
: # Add select and insert into the catalog.
: self.client.execute("grant refresh on server to %s" %
unique_role)
:
[Impala-ASF-CR] IMPALA-1048: show sinks in exec summary
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/11967 ) Change subject: IMPALA-1048: show sinks in exec summary .. IMPALA-1048: show sinks in exec summary The exec summary now includes the total time taken and memory consumed by the data sink at the root of each fragment. Previously the exec summary could hide where time and memory went while executing a query. The high-level changes are: * Generalising logic in the exec summary and runtime profile to handle data sinks, not just plan nodes, including adding richer metadata to runtime profile nodes. * Threading through metadata about the data sinks, like names and estimates, so that it can appear in the exec summary. The major potential downside is that the new timings reported for data stream sender can overlap with the receiver's time and potentially cause confusion. [localhost:21000] default> select count(distinct l_comment) from tpch_parquet.lineitem; summary; Query: select count(distinct l_comment) from tpch_parquet.lineitem Query submitted at: 2018-11-20 16:47:03 (Coordinator: http://tarmstrong-box:25000) Query progress can be monitored at: http://tarmstrong-box:25000/query_plan?query_id=f5464383a3bb6878:54b5252b +---+ | count(distinct l_comment) | +---+ | 4580667 | +---+ Fetched 1 row(s) in 4.53s +-++--+--+---++---+---+---+ | Operator| #Hosts | Avg Time | Max Time | #Rows | Est. #Rows | Peak Mem | Est. Peak Mem | Detail| +-++--+--+---++---+---+---+ | F02:ROOT| 1 | 50.56us | 50.56us | || 0 B | 0 B | | | 06:AGGREGATE| 1 | 275.89us | 275.89us | 1 | 1 | 16.00 KB | 10.00 MB | FINALIZE | | 05:EXCHANGE | 1 | 49.08us | 49.08us | 3 | 1 | 32.00 KB | 16.00 KB | UNPARTITIONED | | F01:EXCHANGE SENDER | 3 | 100.06us | 113.49us | || 16.00 KB | 0 B | | | 02:AGGREGATE| 3 | 19.32ms | 19.57ms | 3 | 1 | 16.00 KB | 10.00 MB | | | 04:AGGREGATE| 3 | 1.29s| 1.43s| 4.58M | 4.65M | 98.02 MB | 62.63 MB | | | 03:EXCHANGE | 3 | 241.64ms | 246.54ms | 5.01M | 4.65M | 9.05 MB | 10.12 MB | HASH(l_comment) | | F00:EXCHANGE SENDER | 3 | 2.43s| 2.58s| || 337.53 KB | 0 B | | | 01:AGGREGATE| 3 | 1.26s| 1.46s| 5.01M | 4.65M | 97.20 MB | 121.17 MB | STREAMING | | 00:SCAN HDFS| 3 | 39.87ms | 41.36ms | 6.00M | 6.00M | 27.87 MB | 80.00 MB | tpch_parquet.lineitem | +-++--+--+---++---+---+---+ Testing: Added a basic observability test. Change-Id: I3fdf7bacae8ff597b255da65af453e174ba53544 Reviewed-on: http://gerrit.cloudera.org:8080/11967 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M be/src/exec/data-sink.cc M be/src/exec/data-sink.h M be/src/exec/exec-node.cc M be/src/exec/exec-node.h M be/src/exec/hbase-table-sink.cc M be/src/exec/hbase-table-sink.h M be/src/exec/hdfs-table-sink.cc M be/src/exec/hdfs-table-sink.h M be/src/exec/kudu-table-sink.cc M be/src/exec/kudu-table-sink.h M be/src/exec/nested-loop-join-builder.cc M be/src/exec/partitioned-hash-join-builder.cc M be/src/exec/plan-root-sink.cc M be/src/exec/plan-root-sink.h M be/src/runtime/coordinator-backend-state.cc M be/src/runtime/coordinator.cc M be/src/runtime/coordinator.h M be/src/runtime/data-stream-test.cc M be/src/runtime/krpc-data-stream-sender.cc M be/src/runtime/krpc-data-stream-sender.h M be/src/util/runtime-profile.cc M be/src/util/runtime-profile.h M be/src/util/summary-util.cc M common/thrift/DataSinks.thrift M common/thrift/ExecStats.thrift M common/thrift/RuntimeProfile.thrift M common/thrift/Types.thrift M fe/src/main/java/org/apache/impala/planner/DataSink.java M fe/src/main/java/org/apache/impala/planner/DataStreamSink.java M fe/src/main/java/org/apache/impala/planner/HBaseTableSink.java M fe/src/main/java/org/apache/impala/planner/HdfsTableSink.java M fe/src/main/java/org/apache/impala/planner/JoinBuildSink.java M fe/src/main/java/org/apache/impala/planner/KuduTableSink.java M fe/src/main/java/org/apache/impala/planner/PlanRootSink.java M shell/impala_client.py M tests/beeswax/impala_beeswax.py M tests/query_test/test_observability.py 37
[Impala-ASF-CR] IMPALA-6214: Determine and warn about stuck fragment instances.
Tim Armstrong has abandoned this change. ( http://gerrit.cloudera.org:8080/11021 ) Change subject: IMPALA-6214: Determine and warn about stuck fragment instances. .. Abandoned Appears stale, reopen if you disagree -- To view, visit http://gerrit.cloudera.org:8080/11021 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: abandon Gerrit-Change-Id: I260a1d0a3477e5c6a46094e664500c3e2ed7de62 Gerrit-Change-Number: 11021 Gerrit-PatchSet: 2 Gerrit-Owner: Pranay Singh (320) Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Pranay Singh (320) Gerrit-Reviewer: Todd Lipcon
[Impala-ASF-CR] IMPALA-7219. Avoid wasting ~160 bytes per IncompleteTable in Catalog Server heap.
Tim Armstrong has abandoned this change. ( http://gerrit.cloudera.org:8080/10982 ) Change subject: IMPALA-7219. Avoid wasting ~160 bytes per IncompleteTable in Catalog Server heap. .. Abandoned Appears stale, reopen if you disagree -- To view, visit http://gerrit.cloudera.org:8080/10982 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: abandon Gerrit-Change-Id: If9c75f65ecb3ba3f2c739fa483a84dc052f471c6 Gerrit-Change-Number: 10982 Gerrit-PatchSet: 8 Gerrit-Owner: Misha Dmitriev Gerrit-Reviewer: Misha Dmitriev Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vuk Ercegovac
[Impala-ASF-CR] IMPALA-3825: [WIP]Distributed RuntimeFiltering Work in progress patch for Distributing runtime filtering aggregation.
Tim Armstrong has abandoned this change. ( http://gerrit.cloudera.org:8080/10937 ) Change subject: IMPALA-3825: [WIP]Distributed RuntimeFiltering Work in progress patch for Distributing runtime filtering aggregation. .. Abandoned https://gerrit.cloudera.org/#/c/11055/ -- To view, visit http://gerrit.cloudera.org:8080/10937 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: abandon Gerrit-Change-Id: I0289ee39787d8e9c13a16c3d7d64f471bc554536 Gerrit-Change-Number: 10937 Gerrit-PatchSet: 1 Gerrit-Owner: Rahul Shivu Mahadev Gerrit-Reviewer: Rahul Shivu Mahadev Gerrit-Reviewer: Sailesh Mukil
[Impala-ASF-CR] IMPALA-6662: Make stress test resilient to hangs due to client crashes
Tim Armstrong has abandoned this change. ( http://gerrit.cloudera.org:8080/9635 ) Change subject: IMPALA-6662: Make stress test resilient to hangs due to client crashes .. Abandoned Appears stale, reopen if you disagree -- To view, visit http://gerrit.cloudera.org:8080/9635 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: abandon Gerrit-Change-Id: I10c5dc9b8c2fffc471bac2279e348bc1d1fec3b7 Gerrit-Change-Number: 9635 Gerrit-PatchSet: 6 Gerrit-Owner: Sailesh Mukil Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Michael Brown Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Nithya Janarthanan Gerrit-Reviewer: Sailesh Mukil
[Impala-ASF-CR] bad python27 set usage
Tim Armstrong has abandoned this change. ( http://gerrit.cloudera.org:8080/10996 ) Change subject: bad python27 set usage .. Abandoned Appears stale, reopen if you disagree -- To view, visit http://gerrit.cloudera.org:8080/10996 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: abandon Gerrit-Change-Id: Ia749dca006485d9bc4a22aee692196f4d0b8409b Gerrit-Change-Number: 10996 Gerrit-PatchSet: 3 Gerrit-Owner: Philip Zeyliger Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger
[Impala-ASF-CR] IMPALA-5031: Extra large shifts are undefined behavior
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/11951 ) Change subject: IMPALA-5031: Extra large shifts are undefined behavior .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/11951 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie9cec6b2a887e62da594975bc971d38c1d38b0a0 Gerrit-Change-Number: 11951 Gerrit-PatchSet: 1 Gerrit-Owner: Jim Apple Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 06 Dec 2018 18:11:05 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7853: Add support to read int64 NANO timestamps from Parquet
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/11984 ) Change subject: IMPALA-7853: Add support to read int64 NANO timestamps from Parquet .. Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/11984/2/be/src/exec/parquet-common.cc File be/src/exec/parquet-common.cc: http://gerrit.cloudera.org:8080/#/c/11984/2/be/src/exec/parquet-common.cc@148 PS2, Line 148: if (e.type == parquet::Type::INT96 && convert_int96_timestamps) needs_conversion = true; > I see your problem now as this looks suboptimal in my opinion too. I would vote for returning to the last solution. My problem with saving it in a member is that another member would be needed to store the physical type, because currently it is not stored whether the timestamp is int64 or int96, as this information is always available in the context when ParquetTimestampDecoder is used (often as template parameter, so without runtime cost). -- To view, visit http://gerrit.cloudera.org:8080/11984 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I932396d8646f43c0b9ca4a6359f164c4d8349d8f Gerrit-Change-Number: 11984 Gerrit-PatchSet: 2 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Thu, 06 Dec 2018 14:47:06 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-5200: Count child time for parent's total time
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/11791 ) Change subject: IMPALA-5200: Count child time for parent's total time .. Patch Set 2: Code-Review+1 (1 comment) LGTM aside from one minor comment. Not sure if Phil plans to take a look. http://gerrit.cloudera.org:8080/#/c/11791/2/be/src/util/runtime-profile.cc File be/src/util/runtime-profile.cc: http://gerrit.cloudera.org:8080/#/c/11791/2/be/src/util/runtime-profile.cc@427 PS2, Line 427: total_time_ns_ = max(children_total_time, total_time_counter()->value()); Might be worth mentioning this JIRA in a comment here to justify taking the max() -- To view, visit http://gerrit.cloudera.org:8080/11791 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id6c1191c39fd18b6be45325366a74cf54908c77e Gerrit-Change-Number: 11791 Gerrit-PatchSet: 2 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 06 Dec 2018 18:21:16 + Gerrit-HasComments: Yes
[Impala-ASF-CR] Print query id when timing out waiting for state
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/12039 ) Change subject: Print query id when timing out waiting for state .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/12039 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ided03536b8031b8699ba65e1bf81a473b762dac7 Gerrit-Change-Number: 12039 Gerrit-PatchSet: 1 Gerrit-Owner: Lars Volker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 06 Dec 2018 18:22:34 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7907: Fix ScalarFunction.toSql()
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/12014 ) Change subject: IMPALA-7907: Fix ScalarFunction.toSql() .. Patch Set 5: Code-Review+2 Thanks for the fixes and the new tests. -- To view, visit http://gerrit.cloudera.org:8080/12014 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id34d6df97760a11c299092dff8edbdb7033bce1c Gerrit-Change-Number: 12014 Gerrit-PatchSet: 5 Gerrit-Owner: Paul Rogers Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Paul Rogers Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 06 Dec 2018 18:30:53 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7903: Fix DCHECK failure in RawValue::PrintValue()
Hello Csaba Ringhofer, Tim Armstrong, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/12021 to look at the new patch set (#3). Change subject: IMPALA-7903: Fix DCHECK failure in RawValue::PrintValue() .. IMPALA-7903: Fix DCHECK failure in RawValue::PrintValue() Occasionally Impala crashed on a DCHECK in RawValue::PrintValue() when running web pages tests. Turned out that queries that have an intermediate step for aggregation hit that DCHECK failure when the log level is set to 3. As these web pages test run in parallel it happened sometimes that a selective query was run after test_log_level set the log level to 3 but before it came to the point to reset it. The solution was to enhance RawValue::PrintValue() to handle TYPE_FIXED_UDA_INTERMEDIATE not to run on the default case of the switch. In addition I extended the log writing for this DCHECK to print out the type it can't handle. Testing: - Extended the web server tests to set the log level to 3 and to run an aggregating query in the same test. Change-Id: I22fe7cbbadd0c7eb6d0bd2d50bf500f0528e6af0 --- M be/src/runtime/raw-value.cc M tests/webserver/test_web_pages.py 2 files changed, 20 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/21/12021/3 -- To view, visit http://gerrit.cloudera.org:8080/12021 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I22fe7cbbadd0c7eb6d0bd2d50bf500f0528e6af0 Gerrit-Change-Number: 12021 Gerrit-PatchSet: 3 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-4475: part 1, reduce size of TExecQueryFInstancesParams
Michael Ho has posted comments on this change. ( http://gerrit.cloudera.org:8080/9251 ) Change subject: IMPALA-4475: part 1, reduce size of TExecQueryFInstancesParams .. Patch Set 1: We may consider reviving it as we look into IMPALA-7467. Let's keep it opened for now. -- To view, visit http://gerrit.cloudera.org:8080/9251 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I195c59efc73e5fd4c310ccfc96b480d2209bde09 Gerrit-Change-Number: 9251 Gerrit-PatchSet: 1 Gerrit-Owner: Vuk Ercegovac Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 06 Dec 2018 18:54:17 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4475: part 1, reduce size of TExecQueryFInstancesParams
Michael Ho has restored this change. ( http://gerrit.cloudera.org:8080/9251 ) Change subject: IMPALA-4475: part 1, reduce size of TExecQueryFInstancesParams .. Restored Relevant for IMPALA-7467 as we may still keep a lot of the Plan related structures from Frontend in Thrift. -- To view, visit http://gerrit.cloudera.org:8080/9251 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: restore Gerrit-Change-Id: I195c59efc73e5fd4c310ccfc96b480d2209bde09 Gerrit-Change-Number: 9251 Gerrit-PatchSet: 1 Gerrit-Owner: Vuk Ercegovac Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Vuk Ercegovac
