[Impala-ASF-CR] IMPALA-7031: Cancel action of debug page should not unregister query
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12926 ) Change subject: IMPALA-7031: Cancel action of debug page should not unregister query .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/2657/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12926 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I56983d40e0542bc734ec5a66c339b5131b7b56c8 Gerrit-Change-Number: 12926 Gerrit-PatchSet: 3 Gerrit-Owner: Alice Fan Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 05 Apr 2019 01:58:11 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8360: Fix race conditions in thread-pool-test
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12916 ) Change subject: IMPALA-8360: Fix race conditions in thread-pool-test .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/12916 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id2d5f8b677e475d8e9d6b4512e990b20bfbefaf1 Gerrit-Change-Number: 12916 Gerrit-PatchSet: 2 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 05 Apr 2019 01:52:47 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8360: Fix race conditions in thread-pool-test
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/12916 ) Change subject: IMPALA-8360: Fix race conditions in thread-pool-test .. IMPALA-8360: Fix race conditions in thread-pool-test There are race conditions in thread-pool-test between the caller thread and the worker thread. Specifically, in some cases, the worker thread seems to be slow in freeing resources. This can lead to asserts failing because the work item has not been freed or because the submit of a task failed when it should not have. This fixes the race conditions: - It breaks up the existing SynchronousThreadPoolTest into two smaller tests so that the two can't interfere with each other. - It adds the ability to sleep and recheck that the work item has been freed. Testing: - Ran thread-pool-test in a loop for 20k iterations Change-Id: Id2d5f8b677e475d8e9d6b4512e990b20bfbefaf1 Reviewed-on: http://gerrit.cloudera.org:8080/12916 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M be/src/util/thread-pool-test.cc 1 file changed, 16 insertions(+), 1 deletion(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/12916 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Id2d5f8b677e475d8e9d6b4512e990b20bfbefaf1 Gerrit-Change-Number: 12916 Gerrit-PatchSet: 3 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-7031: Cancel action of debug page should not unregister query
Hello Bikramjeet Vig, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/12926 to look at the new patch set (#3). Change subject: IMPALA-7031: Cancel action of debug page should not unregister query .. IMPALA-7031: Cancel action of debug page should not unregister query When a running query is cancelled from the Cancel button of debug page (impalad WebUI), it will unregister the query. But the client is unaware of the query is unregistered, it will get error/exception on other attempts. The patch updated the ImpalaHttpHandler to make "Cancel" button will only cancel the query instead of unregister it. Testing: - Added a test to test_web_pages.py to test if query is unregistered when cancel from debug page - Run all webserver tests Change-Id: I56983d40e0542bc734ec5a66c339b5131b7b56c8 --- M be/src/service/impala-http-handler.cc M tests/webserver/test_web_pages.py 2 files changed, 18 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/26/12926/3 -- To view, visit http://gerrit.cloudera.org:8080/12926 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I56983d40e0542bc734ec5a66c339b5131b7b56c8 Gerrit-Change-Number: 12926 Gerrit-PatchSet: 3 Gerrit-Owner: Alice Fan Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-7031: Cancel action of debug page should not unregister query
Hello Bikramjeet Vig, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/12926 to look at the new patch set (#4). Change subject: IMPALA-7031: Cancel action of debug page should not unregister query .. IMPALA-7031: Cancel action of debug page should not unregister query When a running query is cancelled from the Cancel button of debug page (impalad WebUI), it will unregister the query. But the client is unaware of the query is unregistered, it will get error/exception on other attempts. The patch updated the ImpalaHttpHandler to make "Cancel" button will only cancel the query instead of unregister it. Testing: - Added a test to test_web_pages.py - Run all webserver tests Change-Id: I56983d40e0542bc734ec5a66c339b5131b7b56c8 --- M be/src/service/impala-http-handler.cc M tests/webserver/test_web_pages.py 2 files changed, 18 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/26/12926/4 -- To view, visit http://gerrit.cloudera.org:8080/12926 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I56983d40e0542bc734ec5a66c339b5131b7b56c8 Gerrit-Change-Number: 12926 Gerrit-PatchSet: 4 Gerrit-Owner: Alice Fan Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-6216: Make PYTHON EGG CACHE configurable
Bharath Vissapragada has posted comments on this change. ( http://gerrit.cloudera.org:8080/12911 ) Change subject: IMPALA-6216: Make PYTHON_EGG_CACHE configurable .. Patch Set 3: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/12911 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I695b2b31d9045eef1a53268f6516858935aed508 Gerrit-Change-Number: 12911 Gerrit-PatchSet: 3 Gerrit-Owner: Yongzhi Chen Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Yongzhi Chen Gerrit-Comment-Date: Fri, 05 Apr 2019 00:43:55 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6216: Make PYTHON EGG CACHE configurable
Bharath Vissapragada has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/12911 ) Change subject: IMPALA-6216: Make PYTHON_EGG_CACHE configurable .. IMPALA-6216: Make PYTHON_EGG_CACHE configurable User can set environment variable PYTHON_EGG_CACHE before call impala-shell. Testing: Run impala-shell with or w/o PYTHON_EGG_CACHE configured Change-Id: I695b2b31d9045eef1a53268f6516858935aed508 Reviewed-on: http://gerrit.cloudera.org:8080/12911 Tested-by: Impala Public Jenkins Reviewed-by: Bharath Vissapragada --- M shell/impala-shell 1 file changed, 5 insertions(+), 3 deletions(-) Approvals: Impala Public Jenkins: Verified Bharath Vissapragada: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/12911 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I695b2b31d9045eef1a53268f6516858935aed508 Gerrit-Change-Number: 12911 Gerrit-PatchSet: 4 Gerrit-Owner: Yongzhi Chen Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Yongzhi Chen
[Impala-ASF-CR] IMPALA-6216: Make PYTHON EGG CACHE configurable
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12911 ) Change subject: IMPALA-6216: Make PYTHON_EGG_CACHE configurable .. Patch Set 3: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/12911 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I695b2b31d9045eef1a53268f6516858935aed508 Gerrit-Change-Number: 12911 Gerrit-PatchSet: 3 Gerrit-Owner: Yongzhi Chen Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Yongzhi Chen Gerrit-Comment-Date: Fri, 05 Apr 2019 00:36:39 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7368: Add initial support for DATE type
Tim Armstrong has posted comments on this change. (
http://gerrit.cloudera.org:8080/12481 )
Change subject: IMPALA-7368: Add initial support for DATE type
..
Patch Set 18:
(2 comments)
Thanks for the changes.
http://gerrit.cloudera.org:8080/#/c/12481/18/be/src/exprs/cast-functions-ir.cc
File be/src/exprs/cast-functions-ir.cc:
http://gerrit.cloudera.org:8080/#/c/12481/18/be/src/exprs/cast-functions-ir.cc@301
PS18, Line 301: "The valid date range in Timestamps is
1400-01-01..-12-31");
Maybe reword a bit:
"The valid date range for the Timestamp type is..."
http://gerrit.cloudera.org:8080/#/c/12481/18/testdata/workloads/functional-query/queries/QueryTest/date.test
File testdata/workloads/functional-query/queries/QueryTest/date.test:
http://gerrit.cloudera.org:8080/#/c/12481/18/testdata/workloads/functional-query/queries/QueryTest/date.test@564
PS18, Line 564: select coalesce('2012-01-01', cast('2012-02-02' as timestamp),
cast('2012-02-02' as timestamp)),
I don't know if it matters but it looks like hive 3 also returns the same for
this line and the next except for an extra digit of padding: 2012-01-01 0:00:00
I.e. there are cases where it matches both that seem to have arbitrarily ended
up here versus the above query.
Again, I don't know if it matters.
--
To view, visit http://gerrit.cloudera.org:8080/12481
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iea8155ef09557e0afa2f8b2d0b2dc9d0896dc30f
Gerrit-Change-Number: 12481
Gerrit-PatchSet: 18
Gerrit-Owner: Attila Jeges
Gerrit-Reviewer: Attila Jeges
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Lars Volker
Gerrit-Reviewer: Tim Armstrong
Gerrit-Reviewer: Todd Lipcon
Gerrit-Comment-Date: Fri, 05 Apr 2019 00:33:45 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8226: Add grant/revoke to/from group for Ranger
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12914 ) Change subject: IMPALA-8226: Add grant/revoke to/from group for Ranger .. Patch Set 11: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/12914 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96 Gerrit-Change-Number: 12914 Gerrit-PatchSet: 11 Gerrit-Owner: Austin Nobis Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: radford nguyen Gerrit-Comment-Date: Fri, 05 Apr 2019 00:04:35 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8226: Add grant/revoke to/from group for Ranger
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/12914 ) Change subject: IMPALA-8226: Add grant/revoke to/from group for Ranger .. IMPALA-8226: Add grant/revoke to/from group for Ranger This patch adds fupport for GRANT privilege statements to GROUP and REVOKE privilege statements from GROUP. The grammar has been updated to support FROM GROUP and TO GROUP for GRANT/REVOKE statements, i.e: GRANT ON TO GROUP REVOKE ON FROM GROUP Currently, only Ranger's authorization implementation supports GROUP based privileges. Sentry will throw an UnsupportedOperationException if it is the enabled authorization provider and this new grammar is used. Testing: - AuthorizationStmtTest was updated to also test for GROUP authorization. - ToSqlTest was updated to test for GROUP changes to the grammar. - A GROUP based E2E test was added to test_ranger.py - ParserTest was updated to test combinations for GrantRevokePrivilege - AnalyzeAuthStmtsTest was updated to test for USER and GROUP identities - Ran all FE tests - Ran authorization E2E tests Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96 Reviewed-on: http://gerrit.cloudera.org:8080/12914 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M common/thrift/CatalogObjects.thrift M fe/src/main/cup/sql-parser.cup M fe/src/main/java/org/apache/impala/authorization/AuthorizationManager.java M fe/src/main/java/org/apache/impala/authorization/NoneAuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerCatalogdAuthorizationManager.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryCatalogdAuthorizationManager.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryImpaladAuthorizationManager.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeAuthStmtsTest.java M fe/src/test/java/org/apache/impala/analysis/AuthorizationStmtTest.java M fe/src/test/java/org/apache/impala/analysis/ParserTest.java M fe/src/test/java/org/apache/impala/analysis/ToSqlTest.java M fe/src/test/resources/ranger-hive-security.xml M testdata/bin/create-load-data.sh A testdata/cluster/ranger/setup/impala_group.json.template M testdata/cluster/ranger/setup/impala_user.json.template M tests/authorization/test_ranger.py 17 files changed, 461 insertions(+), 323 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/12914 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96 Gerrit-Change-Number: 12914 Gerrit-PatchSet: 12 Gerrit-Owner: Austin Nobis Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: radford nguyen
[Impala-ASF-CR] IMPALA-8363: fix e2e start with impala log dir
Tim Armstrong has posted comments on this change. (
http://gerrit.cloudera.org:8080/12935 )
Change subject: IMPALA-8363: fix e2e start with impala_log_dir
..
Patch Set 1:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/12935/1/tests/common/custom_cluster_test_suite.py
File tests/common/custom_cluster_test_suite.py:
http://gerrit.cloudera.org:8080/#/c/12935/1/tests/common/custom_cluster_test_suite.py@147
PS1, Line 147: if IMPALA_LOG_DIR in method.func_dict:
: self._start_impala_cluster(cluster_args,
:
default_query_options=method.func_dict.get(DEFAULT_QUERY_OPTIONS),
: impala_log_dir=method.func_dict[IMPALA_LOG_DIR],
cluster_size=cluster_size,
: num_coordinators=cluster_size,
expected_num_executors=cluster_size)
: else:
: self._start_impala_cluster(cluster_args,
:
default_query_options=method.func_dict.get(DEFAULT_QUERY_OPTIONS),
: cluster_size=cluster_size,
num_coordinators=cluster_size,
: expected_num_executors=cluster_size)
> i think this code may end up getting inconsistent if add more parameter. Ma
Agree with Fredy. Could also do something clever with kwargs, e.g.
kwargs = {}
if IMPALA_LOG_DIR in method.func_dict:
kwargs["impala_log_dir"] = ...
self._start_impala_cluster(cluster_args, default_query_options=...,
**kwargs)
I'm just bikeshedding at this point though, Fredy's suggestion works fine.
--
To view, visit http://gerrit.cloudera.org:8080/12935
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4f46f40474b4b380abe88647a37e8e4d2231d745
Gerrit-Change-Number: 12935
Gerrit-PatchSet: 1
Gerrit-Owner: radford nguyen
Gerrit-Reviewer: Austin Nobis
Gerrit-Reviewer: Fredy Wijaya
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Tim Armstrong
Gerrit-Reviewer: radford nguyen
Gerrit-Comment-Date: Thu, 04 Apr 2019 23:51:07 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8363: fix e2e start with impala log dir
Fredy Wijaya has posted comments on this change. (
http://gerrit.cloudera.org:8080/12935 )
Change subject: IMPALA-8363: fix e2e start with impala_log_dir
..
Patch Set 1:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/12935/1/tests/common/custom_cluster_test_suite.py
File tests/common/custom_cluster_test_suite.py:
http://gerrit.cloudera.org:8080/#/c/12935/1/tests/common/custom_cluster_test_suite.py@147
PS1, Line 147: if IMPALA_LOG_DIR in method.func_dict:
: self._start_impala_cluster(cluster_args,
:
default_query_options=method.func_dict.get(DEFAULT_QUERY_OPTIONS),
: impala_log_dir=method.func_dict[IMPALA_LOG_DIR],
cluster_size=cluster_size,
: num_coordinators=cluster_size,
expected_num_executors=cluster_size)
: else:
: self._start_impala_cluster(cluster_args,
:
default_query_options=method.func_dict.get(DEFAULT_QUERY_OPTIONS),
: cluster_size=cluster_size,
num_coordinators=cluster_size,
: expected_num_executors=cluster_size)
i think this code may end up getting inconsistent if add more parameter. Maybe
we should keep a single call instead.
DEFAULT_IMPALA_LOG_DIR = os.getenv('LOG_DIR', "/tmp/")
impala_log_dir = method.func_dict[IMPALA_LOG_DIR] if IMPALA_LOG_DIR else
DEFAULT_IMPALA_LOG_DIR
self._start_impala_cluster(cluster_args,
default_query_options=method.func_dict.get(DEFAULT_QUERY_OPTIONS),
impala_log_dir=impala_log_dir, cluster_size=cluster_size,
num_coordinators=cluster_size, expected_num_executors=cluster_size)
--
To view, visit http://gerrit.cloudera.org:8080/12935
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4f46f40474b4b380abe88647a37e8e4d2231d745
Gerrit-Change-Number: 12935
Gerrit-PatchSet: 1
Gerrit-Owner: radford nguyen
Gerrit-Reviewer: Austin Nobis
Gerrit-Reviewer: Fredy Wijaya
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Tim Armstrong
Gerrit-Reviewer: radford nguyen
Gerrit-Comment-Date: Thu, 04 Apr 2019 23:33:48 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-4865: Reject Expr Rewrite When Appropriate
Paul Rogers has posted comments on this change. ( http://gerrit.cloudera.org:8080/12814 ) Change subject: IMPALA-4865: Reject Expr Rewrite When Appropriate .. Patch Set 6: (6 comments) http://gerrit.cloudera.org:8080/#/c/12814/6/fe/src/main/java/org/apache/impala/analysis/AnalyticExpr.java File fe/src/main/java/org/apache/impala/analysis/AnalyticExpr.java: http://gerrit.cloudera.org:8080/#/c/12814/6/fe/src/main/java/org/apache/impala/analysis/AnalyticExpr.java@406 PS6, Line 406:analyzer.getQueryCtx(), 0); Not sure we want to do it this way. I'd suggest creating another version of EvalExprWithoutRow that takes two parameters, and have that turn around and call the three-parameter version. http://gerrit.cloudera.org:8080/#/c/12814/6/fe/src/main/java/org/apache/impala/analysis/ColumnDef.java File fe/src/main/java/org/apache/impala/analysis/ColumnDef.java: http://gerrit.cloudera.org:8080/#/c/12814/6/fe/src/main/java/org/apache/impala/analysis/ColumnDef.java@273 PS6, Line 273: FeSupport.MAX_STRING_LEN); Here, rather than decode the meaning as a parameter, just have another create() function, maybe createBounded, that takes a limit. Note that, here, we are creating a literal. In fact, we are creating a date time literals. The date time can never overflow. If we return an expression, rather than a literal, "bad things" will happen. http://gerrit.cloudera.org:8080/#/c/12814/6/fe/src/main/java/org/apache/impala/analysis/LiteralExpr.java File fe/src/main/java/org/apache/impala/analysis/LiteralExpr.java: http://gerrit.cloudera.org:8080/#/c/12814/6/fe/src/main/java/org/apache/impala/analysis/LiteralExpr.java@182 PS6, Line 182: public static LiteralExpr create(Expr constExpr, TQueryCtx queryCtx, int maxResultSize) There are only two cases: limited or unlimited. Best to have two functions: create and createBounded (say) that express those two cases, and encode the limits in those new functions. http://gerrit.cloudera.org:8080/#/c/12814/6/fe/src/main/java/org/apache/impala/analysis/RangePartition.java File fe/src/main/java/org/apache/impala/analysis/RangePartition.java: http://gerrit.cloudera.org:8080/#/c/12814/6/fe/src/main/java/org/apache/impala/analysis/RangePartition.java@186 PS6, Line 186: literal = LiteralExpr.create(e, analyzer.getQueryCtx(), 65_536); Magic constants are generally frowned upon. Make this a declared constant? http://gerrit.cloudera.org:8080/#/c/12814/6/fe/src/main/java/org/apache/impala/authorization/AuthorizationFactory.java File fe/src/main/java/org/apache/impala/authorization/AuthorizationFactory.java: http://gerrit.cloudera.org:8080/#/c/12814/6/fe/src/main/java/org/apache/impala/authorization/AuthorizationFactory.java@23 PS6, Line 23: import org.apache.impala.authorization.*; ? http://gerrit.cloudera.org:8080/#/c/12814/6/fe/src/main/java/org/apache/impala/rewrite/FoldConstantsRule.java File fe/src/main/java/org/apache/impala/rewrite/FoldConstantsRule.java: http://gerrit.cloudera.org:8080/#/c/12814/6/fe/src/main/java/org/apache/impala/rewrite/FoldConstantsRule.java@43 PS6, Line 43: public static final int MAX_STRING_LITERAL_SIZE = 65_536; This guy should be moved to LiteralExpr: it is a property of the rewrite operation, not of this rule. -- To view, visit http://gerrit.cloudera.org:8080/12814 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8b078113ccc1aa49b0cea0c86dff2e02e1dd0e23 Gerrit-Change-Number: 12814 Gerrit-PatchSet: 6 Gerrit-Owner: Fang-Yu Rao Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Paul Rogers Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 04 Apr 2019 23:23:19 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8363: fix e2e start with impala log dir
Fredy Wijaya has posted comments on this change. ( http://gerrit.cloudera.org:8080/12935 ) Change subject: IMPALA-8363: fix e2e start with impala_log_dir .. Patch Set 1: (2 comments) http://gerrit.cloudera.org:8080/#/c/12935/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/12935/1//COMMIT_MSG@7 PS1, Line 7: fix e2e start with impala_log_dir nit: update the title to "Fix E2E start Impala cluster when impala_log_dir parameter is set"? http://gerrit.cloudera.org:8080/#/c/12935/1//COMMIT_MSG@16 PS1, Line 16: tests/authorization/test_ranger.py I think this file has nothing to do with the bug. Maybe mention that you manually updated this file to test the bug and verify the bug fix. -- To view, visit http://gerrit.cloudera.org:8080/12935 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4f46f40474b4b380abe88647a37e8e4d2231d745 Gerrit-Change-Number: 12935 Gerrit-PatchSet: 1 Gerrit-Owner: radford nguyen Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: radford nguyen Gerrit-Comment-Date: Thu, 04 Apr 2019 23:08:46 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8363: fix e2e start with impala log dir
Fredy Wijaya has posted comments on this change. ( http://gerrit.cloudera.org:8080/12935 ) Change subject: IMPALA-8363: fix e2e start with impala_log_dir .. Patch Set 1: Just couple nits. LGTM. -- To view, visit http://gerrit.cloudera.org:8080/12935 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4f46f40474b4b380abe88647a37e8e4d2231d745 Gerrit-Change-Number: 12935 Gerrit-PatchSet: 1 Gerrit-Owner: radford nguyen Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: radford nguyen Gerrit-Comment-Date: Thu, 04 Apr 2019 23:09:03 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8363: fix e2e start with impala log dir
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12935 ) Change subject: IMPALA-8363: fix e2e start with impala_log_dir .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/2656/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12935 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4f46f40474b4b380abe88647a37e8e4d2231d745 Gerrit-Change-Number: 12935 Gerrit-PatchSet: 1 Gerrit-Owner: radford nguyen Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: radford nguyen Gerrit-Comment-Date: Thu, 04 Apr 2019 23:03:34 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4865: Reject Expr Rewrite When Appropriate
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/12814 ) Change subject: IMPALA-4865: Reject Expr Rewrite When Appropriate .. Patch Set 6: Code-Review+1 (1 comment) Look good aside from one issue. I'll let Paul do the final pass over it since there were non-trivial changes in the frontend. http://gerrit.cloudera.org:8080/#/c/12814/6/fe/src/main/java/org/apache/impala/planner/HBaseScanNode.java File fe/src/main/java/org/apache/impala/planner/HBaseScanNode.java: http://gerrit.cloudera.org:8080/#/c/12814/6/fe/src/main/java/org/apache/impala/planner/HBaseScanNode.java@261 PS6, Line 261: analyzer.getQueryCtx(), 65_536); Missed updating this one -- To view, visit http://gerrit.cloudera.org:8080/12814 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8b078113ccc1aa49b0cea0c86dff2e02e1dd0e23 Gerrit-Change-Number: 12814 Gerrit-PatchSet: 6 Gerrit-Owner: Fang-Yu Rao Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Paul Rogers Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 04 Apr 2019 23:01:25 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8377: bump toolchain version to 107-acaeac961d
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12928 ) Change subject: IMPALA-8377: bump toolchain version to 107-acaeac961d .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/12928 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4ac25aa230b9d2559cd4eb6166ab985b18ef7e2a Gerrit-Change-Number: 12928 Gerrit-PatchSet: 2 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Hector Acosta Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 04 Apr 2019 22:40:59 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8377: bump toolchain version to 107-acaeac961d
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/12928 ) Change subject: IMPALA-8377: bump toolchain version to 107-acaeac961d .. IMPALA-8377: bump toolchain version to 107-acaeac961d This fixes an issue with the previous toolchain version where the Kudu client was broken and caused all binaries to crash on startup due to an issue with linked libstdc++ It also fixes an issue where fastbinary.so wasn't being properly included with Thrift. Testing: - Built successfully on redhat6/7, ubuntu16/18, sles12, debian8 - Built and ran a full core test run with both USE_CDH_KUDU=true/false Change-Id: I4ac25aa230b9d2559cd4eb6166ab985b18ef7e2a Reviewed-on: http://gerrit.cloudera.org:8080/12928 Reviewed-by: Thomas Marshall Tested-by: Impala Public Jenkins --- M bin/impala-config.sh 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: Thomas Marshall: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/12928 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I4ac25aa230b9d2559cd4eb6166ab985b18ef7e2a Gerrit-Change-Number: 12928 Gerrit-PatchSet: 3 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Hector Acosta Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-8363: fix e2e start with impala log dir
radford nguyen has uploaded this change for review. ( http://gerrit.cloudera.org:8080/12935 Change subject: IMPALA-8363: fix e2e start with impala_log_dir .. IMPALA-8363: fix e2e start with impala_log_dir This commit fixes the `CustomClusterTestSuite` to wait for the correct number of executors when `impala_log_dir` is specified in the test decorator. Previously, the default value of 3 was always used, regardless of `cluster_size`. Testing: - Ran tests/authorization/test_ranger.py with custom `impala_log_dir` and `cluster_size` arguments and observed pass. Observed to fail before changes - Ran all original e2e tests Change-Id: I4f46f40474b4b380abe88647a37e8e4d2231d745 --- M tests/common/custom_cluster_test_suite.py 1 file changed, 2 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/35/12935/1 -- To view, visit http://gerrit.cloudera.org:8080/12935 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I4f46f40474b4b380abe88647a37e8e4d2231d745 Gerrit-Change-Number: 12935 Gerrit-PatchSet: 1 Gerrit-Owner: radford nguyen
[Impala-ASF-CR] Clean up generation of XML configuration files
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/12930 ) Change subject: Clean up generation of XML configuration files .. Patch Set 3: (1 comment) http://gerrit.cloudera.org:8080/#/c/12930/3/bin/jenkins/critique-gerrit-review.py File bin/jenkins/critique-gerrit-review.py: http://gerrit.cloudera.org:8080/#/c/12930/3/bin/jenkins/critique-gerrit-review.py@72 PS3, Line 72: > flake8: E261 at least two spaces before inline comment I'm tempted to disable this warning. I guess it's part of PEP-8 and therefore standard but I find it very hard to see how it makes anything more readable: https://www.python.org/dev/peps/pep-0008/#inline-comments -- To view, visit http://gerrit.cloudera.org:8080/12930 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ief4434d80baae0fd7be7ffe7b2e07bae1ac45e47 Gerrit-Change-Number: 12930 Gerrit-PatchSet: 3 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Adam Holley Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Thu, 04 Apr 2019 21:40:25 + Gerrit-HasComments: Yes
[Impala-ASF-CR] Clean up generation of XML configuration files
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/12930 ) Change subject: Clean up generation of XML configuration files .. Patch Set 3: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/12930 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ief4434d80baae0fd7be7ffe7b2e07bae1ac45e47 Gerrit-Change-Number: 12930 Gerrit-PatchSet: 3 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Adam Holley Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Thu, 04 Apr 2019 21:38:51 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12927 ) Change subject: IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/2655/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12927 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If46b4bf24d916e4a4ea8a36ff4acfd95d5f45c8e Gerrit-Change-Number: 12927 Gerrit-PatchSet: 4 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Thu, 04 Apr 2019 21:33:08 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8360: Fix race conditions in thread-pool-test
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12916 ) Change subject: IMPALA-8360: Fix race conditions in thread-pool-test .. Patch Set 2: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3986/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/12916 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id2d5f8b677e475d8e9d6b4512e990b20bfbefaf1 Gerrit-Change-Number: 12916 Gerrit-PatchSet: 2 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 04 Apr 2019 21:13:27 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8224: [DOCS] Add the missing contents about Impala web UI pages
Alex Rodoni has posted comments on this change. ( http://gerrit.cloudera.org:8080/12934 ) Change subject: IMPALA-8224: [DOCS] Add the missing contents about Impala web UI pages .. Patch Set 1: The google doc was updated with the latest patch. Please review the new content. Thank you! -- To view, visit http://gerrit.cloudera.org:8080/12934 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic30c26fe1d19c1a81704723cd5ebed667982d5b6 Gerrit-Change-Number: 12934 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Balazs Jeszenszky Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sahil Takiar Gerrit-Comment-Date: Thu, 04 Apr 2019 21:16:40 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8224: [DOCS] Add the missing contents about Impala web UI pages
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12934 ) Change subject: IMPALA-8224: [DOCS] Add the missing contents about Impala web UI pages .. Patch Set 1: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/298/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/12934 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic30c26fe1d19c1a81704723cd5ebed667982d5b6 Gerrit-Change-Number: 12934 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Balazs Jeszenszky Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sahil Takiar Gerrit-Comment-Date: Thu, 04 Apr 2019 21:23:20 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8224: [DOCS] Add the missing contents about Impala web UI pages
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12934 ) Change subject: IMPALA-8224: [DOCS] Add the missing contents about Impala web UI pages .. Patch Set 1: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/298/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/12934 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic30c26fe1d19c1a81704723cd5ebed667982d5b6 Gerrit-Change-Number: 12934 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 04 Apr 2019 21:12:23 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8360: Fix race conditions in thread-pool-test
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12916 ) Change subject: IMPALA-8360: Fix race conditions in thread-pool-test .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/12916 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id2d5f8b677e475d8e9d6b4512e990b20bfbefaf1 Gerrit-Change-Number: 12916 Gerrit-PatchSet: 2 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 04 Apr 2019 21:13:26 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8224: [DOCS] Add the missing contents about Impala web UI pages
Alex Rodoni has uploaded this change for review. ( http://gerrit.cloudera.org:8080/12934 Change subject: IMPALA-8224: [DOCS] Add the missing contents about Impala web UI pages .. IMPALA-8224: [DOCS] Add the missing contents about Impala web UI pages Change-Id: Ic30c26fe1d19c1a81704723cd5ebed667982d5b6 --- M docs/topics/impala_metadata.xml M docs/topics/impala_webui.xml 2 files changed, 790 insertions(+), 283 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/34/12934/1 -- To view, visit http://gerrit.cloudera.org:8080/12934 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ic30c26fe1d19c1a81704723cd5ebed667982d5b6 Gerrit-Change-Number: 12934 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni
[Impala-ASF-CR] IMPALA-7995: part 1: fixes for e2e dockerised impala tests
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12639 ) Change subject: IMPALA-7995: part 1: fixes for e2e dockerised impala tests .. Patch Set 13: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/12639 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iee86cbd2c4631a014af1e8cef8e1cd523a812755 Gerrit-Change-Number: 12639 Gerrit-PatchSet: 13 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 04 Apr 2019 21:09:38 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger
Fredy Wijaya has posted comments on this change. ( http://gerrit.cloudera.org:8080/12927 ) Change subject: IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger .. Patch Set 4: Bharath, this is ready for review now. -- To view, visit http://gerrit.cloudera.org:8080/12927 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If46b4bf24d916e4a4ea8a36ff4acfd95d5f45c8e Gerrit-Change-Number: 12927 Gerrit-PatchSet: 4 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Thu, 04 Apr 2019 20:59:52 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger
Fredy Wijaya has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/12927 ) Change subject: IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger .. IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger This patch updates the Ranger authorization checker code to deny access when column masking and row filtering is enabled in Ranger for queries that that have columns/tables specified in column mask and row filter policies. This is to prevent data leak, such that the data that is masked/filtered in Hive should not be visible at all in Impala until Impala has full support for column masking and row filtering. Testing: - Added tests in AuthorizationStmtTest to test queries with column masking and row filtering enabled. - Ran all FE tests - Ran all E2E tests Change-Id: If46b4bf24d916e4a4ea8a36ff4acfd95d5f45c8e --- M fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java M fe/src/main/java/org/apache/impala/analysis/CreateOrAlterViewStmtBase.java M fe/src/main/java/org/apache/impala/analysis/ModifyStmt.java M fe/src/main/java/org/apache/impala/authorization/AuthorizationChecker.java M fe/src/main/java/org/apache/impala/authorization/AuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/AuthorizationProvider.java R fe/src/main/java/org/apache/impala/authorization/NoopAuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryAuthorizationChecker.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryAuthorizationFactory.java M fe/src/main/java/org/apache/impala/service/JniCatalog.java M fe/src/main/java/org/apache/impala/service/JniFrontend.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeAuthStmtsTest.java M fe/src/test/java/org/apache/impala/analysis/AuthorizationStmtTest.java M fe/src/test/java/org/apache/impala/analysis/StmtMetadataLoaderTest.java M fe/src/test/java/org/apache/impala/catalog/AlterDatabaseTest.java M fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java M fe/src/test/java/org/apache/impala/common/FrontendFixture.java M fe/src/test/java/org/apache/impala/common/FrontendTestBase.java M fe/src/test/java/org/apache/impala/common/QueryFixture.java M fe/src/test/java/org/apache/impala/testutil/ImpaladTestCatalog.java M fe/src/test/java/org/apache/impala/testutil/PlannerTestCaseLoader.java 23 files changed, 486 insertions(+), 81 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/27/12927/4 -- To view, visit http://gerrit.cloudera.org:8080/12927 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: If46b4bf24d916e4a4ea8a36ff4acfd95d5f45c8e Gerrit-Change-Number: 12927 Gerrit-PatchSet: 4 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon
[Impala-ASF-CR] IMPALA-7995: part 1: fixes for e2e dockerised impala tests
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12639 ) Change subject: IMPALA-7995: part 1: fixes for e2e dockerised impala tests .. Patch Set 14: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/2654/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12639 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iee86cbd2c4631a014af1e8cef8e1cd523a812755 Gerrit-Change-Number: 12639 Gerrit-PatchSet: 14 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 04 Apr 2019 20:57:54 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8360: Fix race conditions in thread-pool-test
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/12916 ) Change subject: IMPALA-8360: Fix race conditions in thread-pool-test .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/12916 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id2d5f8b677e475d8e9d6b4512e990b20bfbefaf1 Gerrit-Change-Number: 12916 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 04 Apr 2019 20:52:13 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12927 ) Change subject: IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/2653/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12927 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If46b4bf24d916e4a4ea8a36ff4acfd95d5f45c8e Gerrit-Change-Number: 12927 Gerrit-PatchSet: 3 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Thu, 04 Apr 2019 20:40:00 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7640: RENAME on managed Kudu table should rename Kudu table
Thomas Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/12179 ) Change subject: IMPALA-7640: RENAME on managed Kudu table should rename Kudu table .. Patch Set 4: Code-Review+2 (1 comment) > (1 comment) > > > (1 comment) > > > > Please be sure to get the docs updated to reflect this change, > eg: > > http://impala.apache.org/docs/build/html/topics/impala_kudu.html > > http://impala.apache.org/docs/build/html/topics/impala_tables.html > > Will do, I can just file a doc JIRA for this right? Yep http://gerrit.cloudera.org:8080/#/c/12179/4/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java File fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java: http://gerrit.cloudera.org:8080/#/c/12179/4/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@2378 PS4, Line 2378: oldTbl instanceof KuduTable && !Table.isExternalTable(msTbl) > The whole logic seems a little confusing to me, but from what I can tell al Oh yeah, I see now that altersKuduTable() doesn't even get called on the RENAME_TABLE path as we exit out of alterTable() early after calling alterTableOrViewRename(). This is all a crazy mess, but fortunately it should be getting a lot better very soon when Impala starts taking advantage of the new Kudu/HMS integration, so I'm fine with this the way it is. -- To view, visit http://gerrit.cloudera.org:8080/12179 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I77e7583ce93cba8f6e743c4bedd9900ae1fae081 Gerrit-Change-Number: 12179 Gerrit-PatchSet: 4 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Mike Percy Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Thu, 04 Apr 2019 20:31:25 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7995: part 1: fixes for e2e dockerised impala tests
Hello David Knupp, Joe McDonnell, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/12639 to look at the new patch set (#14). Change subject: IMPALA-7995: part 1: fixes for e2e dockerised impala tests .. IMPALA-7995: part 1: fixes for e2e dockerised impala tests This fixes all core e2e tests running on my local dockerised minicluster build. I do not yet have a CI job or script running but I wanted to get feedback on these changes sooner. The second part of the change will include the CI script and any follow-on fixes required for the exhaustive tests. The following fixes were required: * Detect docker_network from TEST_START_CLUSTER_ARGS * get_webserver_port() does not depend on the caller passing in the default webserver port. It failed previously because it relied on start-impala-cluster.py setting -webserver_port for *all* processes. * Add SkipIf markers for tests that don't make sense or are non-trivial to fix for containerised Impala. * Support loading Impala-lzo plugin from host for tests that depend on it. * Fix some tests that had 'localhost' hardcoded - instead it should be $INTERNAL_LISTEN_HOST, which defaults to localhost. * Fix bug with sorting impala daemons by backend port, which is the same for all dockerised impalads. Testing: I ran tests locally as follows after having set up a docker network and starting other services: ./buildall.sh -noclean -notests -ninja ninja -j $IMPALA_BUILD_THREADS docker_images export TEST_START_CLUSTER_ARGS="--docker_network=impala-cluster" export FE_TEST=false export BE_TEST=false export JDBC_TEST=false export CLUSTER_TEST=false ./bin/run-all-tests.sh Change-Id: Iee86cbd2c4631a014af1e8cef8e1cd523a812755 --- M bin/start-impala-cluster.py M docker/daemon_entrypoint.sh M docker/impala_base/Dockerfile M testdata/workloads/functional-query/queries/QueryTest/kudu_alter.test M testdata/workloads/functional-query/queries/QueryTest/kudu_create.test M tests/catalog_service/test_catalog_service_client.py M tests/common/custom_cluster_test_suite.py M tests/common/environ.py M tests/common/impala_cluster.py M tests/common/impala_test_suite.py M tests/common/skip.py M tests/conftest.py M tests/custom_cluster/test_jvm_mem_tracking.py M tests/custom_cluster/test_krpc_mem_usage.py M tests/custom_cluster/test_rpc_timeout.py M tests/custom_cluster/test_udf_concurrency.py M tests/hs2/test_fetch_first.py M tests/hs2/test_hs2.py M tests/hs2/test_json_endpoints.py M tests/metadata/test_compute_stats.py M tests/metadata/test_ddl.py M tests/observability/test_log_fragments.py M tests/query_test/test_hash_join_timer.py M tests/query_test/test_hdfs_caching.py M tests/query_test/test_insert.py M tests/query_test/test_insert_behaviour.py M tests/query_test/test_kudu.py M tests/query_test/test_lifecycle.py M tests/query_test/test_local_fs.py M tests/query_test/test_mem_usage_scaling.py M tests/query_test/test_queries.py M tests/query_test/test_udfs.py M tests/run-tests.py M tests/statestore/test_statestore.py M tests/stress/test_mini_stress.py M tests/webserver/test_web_pages.py 36 files changed, 183 insertions(+), 80 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/39/12639/14 -- To view, visit http://gerrit.cloudera.org:8080/12639 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Iee86cbd2c4631a014af1e8cef8e1cd523a812755 Gerrit-Change-Number: 12639 Gerrit-PatchSet: 14 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-7368: Add initial support for DATE type
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12481 ) Change subject: IMPALA-7368: Add initial support for DATE type .. Patch Set 18: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/2652/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12481 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iea8155ef09557e0afa2f8b2d0b2dc9d0896dc30f Gerrit-Change-Number: 12481 Gerrit-PatchSet: 18 Gerrit-Owner: Attila Jeges Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Thu, 04 Apr 2019 20:29:21 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8309: add user authorization provider flag
radford nguyen has posted comments on this change. (
http://gerrit.cloudera.org:8080/12901 )
Change subject: IMPALA-8309: add user authorization_provider flag
..
Patch Set 4:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/12901/2/fe/src/main/java/org/apache/impala/authorization/sentry/SentryAuthorizationConfig.java
File
fe/src/main/java/org/apache/impala/authorization/sentry/SentryAuthorizationConfig.java:
http://gerrit.cloudera.org:8080/#/c/12901/2/fe/src/main/java/org/apache/impala/authorization/sentry/SentryAuthorizationConfig.java@130
PS2, Line 130: public String getProviderName() { return "sentry"; }
> Can we make "sentry" a public static final String?
I dunno, in this case the magic string is returned as the implementation of the
abstract `getProviderName` method. IOW, the method itself kind of acts as the
public contract for getting the name, and any other entity which wants to get
that value should be calling the method (as opposed to accessing a public
static constant).
--
To view, visit http://gerrit.cloudera.org:8080/12901
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I001c20505ba4f0562b60fdef73d15308e8500c19
Gerrit-Change-Number: 12901
Gerrit-PatchSet: 4
Gerrit-Owner: radford nguyen
Gerrit-Reviewer: Austin Nobis
Gerrit-Reviewer: Fredy Wijaya
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Todd Lipcon
Gerrit-Reviewer: radford nguyen
Gerrit-Comment-Date: Thu, 04 Apr 2019 20:24:10 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7640: RENAME on managed Kudu table should rename Kudu table
Sahil Takiar has posted comments on this change. ( http://gerrit.cloudera.org:8080/12179 ) Change subject: IMPALA-7640: RENAME on managed Kudu table should rename Kudu table .. Patch Set 4: (1 comment) > (1 comment) > > Please be sure to get the docs updated to reflect this change, eg: > http://impala.apache.org/docs/build/html/topics/impala_kudu.html > http://impala.apache.org/docs/build/html/topics/impala_tables.html Will do, I can just file a doc JIRA for this right? http://gerrit.cloudera.org:8080/#/c/12179/4/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java File fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java: http://gerrit.cloudera.org:8080/#/c/12179/4/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@2378 PS4, Line 2378: oldTbl instanceof KuduTable && !Table.isExternalTable(msTbl) > Is it possible to do this check as part of altersKuduTable() (eg. by adding The whole logic seems a little confusing to me, but from what I can tell altersKuduTable and alterKuduTable is only meant for ADD_COLUMNS, REPLACE_COLUMNS, DROP_COLUMN, ALTER_COLUMN, and ADD_DROP_RANGE_PARTITION. Whereas alterTableOrViewRename is specifically meant to handle RENAME_VIEW and RENAME_TABLE. So I think it would only make sense to move this to altersKuduTable / alterKuduTable if we move this whole method as well. The reason I put this change here is because the alterTableOrViewRename method seems to handle all the logic for RENAME_TABLE and RENAME_VIEW, so I want to keep all the handling for these TAlterTableTypes in one place. It also looks like alterTableOrViewRename is called while the catalog writeLock is acquired, whereas alterKuduTable is not. -- To view, visit http://gerrit.cloudera.org:8080/12179 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I77e7583ce93cba8f6e743c4bedd9900ae1fae081 Gerrit-Change-Number: 12179 Gerrit-PatchSet: 4 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Mike Percy Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Thu, 04 Apr 2019 20:15:49 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7368: Add initial support for DATE type
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12481 ) Change subject: IMPALA-7368: Add initial support for DATE type .. Patch Set 17: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/2651/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12481 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iea8155ef09557e0afa2f8b2d0b2dc9d0896dc30f Gerrit-Change-Number: 12481 Gerrit-PatchSet: 17 Gerrit-Owner: Attila Jeges Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Thu, 04 Apr 2019 20:14:01 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger
Fredy Wijaya has posted comments on this change. (
http://gerrit.cloudera.org:8080/12927 )
Change subject: IMPALA-8363: Deny access when column masking or row filtering
is enabled in Ranger
..
Patch Set 3:
(4 comments)
http://gerrit.cloudera.org:8080/#/c/12927/2/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java
File
fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java:
http://gerrit.cloudera.org:8080/#/c/12927/2/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java@70
PS2, Line 70:
> Wouldn't this represent a column and not a table?
Done
http://gerrit.cloudera.org:8080/#/c/12927/2/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java@246
PS2, Line 246: List exprs = Collections.singletonList(((DeleteStmt)
stmt).getWhereClause());
> So you're iterating over the `Table` which is actually more like a `Column`
For row filter, we don't need to care for columns. My next PS handles the
duplicate entries.
http://gerrit.cloudera.org:8080/#/c/12927/2/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java@269
PS2, Line 269: for (RangerAccessResourceImpl resource: colMaskTables) {
> Is this getting the tables or the columns?
Updated in the next PS.
http://gerrit.cloudera.org:8080/#/c/12927/2/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java@279
PS2, Line 279: /**
> If it is a complex type do you need to parse the complex type for more pote
I think Ranger doesn't support column masking with complex types.
--
To view, visit http://gerrit.cloudera.org:8080/12927
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If46b4bf24d916e4a4ea8a36ff4acfd95d5f45c8e
Gerrit-Change-Number: 12927
Gerrit-PatchSet: 3
Gerrit-Owner: Fredy Wijaya
Gerrit-Reviewer: Austin Nobis
Gerrit-Reviewer: Bharath Vissapragada
Gerrit-Reviewer: Fredy Wijaya
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Todd Lipcon
Gerrit-Comment-Date: Thu, 04 Apr 2019 19:59:02 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] Clean up generation of XML configuration files
Fredy Wijaya has posted comments on this change. ( http://gerrit.cloudera.org:8080/12930 ) Change subject: Clean up generation of XML configuration files .. Patch Set 3: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/12930 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ief4434d80baae0fd7be7ffe7b2e07bae1ac45e47 Gerrit-Change-Number: 12930 Gerrit-PatchSet: 3 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Adam Holley Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Thu, 04 Apr 2019 20:03:31 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger
Fredy Wijaya has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/12927 ) Change subject: IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger .. IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger This patch updates the Ranger authorization checker code to deny access when column masking and row filtering is enabled in Ranger for queries that that have columns/tables specified in column mask and row filter policies. This is to prevent data leak, such that the data that is masked/filtered in Hive should not be visible at all in Impala until Impala has full support for column masking and row filtering. Testing: - Added tests in AuthorizationStmtTest to test queries with column masking and row filtering enabled. - Ran all FE tests - Ran all E2E tests Change-Id: If46b4bf24d916e4a4ea8a36ff4acfd95d5f45c8e --- M fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java M fe/src/main/java/org/apache/impala/analysis/CreateOrAlterViewStmtBase.java M fe/src/main/java/org/apache/impala/analysis/ModifyStmt.java M fe/src/main/java/org/apache/impala/authorization/AuthorizationChecker.java M fe/src/main/java/org/apache/impala/authorization/AuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/AuthorizationProvider.java R fe/src/main/java/org/apache/impala/authorization/NoopAuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryAuthorizationChecker.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryAuthorizationFactory.java M fe/src/main/java/org/apache/impala/service/JniCatalog.java M fe/src/main/java/org/apache/impala/service/JniFrontend.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeAuthStmtsTest.java M fe/src/test/java/org/apache/impala/analysis/AuthorizationStmtTest.java M fe/src/test/java/org/apache/impala/analysis/StmtMetadataLoaderTest.java M fe/src/test/java/org/apache/impala/catalog/AlterDatabaseTest.java M fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java M fe/src/test/java/org/apache/impala/common/FrontendFixture.java M fe/src/test/java/org/apache/impala/common/FrontendTestBase.java M fe/src/test/java/org/apache/impala/common/QueryFixture.java M fe/src/test/java/org/apache/impala/testutil/ImpaladTestCatalog.java M fe/src/test/java/org/apache/impala/testutil/PlannerTestCaseLoader.java 23 files changed, 486 insertions(+), 81 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/27/12927/3 -- To view, visit http://gerrit.cloudera.org:8080/12927 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: If46b4bf24d916e4a4ea8a36ff4acfd95d5f45c8e Gerrit-Change-Number: 12927 Gerrit-PatchSet: 3 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon
[Impala-ASF-CR] IMPALA-7368: Add initial support for DATE type
Attila Jeges has uploaded a new patch set (#18). (
http://gerrit.cloudera.org:8080/12481 )
Change subject: IMPALA-7368: Add initial support for DATE type
..
IMPALA-7368: Add initial support for DATE type
DATE values describe a particular year/month/day in the form
-MM-dd. For example: DATE '2019-02-15'. DATE values do not have a
time of day component. The range of values supported for the DATE type
is -01-01 to -12-31.
This initial DATE type support covers TEXT and HBASE fileformats only.
'DateValue' is used as the internal type to represent DATE values.
The changes are as follows:
- Support for DATE literal syntax.
- Explicit casting between DATE and other types:
- from STRING to DATE. The string value must be formatted as
-MM-dd HH:mm:ss.S. The date component is mandatory,
the time component is optional. If the time component is
present, it will be truncted silently.
- from DATE to STRING. The resulting string value is formatted as
-MM-dd.
- from TIMESTAMP to DATE. The source timestamp's time of day
component is ignored.
- from DATE to TIMESTAMP. The target timestamp's time of day
component is set to 00:00:00.
- Implicit casting between DATE and other types:
- from STRING to DATE if the source string value is used in a
context where a DATE value is expected.
- from DATE to TIMESTAMP if the source date value is used in a
context where a TIMESTAMP value is expected.
- Since STRING -> DATE, STRING -> TIMESTAMP and DATE -> TIMESTAMP
implicit conversions are now all possible, the existing function
overload resolution logic is not adequate anymore.
For example, it resolves the
if(false, '2011-01-01', DATE '1499-02-02') function call to the
if(BOOLEAN, TIMESTAMP, TIMESTAMP) version of the overloaded
function, instead of the if(BOOLEAN, DATE, DATE) version.
This is clearly wrong, so the function overload resolution logic had
to be changed to resolve function calls to the best-fit overloaded
function definition if there are multiple applicable candidates.
An overloaded function definition is an applicable candidate for a
function call if each actual parameter in the function call either
matches the corresponding formal parameter's type (without casting)
or is implicitly castable to that type.
When looking for the best-fit applicable candidate, a parameter
match score (i.e. the number of actual parameters in the function
call that match their corresponding formal parameter's type without
casting) is calculated and the applicable candidate with the highest
parameter match score is chosen.
There's one more issue that the new resolution logic has to address:
if two applicable candidates have the same parameter match score and
the only difference between the two is that the first one requires a
STRING -> TIMESTAMP implicit cast for some of its parameters while
the second one requires a STRING -> DATE implicit cast for the same
parameters then the first candidate has to be chosen not to break
backward compatibility.
E.g: year('2019-02-15') function call must resolve to
year(TIMESTAMP) instead of year(DATE). Note, that year(DATE) is not
implemented yet, so this is not an issue at the moment but it will
be in the future.
When the resolution algorithm considers overloaded function
definitions, first it orders them lexicographically by the types in
their parameter lists. To ensure the backward compatible behavior
Primitivetype.DATE enum value has to come after
PrimitiveType.TIMESTAMP.
- Codegen infrastructure changes for expression evaluation.
- 'IS [NOT] NULL' and '[NOT] IN' predicates.
- Common comparison operators (including the 'BETWEEN' operator).
- Infrastructure changes for built-in functions.
- Some built-in functions: conditional, aggregate, analytical and
math functions.
- C++ UDF/UDA support.
- Support partitioning and grouping by DATE.
- Beeswax, HiveServer2 support.
These items are tightly coupled and it makes sense to implement them
in one change-set.
Testing:
- A new partitioned TEXT table 'functional.date_tbl' (and the
corresponding HBASE table 'functional_hbase.date_tbl') was
introduced for DATE-related tests.
- BE and FE tests were extended to cover DATE type.
- E2E tests:
- since DATE type is supported for TEXT and HBASE fileformats
only, most DATE tests were implemented separately in
tests/query_test/test_date_queries.py.
Note, that this change-set is not a complete DATE type implementation,
but it lays the foundation for future work:
- Add date support to the random query generator.
- Implement a complete set of built-in functions.
- Add Parquet support.
- Add Kudu support.
- Optionally support Avro and ORC.
For further details, see IMPALA-6169.
Change-Id: Iea8155ef09557e0afa2f8b2d0b2dc9d0896dc30f
---
M
[Impala-ASF-CR] IMPALA-8226: Add grant/revoke to/from group for Ranger
radford nguyen has posted comments on this change. (
http://gerrit.cloudera.org:8080/12914 )
Change subject: IMPALA-8226: Add grant/revoke to/from group for Ranger
..
Patch Set 11:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/12914/7/fe/src/main/java/org/apache/impala/authorization/ranger/RangerCatalogdAuthorizationManager.java
File
fe/src/main/java/org/apache/impala/authorization/ranger/RangerCatalogdAuthorizationManager.java:
http://gerrit.cloudera.org:8080/#/c/12914/7/fe/src/main/java/org/apache/impala/authorization/ranger/RangerCatalogdAuthorizationManager.java@193
PS7, Line 193: String user, List groups, String clusterName,
List privileges) {
nit: We could probably use a `Collection groups` to be more general
here, since the group's items are copied into a `List` when creating the
request. Same with `privileges`.
http://gerrit.cloudera.org:8080/#/c/12914/7/fe/src/main/java/org/apache/impala/authorization/ranger/RangerCatalogdAuthorizationManager.java@233
PS7, Line 233: if (!groups.isEmpty()) request.getGroups().addAll(groups);
nit: is the `if` statement really necessary given the contract of `addAll`?
http://gerrit.cloudera.org:8080/#/c/12914/10/fe/src/main/java/org/apache/impala/authorization/sentry/SentryImpaladAuthorizationManager.java
File
fe/src/main/java/org/apache/impala/authorization/sentry/SentryImpaladAuthorizationManager.java:
http://gerrit.cloudera.org:8080/#/c/12914/10/fe/src/main/java/org/apache/impala/authorization/sentry/SentryImpaladAuthorizationManager.java@170
PS10, Line 170: "%s is not supported in Impalad",
ClassUtil.getMethodName()));
Isn't it more accurate to say that this isn't supported with sentry?
--
To view, visit http://gerrit.cloudera.org:8080/12914
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96
Gerrit-Change-Number: 12914
Gerrit-PatchSet: 11
Gerrit-Owner: Austin Nobis
Gerrit-Reviewer: Austin Nobis
Gerrit-Reviewer: Fredy Wijaya
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: radford nguyen
Gerrit-Comment-Date: Thu, 04 Apr 2019 19:57:09 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] Clean up generation of XML configuration files
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12930 ) Change subject: Clean up generation of XML configuration files .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/2650/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12930 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ief4434d80baae0fd7be7ffe7b2e07bae1ac45e47 Gerrit-Change-Number: 12930 Gerrit-PatchSet: 3 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Adam Holley Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Thu, 04 Apr 2019 19:56:50 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6216: Make PYTHON EGG CACHE configurable
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12911 ) Change subject: IMPALA-6216: Make PYTHON_EGG_CACHE configurable .. Patch Set 3: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3985/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/12911 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I695b2b31d9045eef1a53268f6516858935aed508 Gerrit-Change-Number: 12911 Gerrit-PatchSet: 3 Gerrit-Owner: Yongzhi Chen Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Yongzhi Chen Gerrit-Comment-Date: Thu, 04 Apr 2019 19:56:06 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7368: Add initial support for DATE type
Impala Public Jenkins has posted comments on this change. (
http://gerrit.cloudera.org:8080/12481 )
Change subject: IMPALA-7368: Add initial support for DATE type
..
Patch Set 17:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/12481/17/be/src/runtime/date-test.cc
File be/src/runtime/date-test.cc:
http://gerrit.cloudera.org:8080/#/c/12481/17/be/src/runtime/date-test.cc@61
PS17, Line 61: const DateValue v4 = ParseValidateDate("1990-10-20
23:59:59.9", true, 1990, 10, 20);
line too long (94 > 90)
--
To view, visit http://gerrit.cloudera.org:8080/12481
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iea8155ef09557e0afa2f8b2d0b2dc9d0896dc30f
Gerrit-Change-Number: 12481
Gerrit-PatchSet: 17
Gerrit-Owner: Attila Jeges
Gerrit-Reviewer: Attila Jeges
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Lars Volker
Gerrit-Reviewer: Tim Armstrong
Gerrit-Reviewer: Todd Lipcon
Gerrit-Comment-Date: Thu, 04 Apr 2019 19:49:22 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7368: Add initial support for DATE type
Attila Jeges has posted comments on this change. (
http://gerrit.cloudera.org:8080/12481 )
Change subject: IMPALA-7368: Add initial support for DATE type
..
Patch Set 17:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/12481/11//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/12481/11//COMMIT_MSG@20
PS11, Line 20: - Explicit casting between DATE and other types:
: - from STRING
> This seems to be stricter than CAST() in postgres, mysql, and hive (see htt
Fixed it to allow and silently truncate the time component.
http://gerrit.cloudera.org:8080/#/c/12481/11//COMMIT_MSG@38
PS11, Line 38: > DATE, STRING -> TIMESTAMP and DATE -> TIMESTAMP
: implicit conversions are now all possible, the existing
function
: overload resolution logic is not adequate anymore.
: For example, it resolves the
> Should we be emitting a WARNING for any out-of-range values? it seems surpr
I've added warnings to indicate that DATE->TIMESTAMP, STRING->DATE,
TIMESTAMP->DATE conversions failed.
Adding warning to STRING->TIMESTAMP conversions needs more work. Since these
conversions are not related to the DATE type I'd prefer to fix them separately.
http://gerrit.cloudera.org:8080/#/c/12481/11/fe/src/main/java/org/apache/impala/catalog/Function.java
File fe/src/main/java/org/apache/impala/catalog/Function.java:
http://gerrit.cloudera.org:8080/#/c/12481/11/fe/src/main/java/org/apache/impala/catalog/Function.java@191
PS11, Line 191: // Compares this to 'other' for 'mode'.
> I'm afraid about introducing new semantics here which we'll need to break a
Thanks for putting this together!
- Impala already works as Hive 3.1 in rows 8-11, 13-14, 17, 20 and 22.
- I've changed STRING->DATE conversions to accept (and silently truncate) the
time component. This fixes rows 1-7, 12 and 21.
- Row 15, 16 and 19 happens because Impala and PostgreSQL convert STRING, DATE,
TIMESTAMP parameters to TIMESTAMP, while Hive converts them to STRING. Fixing
these might not be possible w/o breaking backward compatibility.
- Row 18 returns an error in Hive 2.1, and probably fails in Hive 3.1 as well
(haven't tried it though). Impala and PostgreSQL convert DATE and TIMESTAMP
parameters to TIMESTAMP.
- Impala's behavior in row 23 is expected as valid timestamps in Impala start
with year 1400. Adding YEAR(DATE) won't fix this problem either as we will
still have to resolve YEAR('0009-02-15') to YEAR(TIMESTAMP) instead of
YEAR(DATE) not to break backward compatibility. Users will have to be more
implicit to avoid this issue and call YEAR(DATE '0009-02-15'). The best we can
do is to issue a warning (which I haven't done here as it requires quite bit of
work and technically it is a TIMESTAMP issue not a DATE issue. I'd prefer to do
it in a separate patch.)
--
To view, visit http://gerrit.cloudera.org:8080/12481
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iea8155ef09557e0afa2f8b2d0b2dc9d0896dc30f
Gerrit-Change-Number: 12481
Gerrit-PatchSet: 17
Gerrit-Owner: Attila Jeges
Gerrit-Reviewer: Attila Jeges
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Lars Volker
Gerrit-Reviewer: Tim Armstrong
Gerrit-Reviewer: Todd Lipcon
Gerrit-Comment-Date: Thu, 04 Apr 2019 19:48:52 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7368: Add initial support for DATE type
Attila Jeges has uploaded a new patch set (#17). (
http://gerrit.cloudera.org:8080/12481 )
Change subject: IMPALA-7368: Add initial support for DATE type
..
IMPALA-7368: Add initial support for DATE type
DATE values describe a particular year/month/day in the form
-MM-dd. For example: DATE '2019-02-15'. DATE values do not have a
time of day component. The range of values supported for the DATE type
is -01-01 to -12-31.
This initial DATE type support covers TEXT and HBASE fileformats only.
'DateValue' is used as the internal type to represent DATE values.
The changes are as follows:
- Support for DATE literal syntax.
- Explicit casting between DATE and other types:
- from STRING to DATE. The string value must be formatted as
-MM-dd HH:mm:ss.S. The date component is mandatory,
the time component is optional. If the time component is
present, it will be truncted silently.
- from DATE to STRING. The resulting string value is formatted as
-MM-dd.
- from TIMESTAMP to DATE. The source timestamp's time of day
component is ignored.
- from DATE to TIMESTAMP. The target timestamp's time of day
component is set to 00:00:00.
- Implicit casting between DATE and other types:
- from STRING to DATE if the source string value is used in a
context where a DATE value is expected.
- from DATE to TIMESTAMP if the source date value is used in a
context where a TIMESTAMP value is expected.
- Since STRING -> DATE, STRING -> TIMESTAMP and DATE -> TIMESTAMP
implicit conversions are now all possible, the existing function
overload resolution logic is not adequate anymore.
For example, it resolves the
if(false, '2011-01-01', DATE '1499-02-02') function call to the
if(BOOLEAN, TIMESTAMP, TIMESTAMP) version of the overloaded
function, instead of the if(BOOLEAN, DATE, DATE) version.
This is clearly wrong, so the function overload resolution logic had
to be changed to resolve function calls to the best-fit overloaded
function definition if there are multiple applicable candidates.
An overloaded function definition is an applicable candidate for a
function call if each actual parameter in the function call either
matches the corresponding formal parameter's type (without casting)
or is implicitly castable to that type.
When looking for the best-fit applicable candidate, a parameter
match score (i.e. the number of actual parameters in the function
call that match their corresponding formal parameter's type without
casting) is calculated and the applicable candidate with the highest
parameter match score is chosen.
There's one more issue that the new resolution logic has to address:
if two applicable candidates have the same parameter match score and
the only difference between the two is that the first one requires a
STRING -> TIMESTAMP implicit cast for some of its parameters while
the second one requires a STRING -> DATE implicit cast for the same
parameters then the first candidate has to be chosen not to break
backward compatibility.
E.g: year('2019-02-15') function call must resolve to
year(TIMESTAMP) instead of year(DATE). Note, that year(DATE) is not
implemented yet, so this is not an issue at the moment but it will
be in the future.
When the resolution algorithm considers overloaded function
definitions, first it orders them lexicographically by the types in
their parameter lists. To ensure the backward compatible behavior
Primitivetype.DATE enum value has to come after
PrimitiveType.TIMESTAMP.
- Codegen infrastructure changes for expression evaluation.
- 'IS [NOT] NULL' and '[NOT] IN' predicates.
- Common comparison operators (including the 'BETWEEN' operator).
- Infrastructure changes for built-in functions.
- Some built-in functions: conditional, aggregate, analytical and
math functions.
- C++ UDF/UDA support.
- Support partitioning and grouping by DATE.
- Beeswax, HiveServer2 support.
These items are tightly coupled and it makes sense to implement them
in one change-set.
Testing:
- A new partitioned TEXT table 'functional.date_tbl' (and the
corresponding HBASE table 'functional_hbase.date_tbl') was
introduced for DATE-related tests.
- BE and FE tests were extended to cover DATE type.
- E2E tests:
- since DATE type is supported for TEXT and HBASE fileformats
only, most DATE tests were implemented separately in
tests/query_test/test_date_queries.py.
Note, that this change-set is not a complete DATE type implementation,
but it lays the foundation for future work:
- Add date support to the random query generator.
- Implement a complete set of built-in functions.
- Add Parquet support.
- Add Kudu support.
- Optionally support Avro and ORC.
For further details, see IMPALA-6169.
Change-Id: Iea8155ef09557e0afa2f8b2d0b2dc9d0896dc30f
---
M
[Impala-ASF-CR] IMPALA-6216: Make PYTHON EGG CACHE configurable
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12911 ) Change subject: IMPALA-6216: Make PYTHON_EGG_CACHE configurable .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/2649/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12911 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I695b2b31d9045eef1a53268f6516858935aed508 Gerrit-Change-Number: 12911 Gerrit-PatchSet: 3 Gerrit-Owner: Yongzhi Chen Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Yongzhi Chen Gerrit-Comment-Date: Thu, 04 Apr 2019 19:42:33 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8226: Add grant/revoke to/from group for Ranger
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12914 ) Change subject: IMPALA-8226: Add grant/revoke to/from group for Ranger .. Patch Set 10: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/2648/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12914 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96 Gerrit-Change-Number: 12914 Gerrit-PatchSet: 10 Gerrit-Owner: Austin Nobis Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: radford nguyen Gerrit-Comment-Date: Thu, 04 Apr 2019 19:36:50 + Gerrit-HasComments: No
[Impala-ASF-CR] Clean up generation of XML configuration files
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12930 ) Change subject: Clean up generation of XML configuration files .. Patch Set 3: (7 comments) http://gerrit.cloudera.org:8080/#/c/12930/3/bin/jenkins/critique-gerrit-review.py File bin/jenkins/critique-gerrit-review.py: http://gerrit.cloudera.org:8080/#/c/12930/3/bin/jenkins/critique-gerrit-review.py@72 PS3, Line 72: flake8: E261 at least two spaces before inline comment http://gerrit.cloudera.org:8080/#/c/12930/3/fe/src/test/resources/hive-site.xml.py File fe/src/test/resources/hive-site.xml.py: http://gerrit.cloudera.org:8080/#/c/12930/3/fe/src/test/resources/hive-site.xml.py@58 PS3, Line 58: 2 flake8: E501 line too long (96 > 90 characters) http://gerrit.cloudera.org:8080/#/c/12930/3/fe/src/test/resources/hive-site.xml.py@67 PS3, Line 67: e flake8: E501 line too long (94 > 90 characters) http://gerrit.cloudera.org:8080/#/c/12930/3/fe/src/test/resources/hive-site.xml.py@82 PS3, Line 82: i flake8: E501 line too long (119 > 90 characters) http://gerrit.cloudera.org:8080/#/c/12930/3/fe/src/test/resources/hive-site.xml.py@92 PS3, Line 92: f flake8: E501 line too long (108 > 90 characters) http://gerrit.cloudera.org:8080/#/c/12930/3/fe/src/test/resources/hive-site.xml.py@93 PS3, Line 93: . flake8: E501 line too long (117 > 90 characters) http://gerrit.cloudera.org:8080/#/c/12930/3/fe/src/test/resources/hive-site.xml.py@113 PS3, Line 113: t flake8: E501 line too long (112 > 90 characters) -- To view, visit http://gerrit.cloudera.org:8080/12930 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ief4434d80baae0fd7be7ffe7b2e07bae1ac45e47 Gerrit-Change-Number: 12930 Gerrit-PatchSet: 3 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Adam Holley Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Thu, 04 Apr 2019 19:31:40 + Gerrit-HasComments: Yes
[Impala-ASF-CR] Clean up generation of XML configuration files
Todd Lipcon has posted comments on this change. (
http://gerrit.cloudera.org:8080/12930 )
Change subject: Clean up generation of XML configuration files
..
Patch Set 2:
(7 comments)
http://gerrit.cloudera.org:8080/#/c/12930/2//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/12930/2//COMMIT_MSG@24
PS2, Line 24: the previous template.
> Could you list the flags out? Just to make it easier to understand. I did a
Done
http://gerrit.cloudera.org:8080/#/c/12930/2//COMMIT_MSG@25
PS2, Line 25:
> We should make sure to test on CentOS 6 since that still has python 2.6, ju
tested it on an el6 box and it appears to work.
http://gerrit.cloudera.org:8080/#/c/12930/2/bin/create-test-configuration.sh
File bin/create-test-configuration.sh:
http://gerrit.cloudera.org:8080/#/c/12930/2/bin/create-test-configuration.sh@32
PS2, Line 32: function generate_config {
> It would be good to leave a comment here or below with thoughts about why g
Done. Added a TODO for myself to convert over the remaining -site.xml files
(hbase, ranger-admin)
http://gerrit.cloudera.org:8080/#/c/12930/2/bin/generate_xml_config.py
File bin/generate_xml_config.py:
http://gerrit.cloudera.org:8080/#/c/12930/2/bin/generate_xml_config.py@80
PS2, Line 80: """ % dict(source_path=os.path.abspath(source_path))
> New-style formatting - .format(source_path=...) and {source_path} is genera
Done
http://gerrit.cloudera.org:8080/#/c/12930/2/bin/generate_xml_config.py@113
PS2, Line 113: os.unlink(tmp_path)
> Does this delete the tmp file as well?
yea, unlink = delete
http://gerrit.cloudera.org:8080/#/c/12930/2/fe/src/test/resources/hive-site.xml.py
File fe/src/test/resources/hive-site.xml.py:
http://gerrit.cloudera.org:8080/#/c/12930/2/fe/src/test/resources/hive-site.xml.py@69
PS2, Line 69: HBSE
> (you don't need to fix it btw, just couldn't resist commenting)
yea I actually wasted a few minutes because I accidentally fixed the typo and
then some stuff broke :) It's set with this typo elsewhere.
http://gerrit.cloudera.org:8080/#/c/12930/2/fe/src/test/resources/sentry-site.xml.py
File fe/src/test/resources/sentry-site.xml.py:
http://gerrit.cloudera.org:8080/#/c/12930/2/fe/src/test/resources/sentry-site.xml.py@49
PS2, Line 49: 'sentry.store.jdbc.url':
'jdbc:postgresql://localhost:5432/${SENTRY_POLICY_DB}/;create=true'
> we should fix this JDBC URL: https://gerrit.cloudera.org/c/12894/3/fe/src/t
Done
--
To view, visit http://gerrit.cloudera.org:8080/12930
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ief4434d80baae0fd7be7ffe7b2e07bae1ac45e47
Gerrit-Change-Number: 12930
Gerrit-PatchSet: 2
Gerrit-Owner: Todd Lipcon
Gerrit-Reviewer: Adam Holley
Gerrit-Reviewer: Fredy Wijaya
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Tim Armstrong
Gerrit-Reviewer: Todd Lipcon
Gerrit-Reviewer: Vihang Karajgaonkar
Gerrit-Comment-Date: Thu, 04 Apr 2019 19:30:49 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] Clean up generation of XML configuration files
Hello Vihang Karajgaonkar, Fredy Wijaya, Tim Armstrong, Impala Public Jenkins,
Adam Holley,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/12930
to look at the new patch set (#3).
Change subject: Clean up generation of XML configuration files
..
Clean up generation of XML configuration files
hive-site.xml and sentry-site.xml in particular had grown multiple
slightly-different variants, differing only in a few small pieces. This
was difficult to maintain: in fact, while attempting to clean them up I
found a number of places that the MySQL and Postgres versions of
hive-site had diverged for no apparent reason.
This moves away from using the sed-based templating for these
configuration files, and instead uses python as a poor man's template
system. That enables much simpler conditional logic.
I briefly considered XSLT for this, but decided that Python is probably
easier for the average developer to follow, modify, and debug.
Along the way, I removed a few flags which appear to be no longer used
by Hive 2 or later, and a few items which were already commented out in
the previous template:
- hive.stats.dbclass
- hive.stats.dbconnectionstring
- hive.stats.jdbcdriver
These are no longer relevant after HIVE-12164 ("Remove jdbc stats
collection mechanism") in Hive 2.0.
- hive.metastore.rawstore.impl
This has always defaulted to 'ObjectStore' in Hive, so there was no
reason to set it explicitly.
- test.log.dir
- test.src.dir
These were listed in the config in a commented-out section. These were
commented out ever since 2012 when the file was first introduced.
This also fixes the postgres URL to not include a misplaced ';create'
parameter (which applies to Derby but not postgres).
Change-Id: Ief4434d80baae0fd7be7ffe7b2e07bae1ac45e47
---
M bin/create-test-configuration.sh
A bin/generate_xml_config.py
M bin/jenkins/critique-gerrit-review.py
A fe/src/test/resources/hive-site.xml.py
D fe/src/test/resources/mysql-hive-site.xml.template
D fe/src/test/resources/postgresql-hive-site.xml.cdp.template
D fe/src/test/resources/postgresql-hive-site.xml.template
A fe/src/test/resources/sentry-site.xml.py
D fe/src/test/resources/sentry-site.xml.template
D fe/src/test/resources/sentry-site_no_oo.xml.template
D fe/src/test/resources/sentry-site_oo.xml.template
D fe/src/test/resources/sentry-site_oo_nogrant.xml.template
12 files changed, 321 insertions(+), 998 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/30/12930/3
--
To view, visit http://gerrit.cloudera.org:8080/12930
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ief4434d80baae0fd7be7ffe7b2e07bae1ac45e47
Gerrit-Change-Number: 12930
Gerrit-PatchSet: 3
Gerrit-Owner: Todd Lipcon
Gerrit-Reviewer: Adam Holley
Gerrit-Reviewer: Fredy Wijaya
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Tim Armstrong
Gerrit-Reviewer: Todd Lipcon
Gerrit-Reviewer: Vihang Karajgaonkar
[Impala-ASF-CR] IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12927 ) Change subject: IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/2646/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12927 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If46b4bf24d916e4a4ea8a36ff4acfd95d5f45c8e Gerrit-Change-Number: 12927 Gerrit-PatchSet: 2 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Thu, 04 Apr 2019 19:27:04 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger
Austin Nobis has posted comments on this change. (
http://gerrit.cloudera.org:8080/12927 )
Change subject: IMPALA-8363: Deny access when column masking or row filtering
is enabled in Ranger
..
Patch Set 2:
(4 comments)
http://gerrit.cloudera.org:8080/#/c/12927/2/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java
File
fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java:
http://gerrit.cloudera.org:8080/#/c/12927/2/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java@70
PS2, Line 70: private static class Table {
Wouldn't this represent a column and not a table?
http://gerrit.cloudera.org:8080/#/c/12927/2/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java@246
PS2, Line 246: for (Table table: tables) {
So you're iterating over the `Table` which is actually more like a `Column` but
ignoring the columnName_?
Won't this cause a lot of duplicate entries in the `rowFilterTables`?
http://gerrit.cloudera.org:8080/#/c/12927/2/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java@269
PS2, Line 269: private static List getTables(List exprs) {
Is this getting the tables or the columns?
http://gerrit.cloudera.org:8080/#/c/12927/2/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java@279
PS2, Line 279: // Column can be null when it's complex type.
If it is a complex type do you need to parse the complex type for more
potential columns?
--
To view, visit http://gerrit.cloudera.org:8080/12927
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If46b4bf24d916e4a4ea8a36ff4acfd95d5f45c8e
Gerrit-Change-Number: 12927
Gerrit-PatchSet: 2
Gerrit-Owner: Fredy Wijaya
Gerrit-Reviewer: Austin Nobis
Gerrit-Reviewer: Bharath Vissapragada
Gerrit-Reviewer: Fredy Wijaya
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Todd Lipcon
Gerrit-Comment-Date: Thu, 04 Apr 2019 19:26:03 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8226: Add grant/revoke to/from group for Ranger
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12914 ) Change subject: IMPALA-8226: Add grant/revoke to/from group for Ranger .. Patch Set 9: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/2647/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12914 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96 Gerrit-Change-Number: 12914 Gerrit-PatchSet: 9 Gerrit-Owner: Austin Nobis Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: radford nguyen Gerrit-Comment-Date: Thu, 04 Apr 2019 19:21:41 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6216: Make PYTHON EGG CACHE configurable
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12911 ) Change subject: IMPALA-6216: Make PYTHON_EGG_CACHE configurable .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/2645/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12911 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I695b2b31d9045eef1a53268f6516858935aed508 Gerrit-Change-Number: 12911 Gerrit-PatchSet: 2 Gerrit-Owner: Yongzhi Chen Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Yongzhi Chen Gerrit-Comment-Date: Thu, 04 Apr 2019 19:16:13 + Gerrit-HasComments: No
[Impala-ASF-CR] Configure Hive 3's HS2 to execute queries using Tez local mode
Vihang Karajgaonkar has posted comments on this change. ( http://gerrit.cloudera.org:8080/12931 ) Change subject: Configure Hive 3's HS2 to execute queries using Tez local mode .. Patch Set 2: Code-Review+1 (1 comment) http://gerrit.cloudera.org:8080/#/c/12931/2/fe/src/test/resources/hive-site.xml.py File fe/src/test/resources/hive-site.xml.py: http://gerrit.cloudera.org:8080/#/c/12931/2/fe/src/test/resources/hive-site.xml.py@79 PS2, Line 79: # Enable Tez and ACID for Hive 3 > Seems this was set to true by default in cbea5ad3a7500632d2c7586865174d6c9f Thanks for confirming. -- To view, visit http://gerrit.cloudera.org:8080/12931 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I76e47fbd1d6ff5103d81a8de430d5465dba284cd Gerrit-Change-Number: 12931 Gerrit-PatchSet: 2 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Thu, 04 Apr 2019 19:15:58 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-4865: Reject Expr Rewrite When Appropriate
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12814 ) Change subject: IMPALA-4865: Reject Expr Rewrite When Appropriate .. Patch Set 6: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/2644/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12814 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8b078113ccc1aa49b0cea0c86dff2e02e1dd0e23 Gerrit-Change-Number: 12814 Gerrit-PatchSet: 6 Gerrit-Owner: Fang-Yu Rao Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Paul Rogers Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 04 Apr 2019 19:11:34 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8226: Add grant/revoke to/from group for Ranger
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12914 ) Change subject: IMPALA-8226: Add grant/revoke to/from group for Ranger .. Patch Set 11: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3984/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/12914 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96 Gerrit-Change-Number: 12914 Gerrit-PatchSet: 11 Gerrit-Owner: Austin Nobis Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: radford nguyen Gerrit-Comment-Date: Thu, 04 Apr 2019 19:09:59 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8226: Add grant/revoke to/from group for Ranger
Fredy Wijaya has posted comments on this change. ( http://gerrit.cloudera.org:8080/12914 ) Change subject: IMPALA-8226: Add grant/revoke to/from group for Ranger .. Patch Set 10: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/12914 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96 Gerrit-Change-Number: 12914 Gerrit-PatchSet: 10 Gerrit-Owner: Austin Nobis Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: radford nguyen Gerrit-Comment-Date: Thu, 04 Apr 2019 19:09:43 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8226: Add grant/revoke to/from group for Ranger
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12914 ) Change subject: IMPALA-8226: Add grant/revoke to/from group for Ranger .. Patch Set 11: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/12914 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96 Gerrit-Change-Number: 12914 Gerrit-PatchSet: 11 Gerrit-Owner: Austin Nobis Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: radford nguyen Gerrit-Comment-Date: Thu, 04 Apr 2019 19:09:58 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6216: Make PYTHON EGG CACHE configurable
Yongzhi Chen has posted comments on this change. ( http://gerrit.cloudera.org:8080/12911 ) Change subject: IMPALA-6216: Make PYTHON_EGG_CACHE configurable .. Patch Set 3: Change to "it" -- To view, visit http://gerrit.cloudera.org:8080/12911 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I695b2b31d9045eef1a53268f6516858935aed508 Gerrit-Change-Number: 12911 Gerrit-PatchSet: 3 Gerrit-Owner: Yongzhi Chen Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Yongzhi Chen Gerrit-Comment-Date: Thu, 04 Apr 2019 19:08:03 + Gerrit-HasComments: No
[Impala-ASF-CR] Clean up generation of XML configuration files
Fredy Wijaya has posted comments on this change. (
http://gerrit.cloudera.org:8080/12930 )
Change subject: Clean up generation of XML configuration files
..
Patch Set 2:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/12930/2/fe/src/test/resources/sentry-site.xml.py
File fe/src/test/resources/sentry-site.xml.py:
http://gerrit.cloudera.org:8080/#/c/12930/2/fe/src/test/resources/sentry-site.xml.py@49
PS2, Line 49: 'sentry.store.jdbc.url':
'jdbc:postgresql://localhost:5432/${SENTRY_POLICY_DB}/;create=true'
we should fix this JDBC URL:
https://gerrit.cloudera.org/c/12894/3/fe/src/test/resources/sentry-site.xml.template
--
To view, visit http://gerrit.cloudera.org:8080/12930
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ief4434d80baae0fd7be7ffe7b2e07bae1ac45e47
Gerrit-Change-Number: 12930
Gerrit-PatchSet: 2
Gerrit-Owner: Todd Lipcon
Gerrit-Reviewer: Adam Holley
Gerrit-Reviewer: Fredy Wijaya
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Tim Armstrong
Gerrit-Reviewer: Todd Lipcon
Gerrit-Reviewer: Vihang Karajgaonkar
Gerrit-Comment-Date: Thu, 04 Apr 2019 19:06:59 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-6216: Make PYTHON EGG CACHE configurable
Hello Bharath Vissapragada, Andrew Sherman, Lars Volker, Laszlo Gaal, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/12911 to look at the new patch set (#3). Change subject: IMPALA-6216: Make PYTHON_EGG_CACHE configurable .. IMPALA-6216: Make PYTHON_EGG_CACHE configurable User can set environment variable PYTHON_EGG_CACHE before call impala-shell. Testing: Run impala-shell with or w/o PYTHON_EGG_CACHE configured Change-Id: I695b2b31d9045eef1a53268f6516858935aed508 --- M shell/impala-shell 1 file changed, 5 insertions(+), 3 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/11/12911/3 -- To view, visit http://gerrit.cloudera.org:8080/12911 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I695b2b31d9045eef1a53268f6516858935aed508 Gerrit-Change-Number: 12911 Gerrit-PatchSet: 3 Gerrit-Owner: Yongzhi Chen Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Yongzhi Chen
[Impala-ASF-CR] Configure Hive 3's HS2 to execute queries using Tez local mode
Todd Lipcon has posted comments on this change. ( http://gerrit.cloudera.org:8080/12931 ) Change subject: Configure Hive 3's HS2 to execute queries using Tez local mode .. Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/12931/2/fe/src/test/resources/hive-site.xml.py File fe/src/test/resources/hive-site.xml.py: http://gerrit.cloudera.org:8080/#/c/12931/2/fe/src/test/resources/hive-site.xml.py@79 PS2, Line 79: # Enable Tez and ACID for Hive 3 > Yea, I had that set for a while, but then found I was able to remove it and Seems this was set to true by default in cbea5ad3a7500632d2c7586865174d6c9f4499ec TEZ-2333 for Tez 0.7 so no longer necessary. Will submit a tez docs patch -- To view, visit http://gerrit.cloudera.org:8080/12931 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I76e47fbd1d6ff5103d81a8de430d5465dba284cd Gerrit-Change-Number: 12931 Gerrit-PatchSet: 2 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Thu, 04 Apr 2019 18:57:28 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8226: Add grant/revoke to/from group for Ranger
Austin Nobis has uploaded a new patch set (#10). ( http://gerrit.cloudera.org:8080/12914 ) Change subject: IMPALA-8226: Add grant/revoke to/from group for Ranger .. IMPALA-8226: Add grant/revoke to/from group for Ranger This patch adds fupport for GRANT privilege statements to GROUP and REVOKE privilege statements from GROUP. The grammar has been updated to support FROM GROUP and TO GROUP for GRANT/REVOKE statements, i.e: GRANT ON TO GROUP REVOKE ON FROM GROUP Currently, only Ranger's authorization implementation supports GROUP based privileges. Sentry will throw an UnsupportedOperationException if it is the enabled authorization provider and this new grammar is used. Testing: - AuthorizationStmtTest was updated to also test for GROUP authorization. - ToSqlTest was updated to test for GROUP changes to the grammar. - A GROUP based E2E test was added to test_ranger.py - ParserTest was updated to test combinations for GrantRevokePrivilege - AnalyzeAuthStmtsTest was updated to test for USER and GROUP identities - Ran all FE tests - Ran authorization E2E tests Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96 --- M common/thrift/CatalogObjects.thrift M fe/src/main/cup/sql-parser.cup M fe/src/main/java/org/apache/impala/authorization/AuthorizationManager.java M fe/src/main/java/org/apache/impala/authorization/NoneAuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerCatalogdAuthorizationManager.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryCatalogdAuthorizationManager.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryImpaladAuthorizationManager.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeAuthStmtsTest.java M fe/src/test/java/org/apache/impala/analysis/AuthorizationStmtTest.java M fe/src/test/java/org/apache/impala/analysis/ParserTest.java M fe/src/test/java/org/apache/impala/analysis/ToSqlTest.java M fe/src/test/resources/ranger-hive-security.xml M testdata/bin/create-load-data.sh A testdata/cluster/ranger/setup/impala_group.json.template M testdata/cluster/ranger/setup/impala_user.json.template M tests/authorization/test_ranger.py 17 files changed, 461 insertions(+), 323 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/14/12914/10 -- To view, visit http://gerrit.cloudera.org:8080/12914 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96 Gerrit-Change-Number: 12914 Gerrit-PatchSet: 10 Gerrit-Owner: Austin Nobis Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: radford nguyen
[Impala-ASF-CR] IMPALA-8226: Add grant/revoke to/from group for Ranger
Austin Nobis has posted comments on this change. ( http://gerrit.cloudera.org:8080/12914 ) Change subject: IMPALA-8226: Add grant/revoke to/from group for Ranger .. Patch Set 9: Fixed an issue caused by the group configuration on the Jenkins host that caused the merge to fail. Tested here: https://master-02.jenkins.cloudera.com/view/Impala/view/Private/job/impala-private-parameterized/4690/ -- To view, visit http://gerrit.cloudera.org:8080/12914 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96 Gerrit-Change-Number: 12914 Gerrit-PatchSet: 9 Gerrit-Owner: Austin Nobis Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: radford nguyen Gerrit-Comment-Date: Thu, 04 Apr 2019 18:56:42 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8226: Add grant/revoke to/from group for Ranger
Austin Nobis has uploaded a new patch set (#9). ( http://gerrit.cloudera.org:8080/12914 ) Change subject: IMPALA-8226: Add grant/revoke to/from group for Ranger .. IMPALA-8226: Add grant/revoke to/from group for Ranger This patch adds fupport for GRANT privilege statements to GROUP and REVOKE privilege statements from GROUP. The grammar has been updated to support FROM GROUP and TO GROUP for GRANT/REVOKE statements, i.e: GRANT ON TO GROUP REVOKE ON FROM GROUP Currently, only Ranger's authorization implementation supports GROUP based privileges. Sentry will throw an UnsupportedOperationException if it is the enabled authorization provider and this new grammar is used. Testing: - AuthorizationStmtTest was updated to also test for GROUP authorization. - ToSqlTest was updated to test for GROUP changes to the grammar. - A GROUP based E2E test was added to test_ranger.py - ParserTest was updated to test combinations for GrantRevokePrivilege - AnalyzeAuthStmtsTest was updated to test for USER and GROUP identities - Ran all FE tests - Ran authorization E2E tests Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96 --- M common/thrift/CatalogObjects.thrift M fe/src/main/cup/sql-parser.cup M fe/src/main/java/org/apache/impala/authorization/AuthorizationManager.java M fe/src/main/java/org/apache/impala/authorization/NoneAuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerCatalogdAuthorizationManager.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryCatalogdAuthorizationManager.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryImpaladAuthorizationManager.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeAuthStmtsTest.java M fe/src/test/java/org/apache/impala/analysis/AuthorizationStmtTest.java M fe/src/test/java/org/apache/impala/analysis/ParserTest.java M fe/src/test/java/org/apache/impala/analysis/ToSqlTest.java M fe/src/test/resources/ranger-hive-security.xml M testdata/bin/create-load-data.sh A testdata/cluster/ranger/setup/impala_group.json.template M testdata/cluster/ranger/setup/impala_user.json.template M tests/authorization/test_ranger.py 17 files changed, 463 insertions(+), 323 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/14/12914/9 -- To view, visit http://gerrit.cloudera.org:8080/12914 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96 Gerrit-Change-Number: 12914 Gerrit-PatchSet: 9 Gerrit-Owner: Austin Nobis Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: radford nguyen
[Impala-ASF-CR] IMPALA-7640: RENAME on managed Kudu table should rename Kudu table
Thomas Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/12179 ) Change subject: IMPALA-7640: RENAME on managed Kudu table should rename Kudu table .. Patch Set 4: (1 comment) Please be sure to get the docs updated to reflect this change, eg: http://impala.apache.org/docs/build/html/topics/impala_kudu.html http://impala.apache.org/docs/build/html/topics/impala_tables.html http://gerrit.cloudera.org:8080/#/c/12179/4/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java File fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java: http://gerrit.cloudera.org:8080/#/c/12179/4/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@2378 PS4, Line 2378: oldTbl instanceof KuduTable && !Table.isExternalTable(msTbl) Is it possible to do this check as part of altersKuduTable() (eg. by adding 'msTbl' as a parameter) and then call your new renameKuduTable() in alterKuduTable() so that this follows the same code path as other 'alter' operations that need to be reflected both in HMS and in Kudu? -- To view, visit http://gerrit.cloudera.org:8080/12179 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I77e7583ce93cba8f6e743c4bedd9900ae1fae081 Gerrit-Change-Number: 12179 Gerrit-PatchSet: 4 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Mike Percy Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Thu, 04 Apr 2019 18:51:40 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger
Fredy Wijaya has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/12927 ) Change subject: IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger .. IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger This patch updates the Ranger authorization checker code to deny access when column masking and row filtering is enabled in Ranger for queries that that have columns/tables specified in column mask and row filter policies. This is to prevent data leak, such that the data that is masked/filtered in Hive should not be visible at all in Impala until Impala has full support for column masking and row filtering. Testing: - Added tests in AuthorizationStmtTest to test queries with column masking and row filtering enabled. - Ran all FE tests - Ran all E2E tests Change-Id: If46b4bf24d916e4a4ea8a36ff4acfd95d5f45c8e --- M fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java M fe/src/main/java/org/apache/impala/analysis/CreateOrAlterViewStmtBase.java M fe/src/main/java/org/apache/impala/analysis/ModifyStmt.java M fe/src/main/java/org/apache/impala/authorization/AuthorizationChecker.java M fe/src/main/java/org/apache/impala/authorization/AuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/AuthorizationProvider.java R fe/src/main/java/org/apache/impala/authorization/NoopAuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryAuthorizationChecker.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryAuthorizationFactory.java M fe/src/main/java/org/apache/impala/service/JniCatalog.java M fe/src/main/java/org/apache/impala/service/JniFrontend.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeAuthStmtsTest.java M fe/src/test/java/org/apache/impala/analysis/AuthorizationStmtTest.java M fe/src/test/java/org/apache/impala/analysis/StmtMetadataLoaderTest.java M fe/src/test/java/org/apache/impala/catalog/AlterDatabaseTest.java M fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java M fe/src/test/java/org/apache/impala/common/FrontendFixture.java M fe/src/test/java/org/apache/impala/common/FrontendTestBase.java M fe/src/test/java/org/apache/impala/common/QueryFixture.java M fe/src/test/java/org/apache/impala/testutil/ImpaladTestCatalog.java M fe/src/test/java/org/apache/impala/testutil/PlannerTestCaseLoader.java 23 files changed, 444 insertions(+), 81 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/27/12927/2 -- To view, visit http://gerrit.cloudera.org:8080/12927 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: If46b4bf24d916e4a4ea8a36ff4acfd95d5f45c8e Gerrit-Change-Number: 12927 Gerrit-PatchSet: 2 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon
[Impala-ASF-CR] IMPALA-6216: Make PYTHON EGG CACHE configurable
Bharath Vissapragada has posted comments on this change. ( http://gerrit.cloudera.org:8080/12911 ) Change subject: IMPALA-6216: Make PYTHON_EGG_CACHE configurable .. Patch Set 2: Code-Review+2 (1 comment) http://gerrit.cloudera.org:8080/#/c/12911/2/shell/impala-shell File shell/impala-shell: http://gerrit.cloudera.org:8080/#/c/12911/2/shell/impala-shell@39 PS2, Line 39: EGG_CACHE nit: May be just say "By default it is set" Seems obvious that we are referring to the the pyegg cache. -- To view, visit http://gerrit.cloudera.org:8080/12911 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I695b2b31d9045eef1a53268f6516858935aed508 Gerrit-Change-Number: 12911 Gerrit-PatchSet: 2 Gerrit-Owner: Yongzhi Chen Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Yongzhi Chen Gerrit-Comment-Date: Thu, 04 Apr 2019 18:39:30 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-6216: Make PYTHON EGG CACHE configurable
Yongzhi Chen has posted comments on this change. ( http://gerrit.cloudera.org:8080/12911 ) Change subject: IMPALA-6216: Make PYTHON_EGG_CACHE configurable .. Patch Set 2: (1 comment) Submit patch 2 to address review issues. http://gerrit.cloudera.org:8080/#/c/12911/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/12911/1//COMMIT_MSG@13 PS1, Line 13: Run impala-shell with or w/o PYTHON_EGG_CACHE configured > I thought this wrapper is bypassed when calling bin/impala-shell.sh . Did y I did not call impala-shell.sh to test, just call shell/impala-shell -- To view, visit http://gerrit.cloudera.org:8080/12911 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I695b2b31d9045eef1a53268f6516858935aed508 Gerrit-Change-Number: 12911 Gerrit-PatchSet: 2 Gerrit-Owner: Yongzhi Chen Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Yongzhi Chen Gerrit-Comment-Date: Thu, 04 Apr 2019 18:32:07 + Gerrit-HasComments: Yes
[Impala-ASF-CR] Configure Hive 3's HS2 to execute queries using Tez local mode
Todd Lipcon has posted comments on this change. ( http://gerrit.cloudera.org:8080/12931 ) Change subject: Configure Hive 3's HS2 to execute queries using Tez local mode .. Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/12931/2/fe/src/test/resources/hive-site.xml.py File fe/src/test/resources/hive-site.xml.py: http://gerrit.cloudera.org:8080/#/c/12931/2/fe/src/test/resources/hive-site.xml.py@79 PS2, Line 79: # Enable Tez and ACID for Hive 3 > Do we need to have a tez-site.xml as well? Also, according to https://tez.a Yea, I had that set for a while, but then found I was able to remove it and it still worked. It may be that it would be necessary for more complex queries. Let me try a multi-stage query over some bigger data and see if I notice either a big perf difference or if it's required for correctness or what. -- To view, visit http://gerrit.cloudera.org:8080/12931 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I76e47fbd1d6ff5103d81a8de430d5465dba284cd Gerrit-Change-Number: 12931 Gerrit-PatchSet: 2 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Thu, 04 Apr 2019 18:34:58 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8101: Thrift 11 and ext-data-source compilation are always run
Sahil Takiar has posted comments on this change. ( http://gerrit.cloudera.org:8080/12290 ) Change subject: IMPALA-8101: Thrift 11 and ext-data-source compilation are always run .. Patch Set 2: I rebased this patch locally and re-ran the build. Ran a few basic queries on the impala-shell and a few of the Python tests, and everything is working as expected. If anyone thinks its necessary, I can re-run the full test suite. Otherwise this patch should be good to. -- To view, visit http://gerrit.cloudera.org:8080/12290 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I52520e4b099c7bac5d088b4ba5d8a335495f727d Gerrit-Change-Number: 12290 Gerrit-PatchSet: 2 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Sahil Takiar Gerrit-Comment-Date: Thu, 04 Apr 2019 18:31:54 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4865: Reject Expr Rewrite When Appropriate
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12814 ) Change subject: IMPALA-4865: Reject Expr Rewrite When Appropriate .. Patch Set 6: (1 comment) http://gerrit.cloudera.org:8080/#/c/12814/6/fe/src/main/java/org/apache/impala/rewrite/FoldConstantsRule.java File fe/src/main/java/org/apache/impala/rewrite/FoldConstantsRule.java: http://gerrit.cloudera.org:8080/#/c/12814/6/fe/src/main/java/org/apache/impala/rewrite/FoldConstantsRule.java@67 PS6, Line 67: Expr result = LiteralExpr.create(expr, analyzer.getQueryCtx(), MAX_STRING_LITERAL_SIZE); line too long (92 > 90) -- To view, visit http://gerrit.cloudera.org:8080/12814 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8b078113ccc1aa49b0cea0c86dff2e02e1dd0e23 Gerrit-Change-Number: 12814 Gerrit-PatchSet: 6 Gerrit-Owner: Fang-Yu Rao Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Paul Rogers Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 04 Apr 2019 18:30:11 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-6216: Make PYTHON EGG CACHE configurable
Hello Bharath Vissapragada, Andrew Sherman, Lars Volker, Laszlo Gaal, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/12911 to look at the new patch set (#2). Change subject: IMPALA-6216: Make PYTHON_EGG_CACHE configurable .. IMPALA-6216: Make PYTHON_EGG_CACHE configurable User can set environment variable PYTHON_EGG_CACHE before call impala-shell. Testing: Run impala-shell with or w/o PYTHON_EGG_CACHE configured Change-Id: I695b2b31d9045eef1a53268f6516858935aed508 --- M shell/impala-shell 1 file changed, 5 insertions(+), 3 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/11/12911/2 -- To view, visit http://gerrit.cloudera.org:8080/12911 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I695b2b31d9045eef1a53268f6516858935aed508 Gerrit-Change-Number: 12911 Gerrit-PatchSet: 2 Gerrit-Owner: Yongzhi Chen Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Laszlo Gaal
[Impala-ASF-CR] IMPALA-4865: Reject Expr Rewrite When Appropriate
Fang-Yu Rao has uploaded a new patch set (#6). ( http://gerrit.cloudera.org:8080/12814 ) Change subject: IMPALA-4865: Reject Expr Rewrite When Appropriate .. IMPALA-4865: Reject Expr Rewrite When Appropriate Avoided rewrite if the resulting string literal exceeds a defined limit. Testing: Added three statements in testFoldConstantsRule() to verify that the expression rewrite is accepted only when the size of the rewritten expression is below a specified threshold. Change-Id: I8b078113ccc1aa49b0cea0c86dff2e02e1dd0e23 --- M .gitignore M be/src/service/fe-support.cc M common/thrift/generate_error_codes.py M fe/src/main/java/org/apache/impala/analysis/AnalyticExpr.java M fe/src/main/java/org/apache/impala/analysis/AnalyticWindow.java M fe/src/main/java/org/apache/impala/analysis/ColumnDef.java M fe/src/main/java/org/apache/impala/analysis/Expr.java M fe/src/main/java/org/apache/impala/analysis/FunctionCallExpr.java M fe/src/main/java/org/apache/impala/analysis/LiteralExpr.java M fe/src/main/java/org/apache/impala/analysis/PartitionKeyValue.java M fe/src/main/java/org/apache/impala/analysis/RangePartition.java M fe/src/main/java/org/apache/impala/authorization/AuthorizationFactory.java M fe/src/main/java/org/apache/impala/planner/HBaseScanNode.java M fe/src/main/java/org/apache/impala/rewrite/FoldConstantsRule.java M fe/src/main/java/org/apache/impala/rewrite/RemoveRedundantStringCast.java M fe/src/main/java/org/apache/impala/service/FeSupport.java M fe/src/main/java/org/apache/impala/util/Graph.java M fe/src/main/java/org/apache/impala/util/KuduUtil.java M fe/src/test/java/org/apache/impala/analysis/ExprRewriteRulesTest.java 19 files changed, 103 insertions(+), 46 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/14/12814/6 -- To view, visit http://gerrit.cloudera.org:8080/12814 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I8b078113ccc1aa49b0cea0c86dff2e02e1dd0e23 Gerrit-Change-Number: 12814 Gerrit-PatchSet: 6 Gerrit-Owner: Fang-Yu Rao Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Paul Rogers Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-8377: bump toolchain version to 107-acaeac961d
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12928 ) Change subject: IMPALA-8377: bump toolchain version to 107-acaeac961d .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/2643/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12928 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4ac25aa230b9d2559cd4eb6166ab985b18ef7e2a Gerrit-Change-Number: 12928 Gerrit-PatchSet: 2 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Hector Acosta Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 04 Apr 2019 18:24:58 + Gerrit-HasComments: No
[Impala-ASF-CR] Revert "Bump toolchain version to 8430101a-b65f-4e4d-921e-767771649ae4-713afc77a8"
Thomas Marshall has abandoned this change. ( http://gerrit.cloudera.org:8080/12905 ) Change subject: Revert "Bump toolchain version to 8430101a-b65f-4e4d-921e-767771649ae4-713afc77a8" .. Abandoned Instead of reverting, we're just solving the issue with: https://gerrit.cloudera.org/#/c/12928/ -- To view, visit http://gerrit.cloudera.org:8080/12905 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: abandon Gerrit-Change-Id: I501721c428b63d035ad618ac7da97acfb828537c Gerrit-Change-Number: 12905 Gerrit-PatchSet: 1 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker
[Impala-ASF-CR] Configure Hive 3's HS2 to execute queries using Tez local mode
Vihang Karajgaonkar has posted comments on this change. ( http://gerrit.cloudera.org:8080/12931 ) Change subject: Configure Hive 3's HS2 to execute queries using Tez local mode .. Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/12931/2/fe/src/test/resources/hive-site.xml.py File fe/src/test/resources/hive-site.xml.py: http://gerrit.cloudera.org:8080/#/c/12931/2/fe/src/test/resources/hive-site.xml.py@79 PS2, Line 79: # Enable Tez and ACID for Hive 3 Do we need to have a tez-site.xml as well? Also, according to https://tez.apache.org/localmode.html looks like we need "tez.runtime.optimize.local.fetch" = "true" although I have no idea on what that does. -- To view, visit http://gerrit.cloudera.org:8080/12931 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I76e47fbd1d6ff5103d81a8de430d5465dba284cd Gerrit-Change-Number: 12931 Gerrit-PatchSet: 2 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Thu, 04 Apr 2019 17:59:38 + Gerrit-HasComments: Yes
[Impala-ASF-CR] Clean up generation of XML configuration files
Vihang Karajgaonkar has posted comments on this change. ( http://gerrit.cloudera.org:8080/12930 ) Change subject: Clean up generation of XML configuration files .. Patch Set 2: Code-Review+1 (1 comment) Just a question related to cleanup on errors. Rest looks good to me. Thanks for the patch. I think its a lot cleaner now. http://gerrit.cloudera.org:8080/#/c/12930/2/bin/generate_xml_config.py File bin/generate_xml_config.py: http://gerrit.cloudera.org:8080/#/c/12930/2/bin/generate_xml_config.py@113 PS2, Line 113: os.unlink(tmp_path) Does this delete the tmp file as well? -- To view, visit http://gerrit.cloudera.org:8080/12930 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ief4434d80baae0fd7be7ffe7b2e07bae1ac45e47 Gerrit-Change-Number: 12930 Gerrit-PatchSet: 2 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Adam Holley Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Thu, 04 Apr 2019 17:49:40 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8377: bump toolchain version to 107-acaeac961d
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12928 ) Change subject: IMPALA-8377: bump toolchain version to 107-acaeac961d .. Patch Set 2: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3983/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/12928 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4ac25aa230b9d2559cd4eb6166ab985b18ef7e2a Gerrit-Change-Number: 12928 Gerrit-PatchSet: 2 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Hector Acosta Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 04 Apr 2019 17:52:47 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8377: bump toolchain version to 107-acaeac961d
Hello Lars Volker, Hector Acosta, Tim Armstrong, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/12928 to look at the new patch set (#2). Change subject: IMPALA-8377: bump toolchain version to 107-acaeac961d .. IMPALA-8377: bump toolchain version to 107-acaeac961d This fixes an issue with the previous toolchain version where the Kudu client was broken and caused all binaries to crash on startup due to an issue with linked libstdc++ It also fixes an issue where fastbinary.so wasn't being properly included with Thrift. Testing: - Built successfully on redhat6/7, ubuntu16/18, sles12, debian8 - Built and ran a full core test run with both USE_CDH_KUDU=true/false Change-Id: I4ac25aa230b9d2559cd4eb6166ab985b18ef7e2a --- M bin/impala-config.sh 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/28/12928/2 -- To view, visit http://gerrit.cloudera.org:8080/12928 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I4ac25aa230b9d2559cd4eb6166ab985b18ef7e2a Gerrit-Change-Number: 12928 Gerrit-PatchSet: 2 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Hector Acosta Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-8377: bump toolchain version to 107-acaeac961d
Thomas Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/12928 ) Change subject: IMPALA-8377: bump toolchain version to 107-acaeac961d .. Patch Set 2: Code-Review+2 carrying forward -- To view, visit http://gerrit.cloudera.org:8080/12928 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4ac25aa230b9d2559cd4eb6166ab985b18ef7e2a Gerrit-Change-Number: 12928 Gerrit-PatchSet: 2 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Hector Acosta Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 04 Apr 2019 17:52:22 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8377: bump toolchain version to 107-acaeac961d
Thomas Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/12928 ) Change subject: IMPALA-8377: bump toolchain version to 107-acaeac961d .. Patch Set 2: > Seems good to me aside from the conflict with Hector's patch - I > think you are figuring that out with him, right? I confirmed that my patch includes his fix, so he's going to abandon his and we'll submit this one -- To view, visit http://gerrit.cloudera.org:8080/12928 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4ac25aa230b9d2559cd4eb6166ab985b18ef7e2a Gerrit-Change-Number: 12928 Gerrit-PatchSet: 2 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Hector Acosta Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 04 Apr 2019 17:52:11 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8383 Bump toolchain version
Hector Acosta has abandoned this change. ( http://gerrit.cloudera.org:8080/12923 ) Change subject: IMPALA-8383 Bump toolchain version .. Abandoned Included in another review -- To view, visit http://gerrit.cloudera.org:8080/12923 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: abandon Gerrit-Change-Id: If221f001bc106530c5faf08b38385028c056983f Gerrit-Change-Number: 12923 Gerrit-PatchSet: 1 Gerrit-Owner: Hector Acosta Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] Clean up generation of XML configuration files
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/12930 ) Change subject: Clean up generation of XML configuration files .. Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/12930/2/fe/src/test/resources/hive-site.xml.py File fe/src/test/resources/hive-site.xml.py: http://gerrit.cloudera.org:8080/#/c/12930/2/fe/src/test/resources/hive-site.xml.py@69 PS2, Line 69: HBSE > I'm so confused by this abbreviation - why omit the A? (you don't need to fix it btw, just couldn't resist commenting) -- To view, visit http://gerrit.cloudera.org:8080/12930 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ief4434d80baae0fd7be7ffe7b2e07bae1ac45e47 Gerrit-Change-Number: 12930 Gerrit-PatchSet: 2 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Adam Holley Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Thu, 04 Apr 2019 17:44:23 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8377: bump toolchain version to 107-acaeac961d
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/12928 ) Change subject: IMPALA-8377: bump toolchain version to 107-acaeac961d .. Patch Set 1: Code-Review+2 Seems good to me aside from the conflict with Hector's patch - I think you are figuring that out with him, right? -- To view, visit http://gerrit.cloudera.org:8080/12928 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4ac25aa230b9d2559cd4eb6166ab985b18ef7e2a Gerrit-Change-Number: 12928 Gerrit-PatchSet: 1 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Hector Acosta Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 04 Apr 2019 17:43:47 + Gerrit-HasComments: No
[Impala-ASF-CR] Clean up generation of XML configuration files
Tim Armstrong has posted comments on this change. (
http://gerrit.cloudera.org:8080/12930 )
Change subject: Clean up generation of XML configuration files
..
Patch Set 2:
(5 comments)
Ok no worries, yeah that makes sense. You could add the file to the list in
bin/jenkins/critique-gerrit-review.py, I have no issue with that if it's
problematic.
Overall this looks good, mainly just some questions about comments.
http://gerrit.cloudera.org:8080/#/c/12930/2//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/12930/2//COMMIT_MSG@24
PS2, Line 24: the previous template.
Could you list the flags out? Just to make it easier to understand. I did a
manual comparison and got:
* hive.stats.dbclass
* hive.metastore.rawstore.impl
http://gerrit.cloudera.org:8080/#/c/12930/2//COMMIT_MSG@25
PS2, Line 25:
We should make sure to test on CentOS 6 since that still has python 2.6, just
in case there's something incompatible in the new Python code. :sadpanda:.
http://gerrit.cloudera.org:8080/#/c/12930/2/bin/create-test-configuration.sh
File bin/create-test-configuration.sh:
http://gerrit.cloudera.org:8080/#/c/12930/2/bin/create-test-configuration.sh@32
PS2, Line 32: function generate_config {
It would be good to leave a comment here or below with thoughts about why
generate_config would be used over the python-based templating (or if we want
to port everything to python-based templating eventually).
Otherwise people will be confused about why two mechanisms exist.
http://gerrit.cloudera.org:8080/#/c/12930/2/bin/generate_xml_config.py
File bin/generate_xml_config.py:
http://gerrit.cloudera.org:8080/#/c/12930/2/bin/generate_xml_config.py@80
PS2, Line 80: """ % dict(source_path=os.path.abspath(source_path))
New-style formatting - .format(source_path=...) and {source_path} is generally
preferred in new code over %. Not a hard requirement but I think it should be
easy to change over here and elsewhere.
http://gerrit.cloudera.org:8080/#/c/12930/2/fe/src/test/resources/hive-site.xml.py
File fe/src/test/resources/hive-site.xml.py:
http://gerrit.cloudera.org:8080/#/c/12930/2/fe/src/test/resources/hive-site.xml.py@69
PS2, Line 69: HBSE
I'm so confused by this abbreviation - why omit the A?
--
To view, visit http://gerrit.cloudera.org:8080/12930
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ief4434d80baae0fd7be7ffe7b2e07bae1ac45e47
Gerrit-Change-Number: 12930
Gerrit-PatchSet: 2
Gerrit-Owner: Todd Lipcon
Gerrit-Reviewer: Adam Holley
Gerrit-Reviewer: Fredy Wijaya
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Tim Armstrong
Gerrit-Reviewer: Todd Lipcon
Gerrit-Reviewer: Vihang Karajgaonkar
Gerrit-Comment-Date: Thu, 04 Apr 2019 17:36:06 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] Clean up generation of XML configuration files
Todd Lipcon has posted comments on this change. ( http://gerrit.cloudera.org:8080/12930 ) Change subject: Clean up generation of XML configuration files .. Patch Set 2: Thanks. I think for this case the "long lines" are probably a good idea for readability vs lots of weird breaking of long strings. I spent some time trying to see if I could disable the "long line" warning on flake8 for a single file, but just ended up finding some bikeshed github issue arguments about whether that's a good feature or not... and I gave up. Looks like we don't have a standard of "flake8 clean" so leaving these for now. -- To view, visit http://gerrit.cloudera.org:8080/12930 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ief4434d80baae0fd7be7ffe7b2e07bae1ac45e47 Gerrit-Change-Number: 12930 Gerrit-PatchSet: 2 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Adam Holley Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Thu, 04 Apr 2019 17:29:28 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7995: part 1: fixes for e2e dockerised impala tests
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12639 ) Change subject: IMPALA-7995: part 1: fixes for e2e dockerised impala tests .. Patch Set 12: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/2642/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12639 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iee86cbd2c4631a014af1e8cef8e1cd523a812755 Gerrit-Change-Number: 12639 Gerrit-PatchSet: 12 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 04 Apr 2019 17:13:28 + Gerrit-HasComments: No
[Impala-ASF-CR] Clean up generation of XML configuration files
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/12930 ) Change subject: Clean up generation of XML configuration files .. Patch Set 2: If you want, you can prefetch the bot comments with ./bin/jenkins/critique-gerrit-review.py --dryrun -- To view, visit http://gerrit.cloudera.org:8080/12930 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ief4434d80baae0fd7be7ffe7b2e07bae1ac45e47 Gerrit-Change-Number: 12930 Gerrit-PatchSet: 2 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Adam Holley Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Thu, 04 Apr 2019 17:11:59 + Gerrit-HasComments: No
[Impala-ASF-CR] Configure Hive 3's HS2 to execute queries using Tez local mode
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12931 ) Change subject: Configure Hive 3's HS2 to execute queries using Tez local mode .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/2641/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12931 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I76e47fbd1d6ff5103d81a8de430d5465dba284cd Gerrit-Change-Number: 12931 Gerrit-PatchSet: 2 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Thu, 04 Apr 2019 16:59:27 + Gerrit-HasComments: No
[Impala-ASF-CR] Clean up generation of XML configuration files
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12930 ) Change subject: Clean up generation of XML configuration files .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/2640/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12930 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ief4434d80baae0fd7be7ffe7b2e07bae1ac45e47 Gerrit-Change-Number: 12930 Gerrit-PatchSet: 2 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Adam Holley Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Thu, 04 Apr 2019 16:52:43 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7995: part 1: fixes for e2e dockerised impala tests
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12639 ) Change subject: IMPALA-7995: part 1: fixes for e2e dockerised impala tests .. Patch Set 13: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3982/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/12639 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iee86cbd2c4631a014af1e8cef8e1cd523a812755 Gerrit-Change-Number: 12639 Gerrit-PatchSet: 13 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 04 Apr 2019 16:31:08 + Gerrit-HasComments: No
