[Impala-ASF-CR] IMPALA-7163: Implement a state machine for the QueryState class
Sailesh Mukil has posted comments on this change. ( http://gerrit.cloudera.org:8080/10813 ) Change subject: IMPALA-7163: Implement a state machine for the QueryState class .. Patch Set 14: (1 comment) http://gerrit.cloudera.org:8080/#/c/10813/13/be/src/runtime/fragment-instance-state.cc File be/src/runtime/fragment-instance-state.cc: http://gerrit.cloudera.org:8080/#/c/10813/13/be/src/runtime/fragment-instance-state.cc@107 PS13, Line 107: er > A simpler alternative to relying on this enum is to save a boolean above wh I think it's better to rely on the enum ordering. Otherwise, the new boolean is just redundant with the one of the states in the state machine. We also rely on enum ordering in other parts in the code, so I think this should be fine. -- To view, visit http://gerrit.cloudera.org:8080/10813 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iec5670a7db83ecae4656d7bb2ea372d3767ba7fe Gerrit-Change-Number: 10813 Gerrit-PatchSet: 14 Gerrit-Owner: Sailesh Mukil Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sailesh Mukil Gerrit-Comment-Date: Mon, 06 Aug 2018 01:14:06 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7387: Set correct MIME type for JSON webpages
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/0 ) Change subject: IMPALA-7387: Set correct MIME type for JSON webpages .. Patch Set 11: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/0 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7ad94343151730851a4ad01a1f4b9326a36a37ea Gerrit-Change-Number: 0 Gerrit-PatchSet: 11 Gerrit-Owner: Bharath Vissapragada Gerrit-Reviewer: Anonymous Coward #168 Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Comment-Date: Sun, 05 Aug 2018 22:15:34 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7213: Port ReportExecStatus() RPC to use KRPC
Sailesh Mukil has posted comments on this change. (
http://gerrit.cloudera.org:8080/10855 )
Change subject: IMPALA-7213: Port ReportExecStatus() RPC to use KRPC
..
Patch Set 5:
(8 comments)
http://gerrit.cloudera.org:8080/#/c/10855/5/be/src/runtime/query-state.h
File be/src/runtime/query-state.h:
http://gerrit.cloudera.org:8080/#/c/10855/5/be/src/runtime/query-state.h@276
PS5, Line 276: to construct a fragment instance's status report
"... to construct a status report ..."
Since if 'fis' is nullptr, then we send a generic report with only the status
of the query.
http://gerrit.cloudera.org:8080/#/c/10855/5/be/src/runtime/query-state.h@277
PS5, Line 277: The runtime profile
"If 'fis' is not 'nullptr', the runtime profile ..."
http://gerrit.cloudera.org:8080/#/c/10855/5/be/src/runtime/query-state.cc
File be/src/runtime/query-state.cc:
http://gerrit.cloudera.org:8080/#/c/10855/5/be/src/runtime/query-state.cc@256
PS5, Line 256: query ID $1
Shouldn't we print the fragment instance ID instead?
http://gerrit.cloudera.org:8080/#/c/10855/5/be/src/runtime/query-state.cc@295
PS5, Line 295: The profile is serialized in Thrift
The profile is a thrift structure serialized to a string ...
http://gerrit.cloudera.org:8080/#/c/10855/5/be/src/runtime/query-state.cc@306
PS5, Line 306: sidecar_buf->assign_copy(profile_str);
We could probably consider moving 'profile_str' to the 'sidecar_buf' to avoid
the copy, but that requires changes in Kudu code, so we can defer for later.
http://gerrit.cloudera.org:8080/#/c/10855/5/be/src/runtime/query-state.cc@304
PS5, Line 304: if (!profile_str.empty()) {
: unique_ptr sidecar_buf =
make_unique();
: sidecar_buf->assign_copy(profile_str);
: unique_ptr sidecar =
RpcSidecar::FromFaststring(move(sidecar_buf));
:
: int sidecar_idx;
: kudu::Status sidecar_status =
: rpc_controller.AddOutboundSidecar(move(sidecar),
_idx);
: CHECK(sidecar_status.ok())
: << FromKuduStatus(sidecar_status, "Failed to add
sidecar").GetDetail();
:
: DCHECK_EQ(report.instance_exec_status_size(), 1);
:
report.mutable_instance_exec_status(0)->set_thrift_profile_sidecar_idx(sidecar_idx);
: }
Can't we just do this once before the loop starts?
http://gerrit.cloudera.org:8080/#/c/10855/5/be/src/util/error-util.cc
File be/src/util/error-util.cc:
http://gerrit.cloudera.org:8080/#/c/10855/5/be/src/util/error-util.cc@144
PS5, Line 144: auto
auto&
http://gerrit.cloudera.org:8080/#/c/10855/5/be/src/util/error-util.cc@172
PS5, Line 172: auto
auto&
--
To view, visit http://gerrit.cloudera.org:8080/10855
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7638583b433dcac066b87198e448743d90415ebe
Gerrit-Change-Number: 10855
Gerrit-PatchSet: 5
Gerrit-Owner: Michael Ho
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Michael Ho
Gerrit-Reviewer: Sailesh Mukil
Gerrit-Reviewer: Todd Lipcon
Gerrit-Comment-Date: Sun, 05 Aug 2018 21:34:47 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7387: Set correct MIME type for JSON webpages
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/0 ) Change subject: IMPALA-7387: Set correct MIME type for JSON webpages .. Patch Set 11: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/196/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/0 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7ad94343151730851a4ad01a1f4b9326a36a37ea Gerrit-Change-Number: 0 Gerrit-PatchSet: 11 Gerrit-Owner: Bharath Vissapragada Gerrit-Reviewer: Anonymous Coward #168 Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Comment-Date: Sun, 05 Aug 2018 19:25:23 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7387: Set correct MIME type for JSON webpages
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/0 ) Change subject: IMPALA-7387: Set correct MIME type for JSON webpages .. Patch Set 11: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2923/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/0 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7ad94343151730851a4ad01a1f4b9326a36a37ea Gerrit-Change-Number: 0 Gerrit-PatchSet: 11 Gerrit-Owner: Bharath Vissapragada Gerrit-Reviewer: Anonymous Coward #168 Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Comment-Date: Sun, 05 Aug 2018 18:54:05 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7387: Set correct MIME type for JSON webpages
Bharath Vissapragada has posted comments on this change. ( http://gerrit.cloudera.org:8080/0 ) Change subject: IMPALA-7387: Set correct MIME type for JSON webpages .. Patch Set 11: (1 comment) http://gerrit.cloudera.org:8080/#/c/0/9/be/src/util/webserver.cc File be/src/util/webserver.cc: http://gerrit.cloudera.org:8080/#/c/0/9/be/src/util/webserver.cc@151 PS9, Line 151: \ > please : https://blog.csdn.net/gufachongyang02/article/details/21627539 Oh, I see what you are saying. You are right we still need the charset for text/html (and) text/plain content-types. Fixed that and tested it manually. -- To view, visit http://gerrit.cloudera.org:8080/0 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7ad94343151730851a4ad01a1f4b9326a36a37ea Gerrit-Change-Number: 0 Gerrit-PatchSet: 11 Gerrit-Owner: Bharath Vissapragada Gerrit-Reviewer: Anonymous Coward #168 Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Comment-Date: Sun, 05 Aug 2018 18:53:12 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7387: Set correct MIME type for JSON webpages
Hello Anonymous Coward #168, Philip Zeyliger, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/0 to look at the new patch set (#11). Change subject: IMPALA-7387: Set correct MIME type for JSON webpages .. IMPALA-7387: Set correct MIME type for JSON webpages Right now Impala sets "text/plain" as the MIME type for all non-HTML pages. As per RFC-4627[1], JSON's prescribed MIME type is "application/json". Additionally, this commit also sets UTF-8 charset for text/[html/plain] encodings. As per [2] application/json does not require it to be set. Testing: - Inspected HTTP headers manually in the browser. - Added tests. [1] http://www.ietf.org/rfc/rfc4627.txt [2] https://www.iana.org/assignments/media-types/application/json Change-Id: I7ad94343151730851a4ad01a1f4b9326a36a37ea --- M be/src/util/webserver.cc M be/src/util/webserver.h M tests/webserver/test_web_pages.py 3 files changed, 38 insertions(+), 5 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/10/0/11 -- To view, visit http://gerrit.cloudera.org:8080/0 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I7ad94343151730851a4ad01a1f4b9326a36a37ea Gerrit-Change-Number: 0 Gerrit-PatchSet: 11 Gerrit-Owner: Bharath Vissapragada Gerrit-Reviewer: Anonymous Coward #168 Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger
[Impala-ASF-CR] IMPALA-7163: Implement a state machine for the QueryState class
Michael Ho has posted comments on this change. ( http://gerrit.cloudera.org:8080/10813 ) Change subject: IMPALA-7163: Implement a state machine for the QueryState class .. Patch Set 13: (1 comment) http://gerrit.cloudera.org:8080/#/c/10813/13/be/src/runtime/fragment-instance-state.cc File be/src/runtime/fragment-instance-state.cc: http://gerrit.cloudera.org:8080/#/c/10813/13/be/src/runtime/fragment-instance-state.cc@107 PS13, Line 107: <= > Hmm, the enums are ordered incorrectly. I think the right thing to do would A simpler alternative to relying on this enum is to save a boolean above which is marked as true if we get past Prepare() successfully. This avoids the reliance that the code actually enforces the state transition based on the order in which it's defined in the enum. -- To view, visit http://gerrit.cloudera.org:8080/10813 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iec5670a7db83ecae4656d7bb2ea372d3767ba7fe Gerrit-Change-Number: 10813 Gerrit-PatchSet: 13 Gerrit-Owner: Sailesh Mukil Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sailesh Mukil Gerrit-Comment-Date: Sun, 05 Aug 2018 17:43:19 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-6789: disable impersonation in hive in minicluster
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/9 ) Change subject: IMPALA-6789: disable impersonation in hive in minicluster .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/195/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/9 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I39d8945e0fe90baf7e9e4b26eebab08d2058a14a Gerrit-Change-Number: 9 Gerrit-PatchSet: 4 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Sun, 05 Aug 2018 16:17:30 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6789: disable impersonation in hive in minicluster
Hello Bharath Vissapragada, Philip Zeyliger, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/9 to look at the new patch set (#4). Change subject: IMPALA-6789: disable impersonation in hive in minicluster .. IMPALA-6789: disable impersonation in hive in minicluster Due to bug of HADOOP-7050, users with dots in their username can't launch HiveServer2 in minicluster. To work arround this, we can set hive.server2.enable.doAs to false to disable impersonation in hive. Impala's authorization only depends on Sentry, so we can disable this without breaking any tests. This patch also quotes the group name in AuthorizationStmtTest#testShow to avoid syntax errors when group name contains dots. Test: * Build succeed with username quanlong.huang locally Change-Id: I39d8945e0fe90baf7e9e4b26eebab08d2058a14a --- M fe/src/test/java/org/apache/impala/analysis/AuthorizationStmtTest.java M fe/src/test/resources/mysql-hive-site.xml.template M fe/src/test/resources/postgresql-hive-site.xml.template 3 files changed, 19 insertions(+), 7 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/19/9/4 -- To view, visit http://gerrit.cloudera.org:8080/9 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I39d8945e0fe90baf7e9e4b26eebab08d2058a14a Gerrit-Change-Number: 9 Gerrit-PatchSet: 4 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Quanlong Huang
[Impala-ASF-CR] IMPALA-6789: disable impersonation in hive in minicluster
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/9 ) Change subject: IMPALA-6789: disable impersonation in hive in minicluster .. Patch Set 3: (1 comment) http://gerrit.cloudera.org:8080/#/c/9/1/fe/src/test/resources/postgresql-hive-site.xml.template File fe/src/test/resources/postgresql-hive-site.xml.template: http://gerrit.cloudera.org:8080/#/c/9/1/fe/src/test/resources/postgresql-hive-site.xml.template@198 PS1, Line 198: : : : > Done The build in our internal Jenkins failed due to I move this into the Kerberos section. Configs in the Kerberos section will be removed if IMPALA_KERBERIZE is not set, see https://github.com/apache/impala/blob/3.0.0/bin/create-test-configuration.sh#L127-L129 . However, we need to disable impersonation in HiveServer2 in all situations in order to launch it. By the way, is it possible to set up a Jenkins job using a username containing DOTs (e.g. "impala.dev")? So our builds won't be broken by tests added in future. -- To view, visit http://gerrit.cloudera.org:8080/9 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I39d8945e0fe90baf7e9e4b26eebab08d2058a14a Gerrit-Change-Number: 9 Gerrit-PatchSet: 3 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Sun, 05 Aug 2018 15:40:51 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7364: Bump RapidJSON version to 1.1.0
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/11124 ) Change subject: IMPALA-7364: Bump RapidJSON version to 1.1.0 .. Patch Set 3: Passed pre-review-test: https://jenkins.impala.io/job/pre-review-test/196/ -- To view, visit http://gerrit.cloudera.org:8080/11124 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I21353b0d769f81c13f506737e41fbac17655245c Gerrit-Change-Number: 11124 Gerrit-PatchSet: 3 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Sun, 05 Aug 2018 06:55:58 + Gerrit-HasComments: No
