[Impala-ASF-CR] IMPALA-7421. Static methods use wrong JNI call function

2018-08-10 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11181 )

Change subject: IMPALA-7421. Static methods use wrong JNI call function
..


Patch Set 2: Verified+1


--
To view, visit http://gerrit.cloudera.org:8080/11181
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If7cde6ca91613b63afe5307f4d819fb24cb17fd6
Gerrit-Change-Number: 11181
Gerrit-PatchSet: 2
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Bharath Vissapragada 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Comment-Date: Sat, 11 Aug 2018 03:51:12 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-7398: Add logged in user alias for effective user

2018-08-10 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11184 )

Change subject: IMPALA-7398: Add logged_in_user alias for effective_user
..


Patch Set 4:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2982/ 
DRY_RUN=false


--
To view, visit http://gerrit.cloudera.org:8080/11184
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id63f243e0fffbe2798f1f9dbc4cc3ebe9d9529a6
Gerrit-Change-Number: 11184
Gerrit-PatchSet: 4
Gerrit-Owner: Adam Holley 
Gerrit-Reviewer: Adam Holley 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Sat, 11 Aug 2018 02:46:51 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-7398: Add logged in user alias for effective user

2018-08-10 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11184 )

Change subject: IMPALA-7398: Add logged_in_user alias for effective_user
..


Patch Set 4: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/11184
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id63f243e0fffbe2798f1f9dbc4cc3ebe9d9529a6
Gerrit-Change-Number: 11184
Gerrit-PatchSet: 4
Gerrit-Owner: Adam Holley 
Gerrit-Reviewer: Adam Holley 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Sat, 11 Aug 2018 02:46:50 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-7423: NoSuchMethodError when starting Sentry

2018-08-10 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/11187 )

Change subject: IMPALA-7423: NoSuchMethodError when starting Sentry
..

IMPALA-7423: NoSuchMethodError when starting Sentry

The issue that is caused by the classpath ordering because of two
conflicting sentry-provider JARs:
- sentry-provider-db-2.0.0-cdh6.x-SNAPSHOT.jar
- sentry-provider-db-2.0.0-cdh6.x-SNAPSHOT-sh.jar (uses shaded Sentry
  Thrift)
This patch fixes the issue by removing sentry-provider-db:2.0.0:sh from
FE dependencies.

Testing:
- Ran a clean build and was able to start Sentry

Change-Id: Ie6cdfd184e06009a8d582499b2f6fd3011de6d5a
Reviewed-on: http://gerrit.cloudera.org:8080/11187
Reviewed-by: Impala Public Jenkins 
Tested-by: Impala Public Jenkins 
---
M fe/pom.xml
M fe/src/main/java/org/apache/impala/util/SentryUtil.java
2 files changed, 2 insertions(+), 17 deletions(-)

Approvals:
  Impala Public Jenkins: Looks good to me, approved; Verified

--
To view, visit http://gerrit.cloudera.org:8080/11187
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ie6cdfd184e06009a8d582499b2f6fd3011de6d5a
Gerrit-Change-Number: 11187
Gerrit-PatchSet: 3
Gerrit-Owner: Fredy Wijaya 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Philip Zeyliger 


[Impala-ASF-CR] IMPALA-7423: NoSuchMethodError when starting Sentry

2018-08-10 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11187 )

Change subject: IMPALA-7423: NoSuchMethodError when starting Sentry
..


Patch Set 2: Verified+1


--
To view, visit http://gerrit.cloudera.org:8080/11187
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie6cdfd184e06009a8d582499b2f6fd3011de6d5a
Gerrit-Change-Number: 11187
Gerrit-PatchSet: 2
Gerrit-Owner: Fredy Wijaya 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Comment-Date: Sat, 11 Aug 2018 01:53:50 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-7421. Static methods use wrong JNI call function

2018-08-10 Thread Bharath Vissapragada (Code Review)
Bharath Vissapragada has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11181 )

Change subject: IMPALA-7421. Static methods use wrong JNI call function
..


Patch Set 2:

(6 comments)

http://gerrit.cloudera.org:8080/#/c/11181/2//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/11181/2//COMMIT_MSG@12
PS2, Line 12: -Xcheck:jni
Wondering if we should enable it in the DEBUG build by default.


http://gerrit.cloudera.org:8080/#/c/11181/2/be/src/service/frontend.cc
File be/src/service/frontend.cc:

http://gerrit.cloudera.org:8080/#/c/11181/2/be/src/service/frontend.cc@229
PS2, Line 229: if (!err.empty()) {
 : return Status(err);
single line.


http://gerrit.cloudera.org:8080/#/c/11181/2/be/src/util/jni-util.h
File be/src/util/jni-util.h:

http://gerrit.cloudera.org:8080/#/c/11181/2/be/src/util/jni-util.h@250
PS2, Line 250:   env_(getJNIEnv()) {
I recently saw an Impalad crash because getJNIEnv() returned a nullptr and it 
was dereferenced. The issue turned out to be an OOM on the JVM and looks like a 
getJNIEnv() after that could return a NULL ptr. We should probably have 
non-debug assert for it.


http://gerrit.cloudera.org:8080/#/c/11181/2/be/src/util/jni-util.h@273
PS2, Line 273: ObjectToResult
WARN_UNUSED_RESULT for these...


http://gerrit.cloudera.org:8080/#/c/11181/2/be/src/util/jni-util.h@451
PS2, Line 451: ||
Make it more strict with ^?


http://gerrit.cloudera.org:8080/#/c/11181/2/be/src/util/jni-util.h@472
PS2, Line 472: jstring jstr = static_cast(obj);
 :   const char *str = env_->GetStringUTFChars(jstr, _copy);
 :   RETURN_ERROR_IF_EXC(env_);
 :   *result = str;
 :   env_->ReleaseStringUTFChars(jstr, str);
 :   RETURN_ERROR_IF_EXC(env_);
Use JniUtfCharGuard?



--
To view, visit http://gerrit.cloudera.org:8080/11181
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If7cde6ca91613b63afe5307f4d819fb24cb17fd6
Gerrit-Change-Number: 11181
Gerrit-PatchSet: 2
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Bharath Vissapragada 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Comment-Date: Sat, 11 Aug 2018 01:27:12 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-7421. Static methods use wrong JNI call function

2018-08-10 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11181 )

Change subject: IMPALA-7421. Static methods use wrong JNI call function
..


Patch Set 2:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2981/ 
DRY_RUN=true


--
To view, visit http://gerrit.cloudera.org:8080/11181
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If7cde6ca91613b63afe5307f4d819fb24cb17fd6
Gerrit-Change-Number: 11181
Gerrit-PatchSet: 2
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Bharath Vissapragada 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Comment-Date: Sat, 11 Aug 2018 00:37:35 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-7421. Static methods use wrong JNI call function

2018-08-10 Thread Todd Lipcon (Code Review)
Todd Lipcon has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11181 )

Change subject: IMPALA-7421. Static methods use wrong JNI call function
..


Patch Set 2:

Data loading failed due to an apparent random number collision. I filed 
https://issues.apache.org/jira/browse/MAPREDUCE-7128


--
To view, visit http://gerrit.cloudera.org:8080/11181
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If7cde6ca91613b63afe5307f4d819fb24cb17fd6
Gerrit-Change-Number: 11181
Gerrit-PatchSet: 2
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Bharath Vissapragada 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Comment-Date: Sat, 11 Aug 2018 00:38:02 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-7421. Static methods use wrong JNI call function

2018-08-10 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11181 )

Change subject: IMPALA-7421. Static methods use wrong JNI call function
..


Patch Set 2: Verified-1

Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/2980/


--
To view, visit http://gerrit.cloudera.org:8080/11181
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If7cde6ca91613b63afe5307f4d819fb24cb17fd6
Gerrit-Change-Number: 11181
Gerrit-PatchSet: 2
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Bharath Vissapragada 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Comment-Date: Sat, 11 Aug 2018 00:29:33 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-7408: add a flag to disable reading fs data from catalogd

2018-08-10 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11191 )

Change subject: IMPALA-7408: add a flag to disable reading fs data from catalogd
..


Patch Set 4:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/296/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/11191
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I15789fb489b285e2a6565025eb17c63cdc726354
Gerrit-Change-Number: 11191
Gerrit-PatchSet: 4
Gerrit-Owner: Vuk Ercegovac 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Sat, 11 Aug 2018 00:21:28 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-7347: Ignore numFilesErasureCoded in TestShowCreateTable

2018-08-10 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11192 )

Change subject: IMPALA-7347: Ignore numFilesErasureCoded in TestShowCreateTable
..


Patch Set 1:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/295/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/11192
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Icc8494fb91c4777cee662a97f750486aa8e79a8e
Gerrit-Change-Number: 11192
Gerrit-PatchSet: 1
Gerrit-Owner: Tianyi Wang 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Sat, 11 Aug 2018 00:02:04 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-7420: Different error messages for internal cancellation

2018-08-10 Thread Tim Armstrong (Code Review)
Tim Armstrong has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11180 )

Change subject: IMPALA-7420: Different error messages for internal cancellation
..


Patch Set 2: Code-Review-1

Talked offline with bikram, I think this might need some rethought - e.g. are 
we sure the error messages don't leak out, should we add some logging when they 
do, should put in the effort now to use a separate error code to enable dchecks 
etc.


-- 
To view, visit http://gerrit.cloudera.org:8080/11180
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Icfd69adfa4b06a68ebef9c350af66f18e0309b8d
Gerrit-Change-Number: 11180
Gerrit-PatchSet: 2
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Bikramjeet Vig 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Pooja Nilangekar 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Sat, 11 Aug 2018 00:01:59 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-7398: Add logged in user alias for effective user

2018-08-10 Thread Tim Armstrong (Code Review)
Tim Armstrong has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11184 )

Change subject: IMPALA-7398: Add logged_in_user alias for effective_user
..


Patch Set 3: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/11184
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id63f243e0fffbe2798f1f9dbc4cc3ebe9d9529a6
Gerrit-Change-Number: 11184
Gerrit-PatchSet: 3
Gerrit-Owner: Adam Holley 
Gerrit-Reviewer: Adam Holley 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Fri, 10 Aug 2018 23:56:54 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-7408: add a flag to disable reading fs data from catalogd

2018-08-10 Thread Vuk Ercegovac (Code Review)
Hello Impala Public Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/11191

to look at the new patch set (#4).

Change subject: IMPALA-7408: add a flag to disable reading fs data from catalogd
..

IMPALA-7408: add a flag to disable reading fs data from catalogd

Add the flag: --disable_catalog_data_ops that skips loading files
from the file-system from catalogd. The flag is by default false
and its hidden. Its intent is to avoid time-consuming accesses to
the file-system when debugging metadata issues and the file-system
contents are not available. For example, a recent ~18 GB catalog
takes 10 hours to load without the flag set vs. 1 hour to load with
the flag. The extra time comes from accessing the file-system, failing,
and logging exceptions.

This flag specifically disables copying jars from the fs when loading
Java functions and it skips loading avro schema files. Additional cases
can be added under this flag if more are needed.

Testing:
- manually confirmed that jars and avro schema files are skipped.
- added a test to check the same behavior in a custom cluster test.
- ran core tests.

Change-Id: I15789fb489b285e2a6565025eb17c63cdc726354
---
M be/src/common/global-flags.cc
M be/src/util/backend-gflag-util.cc
M common/thrift/BackendGflags.thrift
M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java
M fe/src/main/java/org/apache/impala/service/BackendConfig.java
M fe/src/main/java/org/apache/impala/util/AvroSchemaUtils.java
M tests/common/custom_cluster_test_suite.py
A tests/custom_cluster/test_disable_catalog_data_ops.py
8 files changed, 88 insertions(+), 10 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/91/11191/4
--
To view, visit http://gerrit.cloudera.org:8080/11191
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I15789fb489b285e2a6565025eb17c63cdc726354
Gerrit-Change-Number: 11191
Gerrit-PatchSet: 4
Gerrit-Owner: Vuk Ercegovac 
Gerrit-Reviewer: Impala Public Jenkins 


[Impala-ASF-CR] IMPALA-7347: Ignore numFilesErasureCoded in TestShowCreateTable

2018-08-10 Thread Tianyi Wang (Code Review)
Tianyi Wang has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/11192


Change subject: IMPALA-7347: Ignore numFilesErasureCoded in TestShowCreateTable
..

IMPALA-7347: Ignore numFilesErasureCoded in TestShowCreateTable

This table properties only exist for HDFS tables. To get the test work
on local tables, it needs to be ignored.

Change-Id: Icc8494fb91c4777cee662a97f750486aa8e79a8e
---
M testdata/workloads/functional-query/queries/QueryTest/show-create-table.test
M tests/metadata/test_show_create_table.py
2 files changed, 1 insertion(+), 2 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/92/11192/1
--
To view, visit http://gerrit.cloudera.org:8080/11192
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Icc8494fb91c4777cee662a97f750486aa8e79a8e
Gerrit-Change-Number: 11192
Gerrit-PatchSet: 1
Gerrit-Owner: Tianyi Wang 


[Impala-ASF-CR] IMPALA-7335: Add logs HdfsScanNode to debug the issue

2018-08-10 Thread Bikramjeet Vig (Code Review)
Bikramjeet Vig has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11174 )

Change subject: IMPALA-7335: Add logs HdfsScanNode to debug the issue
..


Patch Set 5: Code-Review+1

(2 comments)

Looks good to me! @Tim can you +2 if you dont see any major concerns?

http://gerrit.cloudera.org:8080/#/c/11174/5//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/11174/5//COMMIT_MSG@7
PS5, Line 7: IMPALA-7335: Add logs HdfsScanNode to debug the issue
   :
   : This log would help determine the condition when the test failures
   : occur.
could you add some more details to the log message. See 
https://gerrit.cloudera.org/#/c/10851/ for some reference


http://gerrit.cloudera.org:8080/#/c/11174/5/be/src/exec/hdfs-scan-node.cc
File be/src/exec/hdfs-scan-node.cc:

http://gerrit.cloudera.org:8080/#/c/11174/5/be/src/exec/hdfs-scan-node.cc@333
PS5, Line 333: #ifndef NDEBUG
 :   VLOG_QUERY << "Non-ok status returned by ProcessSplit = " 
<< status.msg().msg()
 :  << " for Scan node (id=" << id()
 :  << ", status_ = " << (status_.ok() ? "ok" : 
status_.msg().msg())
 :  << ", done_ = " << (done_ ? "1" : "0") << ")";
 : #endif
if you plan to remove this after you figure out the cause, you can add a TODO 
here, with the JIRA ID so it doesnt get unnoticed.



--
To view, visit http://gerrit.cloudera.org:8080/11174
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I68698c90031edc6ee8c31e9ce3d52dade9d8f6f1
Gerrit-Change-Number: 11174
Gerrit-PatchSet: 5
Gerrit-Owner: Pooja Nilangekar 
Gerrit-Reviewer: Bikramjeet Vig 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Pooja Nilangekar 
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Fri, 10 Aug 2018 23:07:30 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-7421. Static methods use wrong JNI call function

2018-08-10 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11181 )

Change subject: IMPALA-7421. Static methods use wrong JNI call function
..


Patch Set 2:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2980/ 
DRY_RUN=true


--
To view, visit http://gerrit.cloudera.org:8080/11181
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If7cde6ca91613b63afe5307f4d819fb24cb17fd6
Gerrit-Change-Number: 11181
Gerrit-PatchSet: 2
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Bharath Vissapragada 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Comment-Date: Fri, 10 Aug 2018 22:55:13 +
Gerrit-HasComments: No


[Impala-ASF-CR] WIP IMPALA-7417: Speed up sub-second unix time->TimestampValue conversions

2018-08-10 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11183 )

Change subject: WIP IMPALA-7417: Speed up sub-second unix time->TimestampValue 
conversions
..


Patch Set 3: Verified-1

Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/2978/


--
To view, visit http://gerrit.cloudera.org:8080/11183
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I572b5876b979ddae58165bd40d5b008ce9d7a4aa
Gerrit-Change-Number: 11183
Gerrit-PatchSet: 3
Gerrit-Owner: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Fri, 10 Aug 2018 22:46:00 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-7423: NoSuchMethodError when starting Sentry

2018-08-10 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11187 )

Change subject: IMPALA-7423: NoSuchMethodError when starting Sentry
..


Patch Set 2:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2979/ 
DRY_RUN=false


--
To view, visit http://gerrit.cloudera.org:8080/11187
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie6cdfd184e06009a8d582499b2f6fd3011de6d5a
Gerrit-Change-Number: 11187
Gerrit-PatchSet: 2
Gerrit-Owner: Fredy Wijaya 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Comment-Date: Fri, 10 Aug 2018 22:40:11 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-7423: NoSuchMethodError when starting Sentry

2018-08-10 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11187 )

Change subject: IMPALA-7423: NoSuchMethodError when starting Sentry
..


Patch Set 2: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/11187
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie6cdfd184e06009a8d582499b2f6fd3011de6d5a
Gerrit-Change-Number: 11187
Gerrit-PatchSet: 2
Gerrit-Owner: Fredy Wijaya 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Comment-Date: Fri, 10 Aug 2018 22:40:09 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-7421. Static methods use wrong JNI call function

2018-08-10 Thread Csaba Ringhofer (Code Review)
Csaba Ringhofer has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11181 )

Change subject: IMPALA-7421. Static methods use wrong JNI call function
..


Patch Set 2: Code-Review+1


--
To view, visit http://gerrit.cloudera.org:8080/11181
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If7cde6ca91613b63afe5307f4d819fb24cb17fd6
Gerrit-Change-Number: 11181
Gerrit-PatchSet: 2
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Bharath Vissapragada 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Comment-Date: Fri, 10 Aug 2018 22:32:01 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-7408: add a flag to disable reading fs data from catalogd

2018-08-10 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11191 )

Change subject: IMPALA-7408: add a flag to disable reading fs data from catalogd
..


Patch Set 3:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/293/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/11191
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I15789fb489b285e2a6565025eb17c63cdc726354
Gerrit-Change-Number: 11191
Gerrit-PatchSet: 3
Gerrit-Owner: Vuk Ercegovac 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Fri, 10 Aug 2018 22:31:33 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-7421. Static methods use wrong JNI call function

2018-08-10 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11181 )

Change subject: IMPALA-7421. Static methods use wrong JNI call function
..


Patch Set 2:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/294/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/11181
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If7cde6ca91613b63afe5307f4d819fb24cb17fd6
Gerrit-Change-Number: 11181
Gerrit-PatchSet: 2
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Bharath Vissapragada 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Comment-Date: Fri, 10 Aug 2018 22:30:57 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-7408: add a flag to disable reading fs data from catalogd

2018-08-10 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11191 )

Change subject: IMPALA-7408: add a flag to disable reading fs data from catalogd
..


Patch Set 2:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/292/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/11191
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I15789fb489b285e2a6565025eb17c63cdc726354
Gerrit-Change-Number: 11191
Gerrit-PatchSet: 2
Gerrit-Owner: Vuk Ercegovac 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Fri, 10 Aug 2018 22:27:16 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-7398: Add logged in user alias for effective user

2018-08-10 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11184 )

Change subject: IMPALA-7398: Add logged_in_user alias for effective_user
..


Patch Set 3:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/291/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/11184
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id63f243e0fffbe2798f1f9dbc4cc3ebe9d9529a6
Gerrit-Change-Number: 11184
Gerrit-PatchSet: 3
Gerrit-Owner: Adam Holley 
Gerrit-Reviewer: Adam Holley 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Fri, 10 Aug 2018 22:09:45 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-7408: add a flag to disable reading fs data from catalogd

2018-08-10 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11191 )

Change subject: IMPALA-7408: add a flag to disable reading fs data from catalogd
..


Patch Set 1:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/290/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/11191
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I15789fb489b285e2a6565025eb17c63cdc726354
Gerrit-Change-Number: 11191
Gerrit-PatchSet: 1
Gerrit-Owner: Vuk Ercegovac 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Fri, 10 Aug 2018 22:07:17 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-7421. Static methods use wrong JNI call function

2018-08-10 Thread Todd Lipcon (Code Review)
Todd Lipcon has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11181 )

Change subject: IMPALA-7421. Static methods use wrong JNI call function
..


Patch Set 2:

(6 comments)

http://gerrit.cloudera.org:8080/#/c/11181/1/be/src/util/jni-util.h
File be/src/util/jni-util.h:

http://gerrit.cloudera.org:8080/#/c/11181/1/be/src/util/jni-util.h@219
PS1, Line 219:  Non-st
> How about an even simpler one? I can make static methods JniCall::static(jc
did this. Also started working on a fancier template variety that verifies 
argument types match the java method signature, etc, but will do that in some 
later patch


http://gerrit.cloudera.org:8080/#/c/11181/1/be/src/util/jni-util.h@227
PS1, Line 227:static JniCall static_method(jclass clazz, jmethodID method) 
WARN_UNUSED_RESULT {
> WARN_UNUSED_RESULT could be added here and in other "fluent" functions.
Done


http://gerrit.cloudera.org:8080/#/c/11181/1/be/src/util/jni-util.h@234
PS1, Line 234:
> The same thing is checked twice here.
Done


http://gerrit.cloudera.org:8080/#/c/11181/1/be/src/util/jni-util.h@410
PS1, Line 410: return JniCall::instance_method(obj, 
method).with_thrift_arg(arg).Call();
> line too long (103 > 90)
Done


http://gerrit.cloudera.org:8080/#/c/11181/1/be/src/util/logging-support.cc
File be/src/util/logging-support.cc:

http://gerrit.cloudera.org:8080/#/c/11181/1/be/src/util/logging-support.cc@139
PS1, Line 139: }
> As this is already modified the RETURN_IF_ERROR could be replaced with simp
Done


http://gerrit.cloudera.org:8080/#/c/11181/1/be/src/util/logging-support.cc@144
PS1, Line 144:
> Same as line 139.
Done



--
To view, visit http://gerrit.cloudera.org:8080/11181
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If7cde6ca91613b63afe5307f4d819fb24cb17fd6
Gerrit-Change-Number: 11181
Gerrit-PatchSet: 2
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Bharath Vissapragada 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Comment-Date: Fri, 10 Aug 2018 22:06:12 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-7421. Static methods use wrong JNI call function

2018-08-10 Thread Todd Lipcon (Code Review)
Hello Bharath Vissapragada, Philip Zeyliger, Csaba Ringhofer, Impala Public 
Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/11181

to look at the new patch set (#2).

Change subject: IMPALA-7421. Static methods use wrong JNI call function
..

IMPALA-7421. Static methods use wrong JNI call function

Various places throughout the code were using the JNI CallObjectMethod
function when in fact they should have been using
CallStaticObjectMethod. It appears that this doesn't cause crashes in
release builds, but if running with -Xcheck:jni, it causes an assertion
failure:

   guarantee(method->size_of_parameters() == size_of_parameters())
  failed: wrong no. of arguments pushed

This patch cleans up JniUtil a bit to add a "fluent" style utility for
calling JNI functions. This calling style should make the above mistake
more obvious because it forces the caller to write 'on_class(...)' for
static methods and 'on_instance(...)' for instance methods.

Change-Id: If7cde6ca91613b63afe5307f4d819fb24cb17fd6
---
M be/src/common/init.cc
M be/src/common/status.h
M be/src/service/fe-support.h
M be/src/service/frontend.cc
M be/src/util/backend-gflag-util.h
M be/src/util/jni-util.cc
M be/src/util/jni-util.h
M be/src/util/logging-support.cc
M be/src/util/logging-support.h
M be/src/util/zip-util.cc
M be/src/util/zip-util.h
11 files changed, 169 insertions(+), 89 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/81/11181/2
--
To view, visit http://gerrit.cloudera.org:8080/11181
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: If7cde6ca91613b63afe5307f4d819fb24cb17fd6
Gerrit-Change-Number: 11181
Gerrit-PatchSet: 2
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Bharath Vissapragada 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Todd Lipcon 


[Impala-ASF-CR] IMPALA-7408: add a flag to disable reading fs data from catalogd

2018-08-10 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11191 )

Change subject: IMPALA-7408: add a flag to disable reading fs data from catalogd
..


Patch Set 3:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/11191/3/tests/custom_cluster/test_disable_catalog_data_ops.py
File tests/custom_cluster/test_disable_catalog_data_ops.py:

http://gerrit.cloudera.org:8080/#/c/11191/3/tests/custom_cluster/test_disable_catalog_data_ops.py@22
PS3, Line 22: class TestDisableCatalogDataOps(CustomClusterTestSuite):
flake8: E302 expected 2 blank lines, found 1


http://gerrit.cloudera.org:8080/#/c/11191/3/tests/custom_cluster/test_disable_catalog_data_ops.py@26
PS3, Line 26: #
flake8: E303 too many blank lines (2)



--
To view, visit http://gerrit.cloudera.org:8080/11191
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I15789fb489b285e2a6565025eb17c63cdc726354
Gerrit-Change-Number: 11191
Gerrit-PatchSet: 3
Gerrit-Owner: Vuk Ercegovac 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Fri, 10 Aug 2018 22:01:48 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-7408: add a flag to disable reading fs data from catalogd

2018-08-10 Thread Vuk Ercegovac (Code Review)
Hello Impala Public Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/11191

to look at the new patch set (#3).

Change subject: IMPALA-7408: add a flag to disable reading fs data from catalogd
..

IMPALA-7408: add a flag to disable reading fs data from catalogd

Add the flag: --disable_catalog_data_ops that skips loading files
from the file-system from catalogd. The flag is by default false
and its hidden. Its intent is to avoid time-consuming accesses to
the file-system when debugging metadata issues and the file-system
contents are not available. For example, a recent ~18 GB catalog
takes 10 hours to load without the flag set vs. 1 hour to load with
the flag. The extra time comes from accessing the file-system, failing,
and logging exceptions.

This flag specifically disables copying jars from the fs when loading
Java functions and it skips loading avro schema files. Additional cases
can be added under this flag if more are needed.

Testing:
- manually confirmed that jars and avro schema files are skipped.
- added a test to check the same behavior in a custom cluster test.
- ran core tests.

Change-Id: I15789fb489b285e2a6565025eb17c63cdc726354
---
M be/src/common/global-flags.cc
M be/src/util/backend-gflag-util.cc
M common/thrift/BackendGflags.thrift
M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java
M fe/src/main/java/org/apache/impala/service/BackendConfig.java
M fe/src/main/java/org/apache/impala/util/AvroSchemaUtils.java
M tests/common/custom_cluster_test_suite.py
A tests/custom_cluster/test_disable_catalog_data_ops.py
8 files changed, 88 insertions(+), 10 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/91/11191/3
--
To view, visit http://gerrit.cloudera.org:8080/11191
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I15789fb489b285e2a6565025eb17c63cdc726354
Gerrit-Change-Number: 11191
Gerrit-PatchSet: 3
Gerrit-Owner: Vuk Ercegovac 
Gerrit-Reviewer: Impala Public Jenkins 


[Impala-ASF-CR] IMPALA-7408: add a flag to disable reading fs data from catalogd

2018-08-10 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11191 )

Change subject: IMPALA-7408: add a flag to disable reading fs data from catalogd
..


Patch Set 2:

(4 comments)

http://gerrit.cloudera.org:8080/#/c/11191/2/tests/common/custom_cluster_test_suite.py
File tests/common/custom_cluster_test_suite.py:

http://gerrit.cloudera.org:8080/#/c/11191/2/tests/common/custom_cluster_test_suite.py@203
PS2, Line 203: +
flake8: W503 line break before binary operator


http://gerrit.cloudera.org:8080/#/c/11191/2/tests/custom_cluster/test_disable_catalog_data_ops.py
File tests/custom_cluster/test_disable_catalog_data_ops.py:

http://gerrit.cloudera.org:8080/#/c/11191/2/tests/custom_cluster/test_disable_catalog_data_ops.py@22
PS2, Line 22: class TestDisableCatalogDataOps(CustomClusterTestSuite):
flake8: E302 expected 2 blank lines, found 1


http://gerrit.cloudera.org:8080/#/c/11191/2/tests/custom_cluster/test_disable_catalog_data_ops.py@26
PS2, Line 26:
flake8: W291 trailing whitespace


http://gerrit.cloudera.org:8080/#/c/11191/2/tests/custom_cluster/test_disable_catalog_data_ops.py@26
PS2, Line 26:   # and adding specifically one java-udf and one avro table.
line has trailing whitespace



--
To view, visit http://gerrit.cloudera.org:8080/11191
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I15789fb489b285e2a6565025eb17c63cdc726354
Gerrit-Change-Number: 11191
Gerrit-PatchSet: 2
Gerrit-Owner: Vuk Ercegovac 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Fri, 10 Aug 2018 21:52:08 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-7408: add a flag to disable reading fs data from catalogd

2018-08-10 Thread Vuk Ercegovac (Code Review)
Hello Impala Public Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/11191

to look at the new patch set (#2).

Change subject: IMPALA-7408: add a flag to disable reading fs data from catalogd
..

IMPALA-7408: add a flag to disable reading fs data from catalogd

Add the flag: --disable_catalog_data_ops that skips loading files
from the file-system from catalogd. The flag is by default false
and its hidden. Its intent is to avoid time-consuming accesses to
the file-system when debugging metadata issues and the file-system
contents are not available. For example, a recent ~18 GB catalog
takes 10 hours to load without the flag set vs. 1 hour to load with
the flag. The extra time comes from accessing the file-system, failing,
and logging exceptions.

This flag specifically disables copying jars from the fs when loading
Java functions and it skips loading avro schema files. Additional cases
can be added under this flag if more are needed.

Testing:
- manually confirmed that jars and avro schema files are skipped.
- added a test to check the same behavior in a custom cluster test.
- ran core tests.

Change-Id: I15789fb489b285e2a6565025eb17c63cdc726354
---
M be/src/common/global-flags.cc
M be/src/util/backend-gflag-util.cc
M common/thrift/BackendGflags.thrift
M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java
M fe/src/main/java/org/apache/impala/service/BackendConfig.java
M fe/src/main/java/org/apache/impala/util/AvroSchemaUtils.java
M tests/common/custom_cluster_test_suite.py
A tests/custom_cluster/test_disable_catalog_data_ops.py
8 files changed, 85 insertions(+), 8 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/91/11191/2
--
To view, visit http://gerrit.cloudera.org:8080/11191
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I15789fb489b285e2a6565025eb17c63cdc726354
Gerrit-Change-Number: 11191
Gerrit-PatchSet: 2
Gerrit-Owner: Vuk Ercegovac 
Gerrit-Reviewer: Impala Public Jenkins 


[Impala-ASF-CR] IMPALA-3825: Distribute Runtime Filtering Aggregation

2018-08-10 Thread Michael Ho (Code Review)
Michael Ho has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11055 )

Change subject: IMPALA-3825: Distribute Runtime Filtering Aggregation
..


Patch Set 4:

(1 comment)

May help to add a unit test to exercise retrying logic when sending a filter to 
the aggregator.

http://gerrit.cloudera.org:8080/#/c/11055/4/common/thrift/ImpalaInternalService.thrift
File common/thrift/ImpalaInternalService.thrift:

http://gerrit.cloudera.org:8080/#/c/11055/4/common/thrift/ImpalaInternalService.thrift@878
PS4, Line 878: optional map> backend_list
May help to add a comment on what this is for.



--
To view, visit http://gerrit.cloudera.org:8080/11055
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I94e183a0353fc46f8d3eccae029d2d52c5cdc40c
Gerrit-Change-Number: 11055
Gerrit-PatchSet: 4
Gerrit-Owner: Rahul Shivu Mahadev 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Rahul Shivu Mahadev 
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Comment-Date: Fri, 10 Aug 2018 21:50:22 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-7335: Add logs HdfsScanNode to debug the issue

2018-08-10 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11174 )

Change subject: IMPALA-7335: Add logs HdfsScanNode to debug the issue
..


Patch Set 5:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/289/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/11174
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I68698c90031edc6ee8c31e9ce3d52dade9d8f6f1
Gerrit-Change-Number: 11174
Gerrit-PatchSet: 5
Gerrit-Owner: Pooja Nilangekar 
Gerrit-Reviewer: Bikramjeet Vig 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Pooja Nilangekar 
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Fri, 10 Aug 2018 21:46:32 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-7408: add a flag to disable reading fs data from catalogd

2018-08-10 Thread Vuk Ercegovac (Code Review)
Vuk Ercegovac has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/11191


Change subject: IMPALA-7408: add a flag to disable reading fs data from catalogd
..

IMPALA-7408: add a flag to disable reading fs data from catalogd

Add the flag: --disable_catalog_data_ops that skips loading files
from the file-system from catalogd. The flag is by default false
and its hidden. Its intent is to avoid time-consuming accesses to
the file-system when debugging metadata issues and the file-system
contents are not available. For example, a recent ~18 GB catalog
takes 10 hours to load without the flag set vs. 1 hour to load with
the flag. The extra time comes from accessing the file-system, failing,
and logging exceptions.

This flag specifically disables copying jars from the fs when loading
Java functions and it skips loading avro schema files. Additional cases
can be added under this flag if more are needed.

Testing:
- manually confirmed that jars and avro schema files are skipped.
- added a test to check the same behavior in a custom cluster test.
- ran core tests.

Change-Id: I15789fb489b285e2a6565025eb17c63cdc726354
---
M be/src/common/global-flags.cc
M be/src/util/backend-gflag-util.cc
M common/thrift/BackendGflags.thrift
M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java
M fe/src/main/java/org/apache/impala/service/BackendConfig.java
M fe/src/main/java/org/apache/impala/util/AvroSchemaUtils.java
M tests/common/custom_cluster_test_suite.py
A tests/custom_cluster/test_disable_catalog_data_ops.py
8 files changed, 83 insertions(+), 8 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/91/11191/1
--
To view, visit http://gerrit.cloudera.org:8080/11191
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I15789fb489b285e2a6565025eb17c63cdc726354
Gerrit-Change-Number: 11191
Gerrit-PatchSet: 1
Gerrit-Owner: Vuk Ercegovac 


[Impala-ASF-CR] IMPALA-7335: Add logs HdfsScanNode to debug the issue

2018-08-10 Thread Pooja Nilangekar (Code Review)
Pooja Nilangekar has uploaded a new patch set (#5). ( 
http://gerrit.cloudera.org:8080/11174 )

Change subject: IMPALA-7335: Add logs HdfsScanNode to debug the issue
..

IMPALA-7335: Add logs HdfsScanNode to debug the issue

This log would help determine the condition when the test failures
occur.

Change-Id: I68698c90031edc6ee8c31e9ce3d52dade9d8f6f1
---
M be/src/exec/hdfs-scan-node.cc
1 file changed, 6 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/74/11174/5
--
To view, visit http://gerrit.cloudera.org:8080/11174
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I68698c90031edc6ee8c31e9ce3d52dade9d8f6f1
Gerrit-Change-Number: 11174
Gerrit-PatchSet: 5
Gerrit-Owner: Pooja Nilangekar 
Gerrit-Reviewer: Bikramjeet Vig 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Pooja Nilangekar 
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Reviewer: Tim Armstrong 


[Impala-ASF-CR] IMPALA-6709: Simplify tests that copy local files to tables

2018-08-10 Thread Csaba Ringhofer (Code Review)
Csaba Ringhofer has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11127 )

Change subject: IMPALA-6709: Simplify tests that copy local files to tables
..


Patch Set 2:

This was missing from my last reply: renaming common/ classes to not start with 
'test' is important IMO, other comments are optional.


--
To view, visit http://gerrit.cloudera.org:8080/11127
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie00a4561825facf8abe2e8e74a6b6e93194f416f
Gerrit-Change-Number: 11127
Gerrit-PatchSet: 2
Gerrit-Owner: Zoltan Borok-Nagy 
Gerrit-Reviewer: Andrew Sherman 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Reviewer: Zoltan Borok-Nagy 
Gerrit-Comment-Date: Fri, 10 Aug 2018 20:36:48 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-7423: NoSuchMethodError when starting Sentry

2018-08-10 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11187 )

Change subject: IMPALA-7423: NoSuchMethodError when starting Sentry
..


Patch Set 1:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/288/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/11187
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie6cdfd184e06009a8d582499b2f6fd3011de6d5a
Gerrit-Change-Number: 11187
Gerrit-PatchSet: 1
Gerrit-Owner: Fredy Wijaya 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Comment-Date: Fri, 10 Aug 2018 20:35:32 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-6709: Simplify tests that copy local files to tables

2018-08-10 Thread Csaba Ringhofer (Code Review)
Csaba Ringhofer has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11127 )

Change subject: IMPALA-6709: Simplify tests that copy local files to tables
..


Patch Set 2:

(3 comments)

http://gerrit.cloudera.org:8080/#/c/11127/2/tests/common/test_files.py
File tests/common/test_files.py:

http://gerrit.cloudera.org:8080/#/c/11127/2/tests/common/test_files.py@1
PS2, Line 1: # Licensed to the Apache Software Foundation (ASF) under one
About the name of the file: test_* is a conventional way to name files that 
actually contain tests. This is a file with utility functions, so it would be 
better to give it a different name.


http://gerrit.cloudera.org:8080/#/c/11127/2/tests/query_test/test_parquet_stats.py
File tests/query_test/test_parquet_stats.py:

http://gerrit.cloudera.org:8080/#/c/11127/2/tests/query_test/test_parquet_stats.py@64
PS2, Line 64: /
I would prefer to start these without '/'.

+ $IMPALA_HOME/testdata/data could be assumed as base similarly to 
create_table_from_parquet - the minority of files that do not reside there 
could use .. in their address.


http://gerrit.cloudera.org:8080/#/c/11127/2/tests/query_test/test_scanners.py
File tests/query_test/test_scanners.py:

http://gerrit.cloudera.org:8080/#/c/11127/2/tests/query_test/test_scanners.py@307
PS2, Line 307: self.clien
Just an idea to make these shorter: this functions could be ImpalaTestSuite 
member functions, so this argument could be moved to the front as "self.".



--
To view, visit http://gerrit.cloudera.org:8080/11127
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie00a4561825facf8abe2e8e74a6b6e93194f416f
Gerrit-Change-Number: 11127
Gerrit-PatchSet: 2
Gerrit-Owner: Zoltan Borok-Nagy 
Gerrit-Reviewer: Andrew Sherman 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Reviewer: Zoltan Borok-Nagy 
Gerrit-Comment-Date: Fri, 10 Aug 2018 20:35:01 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-7423: NoSuchMethodError when starting Sentry

2018-08-10 Thread Fredy Wijaya (Code Review)
Fredy Wijaya has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/11187


Change subject: IMPALA-7423: NoSuchMethodError when starting Sentry
..

IMPALA-7423: NoSuchMethodError when starting Sentry

The issue that is caused by the classpath ordering because of two
conflicting sentry-provider JARs:
- sentry-provider-db-2.0.0-cdh6.x-SNAPSHOT.jar
- sentry-provider-db-2.0.0-cdh6.x-SNAPSHOT-sh.jar (uses shaded Sentry
  Thrift)
This patch fixes the issue by removing sentry-provider-db:2.0.0:sh from
FE dependencies.

Testing:
- Ran a clean build and was able to start Sentry

Change-Id: Ie6cdfd184e06009a8d582499b2f6fd3011de6d5a
---
M fe/pom.xml
M fe/src/main/java/org/apache/impala/util/SentryUtil.java
2 files changed, 2 insertions(+), 17 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/87/11187/1
--
To view, visit http://gerrit.cloudera.org:8080/11187
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ie6cdfd184e06009a8d582499b2f6fd3011de6d5a
Gerrit-Change-Number: 11187
Gerrit-PatchSet: 1
Gerrit-Owner: Fredy Wijaya 


[Impala-ASF-CR] IMPALA-7415: Fix flakiness in test multiline queries in history

2018-08-10 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/11175 )

Change subject: IMPALA-7415: Fix flakiness in test_multiline_queries_in_history
..

IMPALA-7415: Fix flakiness in test_multiline_queries_in_history

This fixes a flakiness in test_multiline_queries_in_history wherein a
part of the shell prompt would be absorbed in a previous regex search
that would ultimately result in the failure of the subsequent regex
search that looks for the prompt.

Also fixed a few formatting issues flagged by flake8.

Change-Id: If7474f832a88bc29b321f21b050c9665294e63d5
Reviewed-on: http://gerrit.cloudera.org:8080/11175
Reviewed-by: Impala Public Jenkins 
Tested-by: Impala Public Jenkins 
---
M tests/shell/test_shell_interactive.py
1 file changed, 12 insertions(+), 9 deletions(-)

Approvals:
  Impala Public Jenkins: Looks good to me, approved; Verified

--
To view, visit http://gerrit.cloudera.org:8080/11175
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: If7474f832a88bc29b321f21b050c9665294e63d5
Gerrit-Change-Number: 11175
Gerrit-PatchSet: 3
Gerrit-Owner: Bikramjeet Vig 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tim Armstrong 


[Impala-ASF-CR] IMPALA-7415: Fix flakiness in test multiline queries in history

2018-08-10 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11175 )

Change subject: IMPALA-7415: Fix flakiness in test_multiline_queries_in_history
..


Patch Set 2: Verified+1


--
To view, visit http://gerrit.cloudera.org:8080/11175
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If7474f832a88bc29b321f21b050c9665294e63d5
Gerrit-Change-Number: 11175
Gerrit-PatchSet: 2
Gerrit-Owner: Bikramjeet Vig 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Fri, 10 Aug 2018 19:50:09 +
Gerrit-HasComments: No


[Impala-ASF-CR] WIP IMPALA-7417: Speed up sub-second unix time->TimestampValue conversions

2018-08-10 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11183 )

Change subject: WIP IMPALA-7417: Speed up sub-second unix time->TimestampValue 
conversions
..


Patch Set 3:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2978/ 
DRY_RUN=true


--
To view, visit http://gerrit.cloudera.org:8080/11183
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I572b5876b979ddae58165bd40d5b008ce9d7a4aa
Gerrit-Change-Number: 11183
Gerrit-PatchSet: 3
Gerrit-Owner: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Fri, 10 Aug 2018 19:47:54 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-7231: group plan nodes into pipelines

2018-08-10 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/10848 )

Change subject: IMPALA-7231: group plan nodes into pipelines
..

IMPALA-7231: group plan nodes into pipelines

This adds some informational output to explain plans and
sends the information to the backend.

The idea is that this will make it easier to explain how Impala's
pipelined execution works and also enable future work on profile
analysis that can more intelligently group plan nodes.

Tests:
* Updated planner tests to include new output.

Change-Id: I1d10eb14d997242f445e5c5fc5362d5410370721
Reviewed-on: http://gerrit.cloudera.org:8080/10848
Reviewed-by: Impala Public Jenkins 
Tested-by: Impala Public Jenkins 
---
M common/thrift/PlanNodes.thrift
M fe/src/main/java/org/apache/impala/planner/JoinNode.java
A fe/src/main/java/org/apache/impala/planner/PipelineMembership.java
M fe/src/main/java/org/apache/impala/planner/PlanFragment.java
M fe/src/main/java/org/apache/impala/planner/PlanNode.java
M fe/src/main/java/org/apache/impala/planner/Planner.java
M fe/src/main/java/org/apache/impala/planner/SubplanNode.java
M fe/src/main/java/org/apache/impala/planner/UnionNode.java
M 
testdata/workloads/functional-planner/queries/PlannerTest/constant-folding.test
M 
testdata/workloads/functional-planner/queries/PlannerTest/fk-pk-join-detection.test
M 
testdata/workloads/functional-planner/queries/PlannerTest/kudu-selectivity.test
M testdata/workloads/functional-planner/queries/PlannerTest/max-row-size.test
M 
testdata/workloads/functional-planner/queries/PlannerTest/min-max-runtime-filters.test
M 
testdata/workloads/functional-planner/queries/PlannerTest/mt-dop-validation.test
M 
testdata/workloads/functional-planner/queries/PlannerTest/parquet-filtering-disabled.test
M 
testdata/workloads/functional-planner/queries/PlannerTest/parquet-filtering.test
M 
testdata/workloads/functional-planner/queries/PlannerTest/partition-pruning.test
M 
testdata/workloads/functional-planner/queries/PlannerTest/resource-requirements.test
M 
testdata/workloads/functional-planner/queries/PlannerTest/sort-expr-materialization.test
M 
testdata/workloads/functional-planner/queries/PlannerTest/spillable-buffer-sizing.test
M testdata/workloads/functional-planner/queries/PlannerTest/tablesample.test
M testdata/workloads/functional-planner/queries/PlannerTest/union.test
M testdata/workloads/functional-query/queries/QueryTest/explain-level2.test
M testdata/workloads/functional-query/queries/QueryTest/explain-level3.test
M testdata/workloads/functional-query/queries/QueryTest/stats-extrapolation.test
25 files changed, 1,174 insertions(+), 21 deletions(-)

Approvals:
  Impala Public Jenkins: Looks good to me, approved; Verified

--
To view, visit http://gerrit.cloudera.org:8080/10848
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I1d10eb14d997242f445e5c5fc5362d5410370721
Gerrit-Change-Number: 10848
Gerrit-PatchSet: 7
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Tim Armstrong 


[Impala-ASF-CR] IMPALA-7231: group plan nodes into pipelines

2018-08-10 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10848 )

Change subject: IMPALA-7231: group plan nodes into pipelines
..


Patch Set 6: Verified+1


--
To view, visit http://gerrit.cloudera.org:8080/10848
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1d10eb14d997242f445e5c5fc5362d5410370721
Gerrit-Change-Number: 10848
Gerrit-PatchSet: 6
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Fri, 10 Aug 2018 19:31:29 +
Gerrit-HasComments: No


[impala-udf-samples-CR] Improve contribution guidelines

2018-08-10 Thread Tim Armstrong (Code Review)
Tim Armstrong has uploaded a new patch set (#2). ( 
http://gerrit.cloudera.org:8080/11188 )

Change subject: Improve contribution guidelines
..

Improve contribution guidelines

Change-Id: I1f65e61b68ba63ba739ee917c5d237ca74a97203
---
M README.md
1 file changed, 41 insertions(+), 5 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/impala-udf-samples 
refs/changes/88/11188/2
--
To view, visit http://gerrit.cloudera.org:8080/11188
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: impala-udf-samples
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1f65e61b68ba63ba739ee917c5d237ca74a97203
Gerrit-Change-Number: 11188
Gerrit-PatchSet: 2
Gerrit-Owner: Tim Armstrong 


[impala-udf-samples-CR] Improve contribution guidelines

2018-08-10 Thread Tim Armstrong (Code Review)
Tim Armstrong has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/11188


Change subject: Improve contribution guidelines
..

Improve contribution guidelines

Change-Id: I1f65e61b68ba63ba739ee917c5d237ca74a97203
---
M README.md
1 file changed, 41 insertions(+), 5 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/impala-udf-samples 
refs/changes/88/11188/1
--
To view, visit http://gerrit.cloudera.org:8080/11188
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: impala-udf-samples
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I1f65e61b68ba63ba739ee917c5d237ca74a97203
Gerrit-Change-Number: 11188
Gerrit-PatchSet: 1
Gerrit-Owner: Tim Armstrong 


[Impala-ASF-CR] IMPALA-7420: Different error messages for internal cancellation

2018-08-10 Thread Pooja Nilangekar (Code Review)
Pooja Nilangekar has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11180 )

Change subject: IMPALA-7420: Different error messages for internal cancellation
..


Patch Set 2: Code-Review+1

Looks good to me. I can't +2 it, so adding Bikram.


--
To view, visit http://gerrit.cloudera.org:8080/11180
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Icfd69adfa4b06a68ebef9c350af66f18e0309b8d
Gerrit-Change-Number: 11180
Gerrit-PatchSet: 2
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Bikramjeet Vig 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Pooja Nilangekar 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Fri, 10 Aug 2018 18:46:33 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-7421. Static methods use wrong JNI call function

2018-08-10 Thread Todd Lipcon (Code Review)
Todd Lipcon has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11181 )

Change subject: IMPALA-7421. Static methods use wrong JNI call function
..


Patch Set 1:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/11181/1/be/src/util/jni-util.h
File be/src/util/jni-util.h:

http://gerrit.cloudera.org:8080/#/c/11181/1/be/src/util/jni-util.h@219
PS1, Line 219: JniCall
> Just an idea: the correct usage of the fluent interface could be enforced b
How about an even simpler one? I can make static methods 
JniCall::static(jclass, jmethod) and JniCall::instance(jobject, jmethod). Then 
the remaining member functions are all valid.

Longer term I'd like to add a wrapper like:

template
struct JniFunction {
  JniFunction(const char* function_name);
  Result Call(Args... args);
}

which could handle generating the signature based on the template args and then 
ensure that the passed args match the expected types and count, etc. I didn't 
want to open that template can of worms for now, though (I was only fixing this 
because my tests started crashing since I run them with -Xcheck:jni by default)


http://gerrit.cloudera.org:8080/#/c/11181/1/be/src/util/jni-util.h@455
PS1, Line 455: Sttring
> possible typo: String
good news is I got a new laptop yesterday that doesn't have this repeating key 
issue!



--
To view, visit http://gerrit.cloudera.org:8080/11181
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If7cde6ca91613b63afe5307f4d819fb24cb17fd6
Gerrit-Change-Number: 11181
Gerrit-PatchSet: 1
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Bharath Vissapragada 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Comment-Date: Fri, 10 Aug 2018 18:01:09 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-6709: Simplify tests that copy local files to tables

2018-08-10 Thread Joe McDonnell (Code Review)
Joe McDonnell has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11127 )

Change subject: IMPALA-6709: Simplify tests that copy local files to tables
..


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/11127/1/tests/query_test/test_scanners.py
File tests/query_test/test_scanners.py:

http://gerrit.cloudera.org:8080/#/c/11127/1/tests/query_test/test_scanners.py@a318
PS1, Line 318:
 :
> Yeah I only used CREATE TABLE LIKE when this behaviour seemed irrelevant to
I was confused by some of our documentation that indicates TINYINT/SMALLINT are 
not interpreted correctly ( 
https://github.com/apache/impala/blob/master/docs/topics/impala_create_table.xml#L1087
 ). It looks like this was fixed in IMPALA-5052. I'll look into a filing a doc 
bug.



--
To view, visit http://gerrit.cloudera.org:8080/11127
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie00a4561825facf8abe2e8e74a6b6e93194f416f
Gerrit-Change-Number: 11127
Gerrit-PatchSet: 1
Gerrit-Owner: Zoltan Borok-Nagy 
Gerrit-Reviewer: Andrew Sherman 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Reviewer: Zoltan Borok-Nagy 
Gerrit-Comment-Date: Fri, 10 Aug 2018 17:51:01 +
Gerrit-HasComments: Yes


[native-toolchain-CR] Improve code contribution guidelines

2018-08-10 Thread Tim Armstrong (Code Review)
Tim Armstrong has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/11177 )

Change subject: Improve code contribution guidelines
..

Improve code contribution guidelines

Includes updated links to ICLA/CCLA.

Change-Id: I68cae7309c74e056c179eceaa7047f1830916f12
Reviewed-on: http://gerrit.cloudera.org:8080/11177
Reviewed-by: Thomas Marshall 
Tested-by: Tim Armstrong 
---
M README.md
1 file changed, 37 insertions(+), 10 deletions(-)

Approvals:
  Thomas Marshall: Looks good to me, approved
  Tim Armstrong: Verified

-- 
To view, visit http://gerrit.cloudera.org:8080/11177
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I68cae7309c74e056c179eceaa7047f1830916f12
Gerrit-Change-Number: 11177
Gerrit-PatchSet: 2
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Thomas Marshall 
Gerrit-Reviewer: Tim Armstrong 


[native-toolchain-CR] Improve code contribution guidelines

2018-08-10 Thread Tim Armstrong (Code Review)
Tim Armstrong has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11177 )

Change subject: Improve code contribution guidelines
..


Patch Set 1: Verified+1


--
To view, visit http://gerrit.cloudera.org:8080/11177
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I68cae7309c74e056c179eceaa7047f1830916f12
Gerrit-Change-Number: 11177
Gerrit-PatchSet: 1
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Thomas Marshall 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Fri, 10 Aug 2018 17:35:24 +
Gerrit-HasComments: No


[native-toolchain-CR] Improve code contribution guidelines

2018-08-10 Thread Thomas Marshall (Code Review)
Thomas Marshall has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11177 )

Change subject: Improve code contribution guidelines
..


Patch Set 1: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/11177
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I68cae7309c74e056c179eceaa7047f1830916f12
Gerrit-Change-Number: 11177
Gerrit-PatchSet: 1
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Thomas Marshall 
Gerrit-Comment-Date: Fri, 10 Aug 2018 17:31:13 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-7420: Different error messages for internal cancellation

2018-08-10 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11180 )

Change subject: IMPALA-7420: Different error messages for internal cancellation
..


Patch Set 2:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/287/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/11180
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Icfd69adfa4b06a68ebef9c350af66f18e0309b8d
Gerrit-Change-Number: 11180
Gerrit-PatchSet: 2
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Pooja Nilangekar 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Fri, 10 Aug 2018 17:16:55 +
Gerrit-HasComments: No


[Impala-ASF-CR] WIP IMPALA-7417: Speed up sub-second unix time->TimestampValue conversions

2018-08-10 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11183 )

Change subject: WIP IMPALA-7417: Speed up sub-second unix time->TimestampValue 
conversions
..


Patch Set 2: Verified-1

Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/2975/


--
To view, visit http://gerrit.cloudera.org:8080/11183
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I572b5876b979ddae58165bd40d5b008ce9d7a4aa
Gerrit-Change-Number: 11183
Gerrit-PatchSet: 2
Gerrit-Owner: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Fri, 10 Aug 2018 17:15:14 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-6709: Simplify tests that copy local files to tables

2018-08-10 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11127 )

Change subject: IMPALA-6709: Simplify tests that copy local files to tables
..


Patch Set 3:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/286/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/11127
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie00a4561825facf8abe2e8e74a6b6e93194f416f
Gerrit-Change-Number: 11127
Gerrit-PatchSet: 3
Gerrit-Owner: Zoltan Borok-Nagy 
Gerrit-Reviewer: Andrew Sherman 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Reviewer: Zoltan Borok-Nagy 
Gerrit-Comment-Date: Fri, 10 Aug 2018 17:08:12 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-6709: Simplify tests that copy local files to tables

2018-08-10 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11127 )

Change subject: IMPALA-6709: Simplify tests that copy local files to tables
..


Patch Set 2:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/285/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/11127
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie00a4561825facf8abe2e8e74a6b6e93194f416f
Gerrit-Change-Number: 11127
Gerrit-PatchSet: 2
Gerrit-Owner: Zoltan Borok-Nagy 
Gerrit-Reviewer: Andrew Sherman 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Reviewer: Zoltan Borok-Nagy 
Gerrit-Comment-Date: Fri, 10 Aug 2018 16:55:32 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-7420: Different error messages for internal cancellation

2018-08-10 Thread Tim Armstrong (Code Review)
Tim Armstrong has uploaded a new patch set (#2). ( 
http://gerrit.cloudera.org:8080/11180 )

Change subject: IMPALA-7420: Different error messages for internal cancellation
..

IMPALA-7420: Different error messages for internal cancellation

I started by converting scan and spill-to-disk because the
cancellation there is always meant to be internal to the scan and
spill-to-disk subsystems.

This is to aid triage and debugging of bugs like IMPALA-7418
where an "internal" cancellation leaks out into the query state.
This will make it easier to determine if an internal cancellation
somehow "leaked" out.

Ultimately we might want to use different error codes, but there is a
significant amount of subtle logic that checks for
TErrorCode::CANCELLED.

Testing:
Ran exhaustive tests.

Change-Id: Icfd69adfa4b06a68ebef9c350af66f18e0309b8d
---
M be/src/common/status.cc
M be/src/common/status.h
M be/src/exec/hdfs-orc-scanner.cc
M be/src/exec/hdfs-parquet-scanner.cc
M be/src/exec/hdfs-scan-node.cc
M be/src/exec/hdfs-scanner.cc
M be/src/exec/scanner-context.cc
M be/src/runtime/io/disk-io-mgr-test.cc
M be/src/runtime/io/request-context.cc
M be/src/runtime/tmp-file-mgr.cc
10 files changed, 43 insertions(+), 19 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/80/11180/2
--
To view, visit http://gerrit.cloudera.org:8080/11180
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Icfd69adfa4b06a68ebef9c350af66f18e0309b8d
Gerrit-Change-Number: 11180
Gerrit-PatchSet: 2
Gerrit-Owner: Tim Armstrong 


[Impala-ASF-CR] IMPALA-7415: Fix flakiness in test multiline queries in history

2018-08-10 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11175 )

Change subject: IMPALA-7415: Fix flakiness in test_multiline_queries_in_history
..


Patch Set 2:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2977/ 
DRY_RUN=false


--
To view, visit http://gerrit.cloudera.org:8080/11175
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If7474f832a88bc29b321f21b050c9665294e63d5
Gerrit-Change-Number: 11175
Gerrit-PatchSet: 2
Gerrit-Owner: Bikramjeet Vig 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Fri, 10 Aug 2018 16:41:47 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-7415: Fix flakiness in test multiline queries in history

2018-08-10 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11175 )

Change subject: IMPALA-7415: Fix flakiness in test_multiline_queries_in_history
..


Patch Set 2: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/11175
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If7474f832a88bc29b321f21b050c9665294e63d5
Gerrit-Change-Number: 11175
Gerrit-PatchSet: 2
Gerrit-Owner: Bikramjeet Vig 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Fri, 10 Aug 2018 16:41:46 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-7231: group plan nodes into pipelines

2018-08-10 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10848 )

Change subject: IMPALA-7231: group plan nodes into pipelines
..


Patch Set 5:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/284/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/10848
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1d10eb14d997242f445e5c5fc5362d5410370721
Gerrit-Change-Number: 10848
Gerrit-PatchSet: 5
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Fri, 10 Aug 2018 16:41:33 +
Gerrit-HasComments: No


[Impala-ASF-CR] WIP: IMPALA-4063

2018-08-10 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11185 )

Change subject: WIP: IMPALA-4063
..


Patch Set 1:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/283/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/11185
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I82dd6d6014f5219550082c3cc59e88bfb7d84ef8
Gerrit-Change-Number: 11185
Gerrit-PatchSet: 1
Gerrit-Owner: Sailesh Mukil 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Michael Ho 
Gerrit-Comment-Date: Fri, 10 Aug 2018 16:31:21 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-6709: Simplify tests that copy local files to tables

2018-08-10 Thread Zoltan Borok-Nagy (Code Review)
Zoltan Borok-Nagy has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11127 )

Change subject: IMPALA-6709: Simplify tests that copy local files to tables
..


Patch Set 3:

(4 comments)

http://gerrit.cloudera.org:8080/#/c/11127/1//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/11127/1//COMMIT_MSG@17
PS1, Line 17:
: Change-Id: Ie00a4561825facf8abe2e8e74a6b6e93194f416f
:
> A function that expects the create statement as argument could help in othe
Thanks for the suggestion.
I added the create_table_and_copy_files() function.


http://gerrit.cloudera.org:8080/#/c/11127/1/tests/common/test_parquet.py
File tests/common/test_parquet.py:

http://gerrit.cloudera.org:8080/#/c/11127/1/tests/common/test_parquet.py@24
PS1, Line 24:
> As this only works for Parquet files I would prefer include "parquet" in th
Done


http://gerrit.cloudera.org:8080/#/c/11127/1/tests/common/test_parquet.py@31
PS1, Line 31:
> This is the nittiest of nits, but I am learning python:
Done


http://gerrit.cloudera.org:8080/#/c/11127/1/tests/query_test/test_scanners.py
File tests/query_test/test_scanners.py:

http://gerrit.cloudera.org:8080/#/c/11127/1/tests/query_test/test_scanners.py@a318
PS1, Line 318:
 :
> A note of caution here. There is a distinction between:
Yeah I only used CREATE TABLE LIKE when this behaviour seemed irrelevant to the 
purpose of the test.



--
To view, visit http://gerrit.cloudera.org:8080/11127
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie00a4561825facf8abe2e8e74a6b6e93194f416f
Gerrit-Change-Number: 11127
Gerrit-PatchSet: 3
Gerrit-Owner: Zoltan Borok-Nagy 
Gerrit-Reviewer: Andrew Sherman 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Reviewer: Zoltan Borok-Nagy 
Gerrit-Comment-Date: Fri, 10 Aug 2018 16:22:40 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-6709: Simplify tests that copy local files to tables

2018-08-10 Thread Zoltan Borok-Nagy (Code Review)
Hello Andrew Sherman, Tim Armstrong, Joe McDonnell, Csaba Ringhofer, Impala 
Public Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/11127

to look at the new patch set (#3).

Change subject: IMPALA-6709: Simplify tests that copy local files to tables
..

IMPALA-6709: Simplify tests that copy local files to tables

We had quite a few tests that created a table and used
"hdfs dfs -copyFromLocal" to copy data files to the
warehouse directory for this table.

This operation needs some boilerplate code that I
refactored to the new functions called
create_table_from_parquet() and
create_table_and_copy_files().

Change-Id: Ie00a4561825facf8abe2e8e74a6b6e93194f416f
---
M testdata/workloads/functional-query/queries/QueryTest/parquet-def-levels.test
A tests/common/test_files.py
M tests/query_test/test_parquet_stats.py
M tests/query_test/test_scanners.py
4 files changed, 114 insertions(+), 159 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/27/11127/3
--
To view, visit http://gerrit.cloudera.org:8080/11127
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ie00a4561825facf8abe2e8e74a6b6e93194f416f
Gerrit-Change-Number: 11127
Gerrit-PatchSet: 3
Gerrit-Owner: Zoltan Borok-Nagy 
Gerrit-Reviewer: Andrew Sherman 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Reviewer: Zoltan Borok-Nagy 


[Impala-ASF-CR] IMPALA-6709: Simplify tests that copy local files to tables

2018-08-10 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11127 )

Change subject: IMPALA-6709: Simplify tests that copy local files to tables
..


Patch Set 2:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/11127/2/tests/common/test_files.py
File tests/common/test_files.py:

http://gerrit.cloudera.org:8080/#/c/11127/2/tests/common/test_files.py@59
PS2, Line 59:
flake8: W391 blank line at end of file



--
To view, visit http://gerrit.cloudera.org:8080/11127
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie00a4561825facf8abe2e8e74a6b6e93194f416f
Gerrit-Change-Number: 11127
Gerrit-PatchSet: 2
Gerrit-Owner: Zoltan Borok-Nagy 
Gerrit-Reviewer: Andrew Sherman 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Reviewer: Zoltan Borok-Nagy 
Gerrit-Comment-Date: Fri, 10 Aug 2018 16:12:07 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-6709: Simplify tests that copy local files to tables

2018-08-10 Thread Zoltan Borok-Nagy (Code Review)
Hello Andrew Sherman, Tim Armstrong, Joe McDonnell, Csaba Ringhofer, Impala 
Public Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/11127

to look at the new patch set (#2).

Change subject: IMPALA-6709: Simplify tests that copy local files to tables
..

IMPALA-6709: Simplify tests that copy local files to tables

We had quite a few tests that created a table and used
"hdfs dfs -copyFromLocal" to copy data files to the
warehouse directory for this table.

This operation needs some boilerplate code that I
refactored to the new functions called
create_table_from_parquet() and
create_table_and_copy_files().

Change-Id: Ie00a4561825facf8abe2e8e74a6b6e93194f416f
---
M testdata/workloads/functional-query/queries/QueryTest/parquet-def-levels.test
A tests/common/test_files.py
M tests/query_test/test_parquet_stats.py
M tests/query_test/test_scanners.py
4 files changed, 115 insertions(+), 159 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/27/11127/2
--
To view, visit http://gerrit.cloudera.org:8080/11127
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ie00a4561825facf8abe2e8e74a6b6e93194f416f
Gerrit-Change-Number: 11127
Gerrit-PatchSet: 2
Gerrit-Owner: Zoltan Borok-Nagy 
Gerrit-Reviewer: Andrew Sherman 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Reviewer: Zoltan Borok-Nagy 


[Impala-ASF-CR] IMPALA-7231: group plan nodes into pipelines

2018-08-10 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10848 )

Change subject: IMPALA-7231: group plan nodes into pipelines
..


Patch Set 6:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2976/ 
DRY_RUN=false


--
To view, visit http://gerrit.cloudera.org:8080/10848
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1d10eb14d997242f445e5c5fc5362d5410370721
Gerrit-Change-Number: 10848
Gerrit-PatchSet: 6
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Fri, 10 Aug 2018 16:10:15 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-7231: group plan nodes into pipelines

2018-08-10 Thread Tim Armstrong (Code Review)
Tim Armstrong has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10848 )

Change subject: IMPALA-7231: group plan nodes into pipelines
..


Patch Set 5: Code-Review+2

Fix newly-added planner tests


--
To view, visit http://gerrit.cloudera.org:8080/10848
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1d10eb14d997242f445e5c5fc5362d5410370721
Gerrit-Change-Number: 10848
Gerrit-PatchSet: 5
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Fri, 10 Aug 2018 16:10:01 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-7231: group plan nodes into pipelines

2018-08-10 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10848 )

Change subject: IMPALA-7231: group plan nodes into pipelines
..


Patch Set 5:

(4 comments)

http://gerrit.cloudera.org:8080/#/c/10848/5/fe/src/main/java/org/apache/impala/planner/JoinNode.java
File fe/src/main/java/org/apache/impala/planner/JoinNode.java:

http://gerrit.cloudera.org:8080/#/c/10848/5/fe/src/main/java/org/apache/impala/planner/JoinNode.java@702
PS5, Line 702:   probePipeline.getId(), probePipeline.getHeight() + 
1, TExecNodePhase.GETNEXT));
line too long (93 > 90)


http://gerrit.cloudera.org:8080/#/c/10848/5/fe/src/main/java/org/apache/impala/planner/PlanNode.java
File fe/src/main/java/org/apache/impala/planner/PlanNode.java:

http://gerrit.cloudera.org:8080/#/c/10848/5/fe/src/main/java/org/apache/impala/planner/PlanNode.java@675
PS5, Line 675:childPipeline.getId(), childPipeline.getHeight() 
+ 1, TExecNodePhase.GETNEXT));
line too long (94 > 90)


http://gerrit.cloudera.org:8080/#/c/10848/5/fe/src/main/java/org/apache/impala/planner/SubplanNode.java
File fe/src/main/java/org/apache/impala/planner/SubplanNode.java:

http://gerrit.cloudera.org:8080/#/c/10848/5/fe/src/main/java/org/apache/impala/planner/SubplanNode.java@127
PS5, Line 127:   leftPipeline.getId(), leftPipeline.getHeight() + 
1, TExecNodePhase.GETNEXT));
line too long (91 > 90)


http://gerrit.cloudera.org:8080/#/c/10848/5/fe/src/main/java/org/apache/impala/planner/UnionNode.java
File fe/src/main/java/org/apache/impala/planner/UnionNode.java:

http://gerrit.cloudera.org:8080/#/c/10848/5/fe/src/main/java/org/apache/impala/planner/UnionNode.java@336
PS5, Line 336:   childPipeline.getId(), childPipeline.getHeight() + 
1, TExecNodePhase.GETNEXT));
line too long (93 > 90)



--
To view, visit http://gerrit.cloudera.org:8080/10848
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1d10eb14d997242f445e5c5fc5362d5410370721
Gerrit-Change-Number: 10848
Gerrit-PatchSet: 5
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Fri, 10 Aug 2018 16:10:09 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-7231: group plan nodes into pipelines

2018-08-10 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10848 )

Change subject: IMPALA-7231: group plan nodes into pipelines
..


Patch Set 6: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/10848
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1d10eb14d997242f445e5c5fc5362d5410370721
Gerrit-Change-Number: 10848
Gerrit-PatchSet: 6
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Fri, 10 Aug 2018 16:10:14 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-7231: group plan nodes into pipelines

2018-08-10 Thread Tim Armstrong (Code Review)
Hello Joe McDonnell, Impala Public Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/10848

to look at the new patch set (#5).

Change subject: IMPALA-7231: group plan nodes into pipelines
..

IMPALA-7231: group plan nodes into pipelines

This adds some informational output to explain plans and
sends the information to the backend.

The idea is that this will make it easier to explain how Impala's
pipelined execution works and also enable future work on profile
analysis that can more intelligently group plan nodes.

Tests:
* Updated planner tests to include new output.

Change-Id: I1d10eb14d997242f445e5c5fc5362d5410370721
---
M common/thrift/PlanNodes.thrift
M fe/src/main/java/org/apache/impala/planner/JoinNode.java
A fe/src/main/java/org/apache/impala/planner/PipelineMembership.java
M fe/src/main/java/org/apache/impala/planner/PlanFragment.java
M fe/src/main/java/org/apache/impala/planner/PlanNode.java
M fe/src/main/java/org/apache/impala/planner/Planner.java
M fe/src/main/java/org/apache/impala/planner/SubplanNode.java
M fe/src/main/java/org/apache/impala/planner/UnionNode.java
M 
testdata/workloads/functional-planner/queries/PlannerTest/constant-folding.test
M 
testdata/workloads/functional-planner/queries/PlannerTest/fk-pk-join-detection.test
M 
testdata/workloads/functional-planner/queries/PlannerTest/kudu-selectivity.test
M testdata/workloads/functional-planner/queries/PlannerTest/max-row-size.test
M 
testdata/workloads/functional-planner/queries/PlannerTest/min-max-runtime-filters.test
M 
testdata/workloads/functional-planner/queries/PlannerTest/mt-dop-validation.test
M 
testdata/workloads/functional-planner/queries/PlannerTest/parquet-filtering-disabled.test
M 
testdata/workloads/functional-planner/queries/PlannerTest/parquet-filtering.test
M 
testdata/workloads/functional-planner/queries/PlannerTest/partition-pruning.test
M 
testdata/workloads/functional-planner/queries/PlannerTest/resource-requirements.test
M 
testdata/workloads/functional-planner/queries/PlannerTest/sort-expr-materialization.test
M 
testdata/workloads/functional-planner/queries/PlannerTest/spillable-buffer-sizing.test
M testdata/workloads/functional-planner/queries/PlannerTest/tablesample.test
M testdata/workloads/functional-planner/queries/PlannerTest/union.test
M testdata/workloads/functional-query/queries/QueryTest/explain-level2.test
M testdata/workloads/functional-query/queries/QueryTest/explain-level3.test
M testdata/workloads/functional-query/queries/QueryTest/stats-extrapolation.test
25 files changed, 1,174 insertions(+), 21 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/48/10848/5
--
To view, visit http://gerrit.cloudera.org:8080/10848
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1d10eb14d997242f445e5c5fc5362d5410370721
Gerrit-Change-Number: 10848
Gerrit-PatchSet: 5
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Tim Armstrong 


[Impala-ASF-CR] WIP: IMPALA-4063

2018-08-10 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11185 )

Change subject: WIP: IMPALA-4063
..


Patch Set 1:

(3 comments)

http://gerrit.cloudera.org:8080/#/c/11185/1/be/src/runtime/coordinator-backend-state.cc
File be/src/runtime/coordinator-backend-state.cc:

http://gerrit.cloudera.org:8080/#/c/11185/1/be/src/runtime/coordinator-backend-state.cc@272
PS1, Line 272:   //<< report_version << " | Last report version: " 
<< instance_stats->last_report_version_
line too long (103 > 90)


http://gerrit.cloudera.org:8080/#/c/11185/1/be/src/runtime/query-state.cc
File be/src/runtime/query-state.cc:

http://gerrit.cloudera.org:8080/#/c/11185/1/be/src/runtime/query-state.cc@516
PS1, Line 516:   // We prioritize thread creation failure as a query 
killing error, even over an error
line too long (91 > 90)


http://gerrit.cloudera.org:8080/#/c/11185/1/be/src/service/client-request-state.cc
File be/src/service/client-request-state.cc:

http://gerrit.cloudera.org:8080/#/c/11185/1/be/src/service/client-request-state.cc@1198
PS1, Line 1198: const ReportExecStatusRequestPB& request, const 
TRuntimeProfileForest& thrift_profiles) {
line too long (93 > 90)



--
To view, visit http://gerrit.cloudera.org:8080/11185
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I82dd6d6014f5219550082c3cc59e88bfb7d84ef8
Gerrit-Change-Number: 11185
Gerrit-PatchSet: 1
Gerrit-Owner: Sailesh Mukil 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Fri, 10 Aug 2018 16:02:00 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-7398: Add logged in user alias for effective user

2018-08-10 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11184 )

Change subject: IMPALA-7398: Add logged_in_user alias for effective_user
..


Patch Set 2:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/282/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/11184
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id63f243e0fffbe2798f1f9dbc4cc3ebe9d9529a6
Gerrit-Change-Number: 11184
Gerrit-PatchSet: 2
Gerrit-Owner: Adam Holley 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Fri, 10 Aug 2018 16:01:02 +
Gerrit-HasComments: No


[Impala-ASF-CR] WIP: IMPALA-4063

2018-08-10 Thread Sailesh Mukil (Code Review)
Sailesh Mukil has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/11185


Change subject: WIP: IMPALA-4063
..

WIP: IMPALA-4063

Change-Id: I82dd6d6014f5219550082c3cc59e88bfb7d84ef8
---
M be/src/runtime/coordinator-backend-state.cc
M be/src/runtime/coordinator-backend-state.h
M be/src/runtime/coordinator.cc
M be/src/runtime/coordinator.h
M be/src/runtime/fragment-instance-state.cc
M be/src/runtime/fragment-instance-state.h
M be/src/runtime/query-exec-mgr.cc
M be/src/runtime/query-state.cc
M be/src/runtime/query-state.h
M be/src/service/client-request-state.cc
M be/src/service/client-request-state.h
M be/src/service/control-service.cc
M common/protobuf/control_service.proto
M common/thrift/RuntimeProfile.thrift
14 files changed, 185 insertions(+), 227 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/85/11185/1
--
To view, visit http://gerrit.cloudera.org:8080/11185
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I82dd6d6014f5219550082c3cc59e88bfb7d84ef8
Gerrit-Change-Number: 11185
Gerrit-PatchSet: 1
Gerrit-Owner: Sailesh Mukil 


[Impala-ASF-CR] IMPALA-7398: Add logged in user alias for effective user

2018-08-10 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11184 )

Change subject: IMPALA-7398: Add logged_in_user alias for effective_user
..


Patch Set 2:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/11184/2/common/function-registry/impala_functions.py
File common/function-registry/impala_functions.py:

http://gerrit.cloudera.org:8080/#/c/11184/2/common/function-registry/impala_functions.py@580
PS2, Line 580: c
flake8: E501 line too long (114 > 90 characters)


http://gerrit.cloudera.org:8080/#/c/11184/2/tests/custom_cluster/test_delegation.py
File tests/custom_cluster/test_delegation.py:

http://gerrit.cloudera.org:8080/#/c/11184/2/tests/custom_cluster/test_delegation.py@37
PS2, Line 37: ;
flake8: E703 statement ends with a semicolon



--
To view, visit http://gerrit.cloudera.org:8080/11184
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id63f243e0fffbe2798f1f9dbc4cc3ebe9d9529a6
Gerrit-Change-Number: 11184
Gerrit-PatchSet: 2
Gerrit-Owner: Adam Holley 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Fri, 10 Aug 2018 15:30:26 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-7398: Add logged in user alias for effective user

2018-08-10 Thread Adam Holley (Code Review)
Adam Holley has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/11184


Change subject: IMPALA-7398: Add logged_in_user alias for effective_user
..

IMPALA-7398: Add logged_in_user alias for effective_user

This patch adds an alias to the effective_user function so that
views created in Hive using the logged_in_user function will work
in Impala.

Example:
CREATE VIEW foo.view1 AS SELECT * FROM foo.table1 WHERE
name=logged_in_user();

Tests:
- Added function and ran delegation tests
- Ran backend tests
- Ran custom-cluster tests including delegation

Change-Id: Id63f243e0fffbe2798f1f9dbc4cc3ebe9d9529a6
---
M be/src/exprs/expr-test.cc
M common/function-registry/impala_functions.py
M tests/custom_cluster/test_delegation.py
3 files changed, 4 insertions(+), 3 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/84/11184/2
--
To view, visit http://gerrit.cloudera.org:8080/11184
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Id63f243e0fffbe2798f1f9dbc4cc3ebe9d9529a6
Gerrit-Change-Number: 11184
Gerrit-PatchSet: 2
Gerrit-Owner: Adam Holley 


[Impala-ASF-CR] WIP IMPALA-7417: Speed up sub-second unix time->TimestampValue conversions

2018-08-10 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11183 )

Change subject: WIP IMPALA-7417: Speed up sub-second unix time->TimestampValue 
conversions
..


Patch Set 2:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/281/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/11183
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I572b5876b979ddae58165bd40d5b008ce9d7a4aa
Gerrit-Change-Number: 11183
Gerrit-PatchSet: 2
Gerrit-Owner: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Fri, 10 Aug 2018 14:37:50 +
Gerrit-HasComments: No


[Impala-ASF-CR] WIP IMPALA-7417: Speed up sub-second unix time->TimestampValue conversions

2018-08-10 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11183 )

Change subject: WIP IMPALA-7417: Speed up sub-second unix time->TimestampValue 
conversions
..


Patch Set 2:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2975/ 
DRY_RUN=true


--
To view, visit http://gerrit.cloudera.org:8080/11183
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I572b5876b979ddae58165bd40d5b008ce9d7a4aa
Gerrit-Change-Number: 11183
Gerrit-PatchSet: 2
Gerrit-Owner: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Fri, 10 Aug 2018 14:04:50 +
Gerrit-HasComments: No


[Impala-ASF-CR] WIP IMPALA-7417: Speed up sub-second unix time->TimestampValue conversions

2018-08-10 Thread Csaba Ringhofer (Code Review)
Csaba Ringhofer has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/11183


Change subject: WIP IMPALA-7417: Speed up sub-second unix time->TimestampValue 
conversions
..

WIP IMPALA-7417: Speed up sub-second unix time->TimestampValue conversions

Change-Id: I572b5876b979ddae58165bd40d5b008ce9d7a4aa
---
M be/src/benchmarks/convert-timestamp-benchmark.cc
M be/src/exec/data-source-scan-node.cc
M be/src/exec/hdfs-orc-scanner.cc
M be/src/runtime/timestamp-value.cc
M be/src/runtime/timestamp-value.h
M be/src/runtime/timestamp-value.inline.h
6 files changed, 117 insertions(+), 75 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/83/11183/2
--
To view, visit http://gerrit.cloudera.org:8080/11183
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I572b5876b979ddae58165bd40d5b008ce9d7a4aa
Gerrit-Change-Number: 11183
Gerrit-PatchSet: 2
Gerrit-Owner: Csaba Ringhofer 


[Impala-ASF-CR] IMPALA-7421. Static methods use wrong JNI call function

2018-08-10 Thread Csaba Ringhofer (Code Review)
Csaba Ringhofer has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11181 )

Change subject: IMPALA-7421. Static methods use wrong JNI call function
..


Patch Set 1:

(6 comments)

http://gerrit.cloudera.org:8080/#/c/11181/1/be/src/util/jni-util.h
File be/src/util/jni-util.h:

http://gerrit.cloudera.org:8080/#/c/11181/1/be/src/util/jni-util.h@219
PS1, Line 219: JniCall
Just an idea: the correct usage of the fluent interface could be enforced by 
the compiler. One way to do this is changing the members to protected and 
creating subclasses of JniCall like JniCallExpectArguments and 
JniCallExpectCall. Functions that are not callable on a freshly constructed 
JniCall could be moved there and the "next stage" could be specified by casting 
the return value to the correct subclass, e.g.

  JniCallExpectArguments& on_class(jclass cls) {
DCHECK(!instance_ && !class_);
class_ = DCHECK_NOTNULL(cls);
return *(JniCallExpectArguments*)this;
  }


http://gerrit.cloudera.org:8080/#/c/11181/1/be/src/util/jni-util.h@227
PS1, Line 227:   JniCall& on_class(jclass cls) {
WARN_UNUSED_RESULT could be added here and in other "fluent" functions.


http://gerrit.cloudera.org:8080/#/c/11181/1/be/src/util/jni-util.h@234
PS1, Line 234: !instance_ && !instance_
The same thing is checked twice here.


http://gerrit.cloudera.org:8080/#/c/11181/1/be/src/util/jni-util.h@455
PS1, Line 455: Sttring
possible typo: String


http://gerrit.cloudera.org:8080/#/c/11181/1/be/src/util/logging-support.cc
File be/src/util/logging-support.cc:

http://gerrit.cloudera.org:8080/#/c/11181/1/be/src/util/logging-support.cc@139
PS1, Line 139:   return Status::OK();
As this is already modified the RETURN_IF_ERROR could be replaced with simply 
return.


http://gerrit.cloudera.org:8080/#/c/11181/1/be/src/util/logging-support.cc@144
PS1, Line 144:   return Status::OK();
Same as line 139.



--
To view, visit http://gerrit.cloudera.org:8080/11181
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If7cde6ca91613b63afe5307f4d819fb24cb17fd6
Gerrit-Change-Number: 11181
Gerrit-PatchSet: 1
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Bharath Vissapragada 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Comment-Date: Fri, 10 Aug 2018 13:37:08 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-7333: remove MarkNeedsDeepCopy() in agg and BTS

2018-08-10 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11156 )

Change subject: IMPALA-7333: remove MarkNeedsDeepCopy() in agg and BTS
..


Patch Set 4: Verified+1


--
To view, visit http://gerrit.cloudera.org:8080/11156
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5f5b8a418d4816f603a64da6287ec392dbd4603f
Gerrit-Change-Number: 11156
Gerrit-PatchSet: 4
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Fri, 10 Aug 2018 09:22:10 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-7333: remove MarkNeedsDeepCopy() in agg and BTS

2018-08-10 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/11156 )

Change subject: IMPALA-7333: remove MarkNeedsDeepCopy() in agg and BTS
..

IMPALA-7333: remove MarkNeedsDeepCopy() in agg and BTS

This takes advantage of work (e.g. IMPALA-3200, IMPALA-5844)
to remove a couple of uses of the API.

Testing:
Ran core, ASAN and exhaustive builds.

Added unit tests to directly test the attaching behaviour.

Change-Id: I5f5b8a418d4816f603a64da6287ec392dbd4603f
Reviewed-on: http://gerrit.cloudera.org:8080/11156
Reviewed-by: Impala Public Jenkins 
Tested-by: Impala Public Jenkins 
---
M be/src/exec/grouping-aggregator.cc
M be/src/runtime/buffered-tuple-stream-test.cc
M be/src/runtime/buffered-tuple-stream.cc
M be/src/runtime/buffered-tuple-stream.h
M be/src/runtime/row-batch.h
M be/src/runtime/tuple.h
6 files changed, 408 insertions(+), 137 deletions(-)

Approvals:
  Impala Public Jenkins: Looks good to me, approved; Verified

--
To view, visit http://gerrit.cloudera.org:8080/11156
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I5f5b8a418d4816f603a64da6287ec392dbd4603f
Gerrit-Change-Number: 11156
Gerrit-PatchSet: 5
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tim Armstrong 


[Impala-ASF-CR] IMPALA-7333: remove MarkNeedsDeepCopy() in agg and BTS

2018-08-10 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11156 )

Change subject: IMPALA-7333: remove MarkNeedsDeepCopy() in agg and BTS
..


Patch Set 4: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/11156
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5f5b8a418d4816f603a64da6287ec392dbd4603f
Gerrit-Change-Number: 11156
Gerrit-PatchSet: 4
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Fri, 10 Aug 2018 06:07:30 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-7333: remove MarkNeedsDeepCopy() in agg and BTS

2018-08-10 Thread Tim Armstrong (Code Review)
Tim Armstrong has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11156 )

Change subject: IMPALA-7333: remove MarkNeedsDeepCopy() in agg and BTS
..


Patch Set 3:

Hit  IMPALA-7415


--
To view, visit http://gerrit.cloudera.org:8080/11156
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5f5b8a418d4816f603a64da6287ec392dbd4603f
Gerrit-Change-Number: 11156
Gerrit-PatchSet: 3
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Fri, 10 Aug 2018 06:07:17 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-7415: Fix flakiness in test multiline queries in history

2018-08-10 Thread Tim Armstrong (Code Review)
Tim Armstrong has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11175 )

Change subject: IMPALA-7415: Fix flakiness in test_multiline_queries_in_history
..


Patch Set 1: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/11175
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If7474f832a88bc29b321f21b050c9665294e63d5
Gerrit-Change-Number: 11175
Gerrit-PatchSet: 1
Gerrit-Owner: Bikramjeet Vig 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Fri, 10 Aug 2018 06:07:32 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-7333: remove MarkNeedsDeepCopy() in agg and BTS

2018-08-10 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/11156 )

Change subject: IMPALA-7333: remove MarkNeedsDeepCopy() in agg and BTS
..


Patch Set 4:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2973/ 
DRY_RUN=false


--
To view, visit http://gerrit.cloudera.org:8080/11156
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5f5b8a418d4816f603a64da6287ec392dbd4603f
Gerrit-Change-Number: 11156
Gerrit-PatchSet: 4
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Fri, 10 Aug 2018 06:07:32 +
Gerrit-HasComments: No