date:20180411

Tim Armstrong has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/10029 )

Change subject: IMPALA-6837: allow multiple networks for distcc server
..

IMPALA-6837: allow multiple networks for distcc server

Testing:
Tested the script on Ubuntu 14.04 and CentOS 6 servers to confirm
that it resulted in a working configuration.

Change-Id: I6d49786b4f6e9c3bfab244e63620a62b2ae4dd64
Reviewed-on: http://gerrit.cloudera.org:8080/10029
Reviewed-by: Philip Zeyliger 
Tested-by: Tim Armstrong 
---
M bin/distcc/distcc_server_setup.sh
1 file changed, 9 insertions(+), 1 deletion(-)

Approvals:
  Philip Zeyliger: Looks good to me, approved
  Tim Armstrong: Verified

--
To view, visit http://gerrit.cloudera.org:8080/10029
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I6d49786b4f6e9c3bfab244e63620a62b2ae4dd64
Gerrit-Change-Number: 10029
Gerrit-PatchSet: 2
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Tim Armstrong

[Impala-ASF-CR] IMPALA-4631: loosen monotonic clock DCHECK

Tim Armstrong has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/10026 )

Change subject: IMPALA-4631: loosen monotonic clock DCHECK
..

IMPALA-4631: loosen monotonic clock DCHECK

We saw another build failure due to hitting one of these DCHECKs.
Let's further loosen the check to avoid the failures on misbehaving
systems.

Change-Id: I72d314518087aede16e8d702c2f904b679a55f6d
Reviewed-on: http://gerrit.cloudera.org:8080/10026
Tested-by: Impala Public Jenkins 
Reviewed-by: Tim Armstrong 
---
M be/src/runtime/fragment-instance-state.cc
M be/src/util/runtime-profile-counters.h
2 files changed, 7 insertions(+), 6 deletions(-)

Approvals:
  Impala Public Jenkins: Verified
  Tim Armstrong: Looks good to me, approved

--
To view, visit http://gerrit.cloudera.org:8080/10026
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I72d314518087aede16e8d702c2f904b679a55f6d
Gerrit-Change-Number: 10026
Gerrit-PatchSet: 3
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Tim Armstrong

[Impala-ASF-CR] IMPALA-4631: loosen monotonic clock DCHECK

Tim Armstrong has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10026 )

Change subject: IMPALA-4631: loosen monotonic clock DCHECK
..


Patch Set 2: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/10026
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I72d314518087aede16e8d702c2f904b679a55f6d
Gerrit-Change-Number: 10026
Gerrit-PatchSet: 2
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Thu, 12 Apr 2018 05:29:02 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6518,IMPALA-6340: Check that decimal types are compatible in FE

Alex Behm has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9930 )

Change subject: IMPALA-6518,IMPALA-6340: Check that decimal types are 
compatible in FE
..


Patch Set 1:

(23 comments)

http://gerrit.cloudera.org:8080/#/c/9930/1/fe/src/main/java/org/apache/impala/analysis/Expr.java
File fe/src/main/java/org/apache/impala/analysis/Expr.java:

http://gerrit.cloudera.org:8080/#/c/9930/1/fe/src/main/java/org/apache/impala/analysis/Expr.java@429
PS1, Line 429:   protected void castForFunctionCall(boolean 
ignoreWildcardDecimals, boolean decimal_v2)
decimalV2 (camel-case)


http://gerrit.cloudera.org:8080/#/c/9930/1/fe/src/main/java/org/apache/impala/analysis/Expr.java@440
PS1, Line 440: if (resolvedWildcardType.isInvalid()) {
Check this immediately after L433?


http://gerrit.cloudera.org:8080/#/c/9930/1/fe/src/main/java/org/apache/impala/analysis/Expr.java@442
PS1, Line 442:   "Unable to resolve the decimal types of the %s 
function arguments. " +
Also include the argument types in the error message


http://gerrit.cloudera.org:8080/#/c/9930/1/fe/src/main/java/org/apache/impala/analysis/Expr.java@461
PS1, Line 461: StringBuilder errorMsg = new StringBuilder();
unused?


http://gerrit.cloudera.org:8080/#/c/9930/1/fe/src/main/java/org/apache/impala/analysis/Expr.java@482
PS1, Line 482:   if (result.isNull()) {
Let's consistently throw either in this function or in the caller. The caller 
has more context to produce a better error message (like you've already shown 
in this change), so probably better to move this check to the caller.

You can remove the throws declaration from this function then.


http://gerrit.cloudera.org:8080/#/c/9930/1/fe/src/main/java/org/apache/impala/catalog/Function.java
File fe/src/main/java/org/apache/impala/catalog/Function.java:

http://gerrit.cloudera.org:8080/#/c/9930/1/fe/src/main/java/org/apache/impala/catalog/Function.java@217
PS1, Line 217:   other.argTypes_[i], this.argTypes_[i], strict, false)) 
{
This looks like a case that shows that the decimalV2 arg and 'strict' should 
possibly be combined. According to the function comment 'strict' should not 
allow loss of precision, so decimalV2 should be true?


http://gerrit.cloudera.org:8080/#/c/9930/1/fe/src/main/java/org/apache/impala/catalog/ScalarType.java
File fe/src/main/java/org/apache/impala/catalog/ScalarType.java:

http://gerrit.cloudera.org:8080/#/c/9930/1/fe/src/main/java/org/apache/impala/catalog/ScalarType.java@141
PS1, Line 141:   public static ScalarType createClippedDecimalType(
It feels cleaner to me to move the decimalV2 check to the callers and not 
inside here, since we are not actually clipping.

Alternatively, we should comment what the decimalV2 argument does here (I still 
prefer checking at callers).


http://gerrit.cloudera.org:8080/#/c/9930/1/fe/src/main/java/org/apache/impala/catalog/ScalarType.java@381
PS1, Line 381:   Preconditions.checkState(scale_ <= 38);
<= MAX_PRECISION


http://gerrit.cloudera.org:8080/#/c/9930/1/fe/src/main/java/org/apache/impala/catalog/ScalarType.java@393
PS1, Line 393:   Preconditions.checkState(scale_ <= 38);
<= MAX_PRECISION


http://gerrit.cloudera.org:8080/#/c/9930/1/fe/src/main/java/org/apache/impala/planner/HashJoinNode.java
File fe/src/main/java/org/apache/impala/planner/HashJoinNode.java:

http://gerrit.cloudera.org:8080/#/c/9930/1/fe/src/main/java/org/apache/impala/planner/HashJoinNode.java@88
PS1, Line 88:   throw new InternalException(String.format(
Is it guaranteed that both types are decimal at this point?


http://gerrit.cloudera.org:8080/#/c/9930/1/fe/src/main/java/org/apache/impala/planner/HashJoinNode.java@89
PS1, Line 89:   "Unable to create a hash join on the following two 
decimal " +
Also print the join predicate in the error message, otherwise it might be very 
hard for users to find the right place to CAST.

Suggestion:

Unable to create a hash join with equi-join predicate  because 
the operands cannot be cast without loss of precision. Operand types:  = 


http://gerrit.cloudera.org:8080/#/c/9930/1/fe/src/test/java/org/apache/impala/analysis/AnalyzeExprsTest.java
File fe/src/test/java/org/apache/impala/analysis/AnalyzeExprsTest.java:

http://gerrit.cloudera.org:8080/#/c/9930/1/fe/src/test/java/org/apache/impala/analysis/AnalyzeExprsTest.java@2570
PS1, Line 2570: "Unable to resolve the decimal types of the 
_impala_builtins.coalesce " +
Can we also print the FunctionCallExpr.toSql() at this point? It might be hard 
for users to figure out where it's going wrong in a huge query.


http://gerrit.cloudera.org:8080/#/c/9930/1/fe/src/test/java/org/apache/impala/analysis/AnalyzeExprsTest.java@2573
PS1, Line 2573: testDecimalExpr("coalesce(cast(0.789 as decimal(19, 19)), " 
+
duplicate test?

[Impala-ASF-CR] IMPALA-6480: [DOCS] DESCRIBE respects column-level privilege

Alex Behm has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9996 )

Change subject: IMPALA-6480: [DOCS] DESCRIBE respects column-level privilege
..


Patch Set 3:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/9996/3/docs/topics/impala_describe.xml
File docs/topics/impala_describe.xml:

http://gerrit.cloudera.org:8080/#/c/9996/3/docs/topics/impala_describe.xml@146
PS3, Line 146:   the LOCATION field. The 
LOCATION data
The LOCATION data is shown if you have any privilege on the table, the 
containing database or the server.



--
To view, visit http://gerrit.cloudera.org:8080/9996
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I094e00c2a4e8b19226e06afd8cf67968265edc4d
Gerrit-Change-Number: 9996
Gerrit-PatchSet: 3
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Adam Holley 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Thu, 12 Apr 2018 03:47:25 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6480: [DOCS] DESCRIBE respects column-level privilege

2018-04-11 Thread Adam Holley (Code Review)

Adam Holley has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9996 )

Change subject: IMPALA-6480: [DOCS] DESCRIBE respects column-level privilege
..


Patch Set 3: Code-Review+1


--
To view, visit http://gerrit.cloudera.org:8080/9996
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I094e00c2a4e8b19226e06afd8cf67968265edc4d
Gerrit-Change-Number: 9996
Gerrit-PatchSet: 3
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Adam Holley 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Thu, 12 Apr 2018 03:45:50 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6713: Fix request for unneeded memory in partial sort

2018-04-11 Thread Thomas Tauber-Marshall (Code Review)

Thomas Tauber-Marshall has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/10031


Change subject: IMPALA-6713: Fix request for unneeded memory in partial sort
..

IMPALA-6713: Fix request for unneeded memory in partial sort

When a Sorter::Run is initialzed, if it is an initial run and has
varlen data, it requests an extra buffer to have space to sort the
varlen data should it need to spill to disk.

This extra buffer is not needed in the case of partial sorts, which
do not spill, and because this extra buffer was not included in the
calculation of the minimum required reservation, requesting it caused
the partial sort to fail in cases where the partial sort only had its
minimum reservation available to use.

The solution is to not request the extra memory for partial sorts.

Testing:
- Added a test to test_sort.py that ensures the partial sort can
  complete successfully even if additional memory requests beyond its
  minimum reservation are denied.

Change-Id: I2d9c0863009021340d8b684669b371a2cfb1ecad
---
M be/src/runtime/sorter.cc
M tests/query_test/test_sort.py
2 files changed, 19 insertions(+), 2 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/31/10031/1
--
To view, visit http://gerrit.cloudera.org:8080/10031
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I2d9c0863009021340d8b684669b371a2cfb1ecad
Gerrit-Change-Number: 10031
Gerrit-PatchSet: 1
Gerrit-Owner: Thomas Tauber-Marshall

[Impala-ASF-CR] IMPALA-6480: [DOCS] DESCRIBE respects column-level privilege

Hello Alex Behm, Impala Public Jenkins, Adam Holley,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/9996

to look at the new patch set (#3).

Change subject: IMPALA-6480: [DOCS] DESCRIBE respects column-level privilege
..

IMPALA-6480: [DOCS] DESCRIBE respects column-level privilege

Change-Id: I094e00c2a4e8b19226e06afd8cf67968265edc4d
---
M docs/topics/impala_describe.xml
1 file changed, 15 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/96/9996/3
--
To view, visit http://gerrit.cloudera.org:8080/9996
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I094e00c2a4e8b19226e06afd8cf67968265edc4d
Gerrit-Change-Number: 9996
Gerrit-PatchSet: 3
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Adam Holley 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins

[Impala-ASF-CR] IMPALA-6480: [DOCS] DESCRIBE respects column-level privilege

Alex Rodoni has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9996 )

Change subject: IMPALA-6480: [DOCS] DESCRIBE respects column-level privilege
..


Patch Set 1:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/9996/1/docs/topics/impala_describe.xml
File docs/topics/impala_describe.xml:

http://gerrit.cloudera.org:8080/#/c/9996/1/docs/topics/impala_describe.xml@137
PS1, Line 137:   in the table, DESCRIBE returns the data 
from the columns
> If you only have the SELECT privilege on a subset of the table columns and
Done


http://gerrit.cloudera.org:8080/#/c/9996/1/docs/topics/impala_describe.xml@142
PS1, Line 142:   If you have the SELECT privilege on a 
subset of columns
> Same suggestion as above
Done



--
To view, visit http://gerrit.cloudera.org:8080/9996
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I094e00c2a4e8b19226e06afd8cf67968265edc4d
Gerrit-Change-Number: 9996
Gerrit-PatchSet: 1
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Adam Holley 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Thu, 12 Apr 2018 03:33:00 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6480: [DOCS] DESCRIBE respects column-level privilege

Hello Alex Behm, Impala Public Jenkins, Adam Holley,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/9996

to look at the new patch set (#2).

Change subject: IMPALA-6480: [DOCS] DESCRIBE respects column-level privilege
..

IMPALA-6480: [DOCS] DESCRIBE respects column-level privilege

Change-Id: I094e00c2a4e8b19226e06afd8cf67968265edc4d
---
M docs/topics/impala_describe.xml
1 file changed, 13 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/96/9996/2
--
To view, visit http://gerrit.cloudera.org:8080/9996
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I094e00c2a4e8b19226e06afd8cf67968265edc4d
Gerrit-Change-Number: 9996
Gerrit-PatchSet: 2
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Adam Holley 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins

[Impala-ASF-CR] IMPALA-6811: Increase timeout on S3 for test exchange delays

2018-04-11 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/9995 )

Change subject: IMPALA-6811: Increase timeout on S3 for test_exchange_delays
..

IMPALA-6811: Increase timeout on S3 for test_exchange_delays

The exchange-delays-zero-rows portion of test_exchange_delays
tests a RPC timeout when sending an EOS with no rows. In
order to send the EOS, the executor must have completed
the scan (which returns zero rows). On S3, the scan has
been slow and can exceed the current receiver delay of
10 seconds, leading to a failure to timeout. In manual
testing, the sender timeout is being applied quickly
once the scan finishes.

This increases the receiver delay for S3 and ADLS to
20 seconds. None of the existing symptoms show the
scan taking that long. Most failures have a scan that
barely exceeds 10 seconds, the current receiver delay.
This has only been seen on S3, but ADLS is included as a
precaution.

Change-Id: I967e6eb336c801219c77d657655c42984910b479
Reviewed-on: http://gerrit.cloudera.org:8080/9995
Reviewed-by: Joe McDonnell 
Tested-by: Impala Public Jenkins 
---
M tests/custom_cluster/test_exchange_delays.py
1 file changed, 7 insertions(+), 2 deletions(-)

Approvals:
  Joe McDonnell: Looks good to me, approved
  Impala Public Jenkins: Verified

--
To view, visit http://gerrit.cloudera.org:8080/9995
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I967e6eb336c801219c77d657655c42984910b479
Gerrit-Change-Number: 9995
Gerrit-PatchSet: 5
Gerrit-Owner: Joe McDonnell 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Michael Brown 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Vuk Ercegovac

[Impala-ASF-CR] IMPALA-6811: Increase timeout on S3 for test exchange delays

2018-04-11 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9995 )

Change subject: IMPALA-6811: Increase timeout on S3 for test_exchange_delays
..


Patch Set 4: Verified+1


--
To view, visit http://gerrit.cloudera.org:8080/9995
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I967e6eb336c801219c77d657655c42984910b479
Gerrit-Change-Number: 9995
Gerrit-PatchSet: 4
Gerrit-Owner: Joe McDonnell 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Michael Brown 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Vuk Ercegovac 
Gerrit-Comment-Date: Thu, 12 Apr 2018 03:30:52 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-4631: loosen monotonic clock DCHECK

2018-04-11 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10026 )

Change subject: IMPALA-4631: loosen monotonic clock DCHECK
..


Patch Set 2: Verified+1


--
To view, visit http://gerrit.cloudera.org:8080/10026
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I72d314518087aede16e8d702c2f904b679a55f6d
Gerrit-Change-Number: 10026
Gerrit-PatchSet: 2
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Thu, 12 Apr 2018 03:27:42 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6480: [DOCS] DESCRIBE respects column-level privilege

2018-04-11 Thread Adam Holley (Code Review)

Adam Holley has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9996 )

Change subject: IMPALA-6480: [DOCS] DESCRIBE respects column-level privilege
..


Patch Set 1:

Should we add a note in the document that the "LOCATION" data for the extended 
describe will only be visible if the user has table level privileges?  It is 
not visible if they have column level privileges.


--
To view, visit http://gerrit.cloudera.org:8080/9996
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I094e00c2a4e8b19226e06afd8cf67968265edc4d
Gerrit-Change-Number: 9996
Gerrit-PatchSet: 1
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Adam Holley 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Thu, 12 Apr 2018 03:24:56 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6480: [DOCS] DESCRIBE respects column-level privilege

Alex Behm has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9996 )

Change subject: IMPALA-6480: [DOCS] DESCRIBE respects column-level privilege
..


Patch Set 1:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/9996/1/docs/topics/impala_describe.xml
File docs/topics/impala_describe.xml:

http://gerrit.cloudera.org:8080/#/c/9996/1/docs/topics/impala_describe.xml@137
PS1, Line 137:   in the table, DESCRIBE returns the data 
from the columns
If you only have the SELECT privilege on a subset of the table columns and no 
other relevant table/database/server-level privileges, then ...


http://gerrit.cloudera.org:8080/#/c/9996/1/docs/topics/impala_describe.xml@142
PS1, Line 142:   If you have the SELECT privilege on a 
subset of columns
Same suggestion as above



--
To view, visit http://gerrit.cloudera.org:8080/9996
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I094e00c2a4e8b19226e06afd8cf67968265edc4d
Gerrit-Change-Number: 9996
Gerrit-PatchSet: 1
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Adam Holley 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Thu, 12 Apr 2018 03:06:27 +
Gerrit-HasComments: Yes

[Impala-ASF-CR](2.x) IMPALA-5717: Support for reading ORC data files

Philip Zeyliger has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9988 )

Change subject: IMPALA-5717: Support for reading ORC data files
..


Patch Set 1:

Thanks for pro-actively doing the cherry-pick. I unblocked most of the 
cherrypick job and now the next failed cherrypick is Tianyi's change and then 
this one.

818cd8fa2721cd8205b304563b728952bffc8b2f IMPALA-5717: Support for reading ORC 
data files (Wed Apr 11 05:13:02 2018 +) - stiga-huang
d28b39afae5b8f1d621dcd2e1884c0353f1c058f IMPALA-6820: Remove _impala_builtins 
from catalogd (Wed Apr 11 03:55:07 2018 +) - Tianyi Wang


--
To view, visit http://gerrit.cloudera.org:8080/9988
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: 2.x
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia7b6ae4ce3b9ee8125b21993702faa87537790a4
Gerrit-Change-Number: 9988
Gerrit-PatchSet: 1
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Reviewer: Tianyi Wang 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Thu, 12 Apr 2018 03:02:47 +
Gerrit-HasComments: No

[Impala-ASF-CR](2.x) IMPALA-6805: Show current database in Impala shell prompt

Philip Zeyliger has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/10024 )

Change subject: IMPALA-6805: Show current database in Impala shell prompt
..

IMPALA-6805: Show current database in Impala shell prompt

Prompt format:
[host:port] db_name>

Testing:
- Added new shell tests
- Ran end-to-end shell tests

Change-Id: Ifb0ae58507321e426e5f0f16518671420974a3fc
Reviewed-on: http://gerrit.cloudera.org:8080/9927
Reviewed-by: Fredy Wijaya 
Reviewed-by: Michael Brown 
Tested-by: Impala Public Jenkins 
Reviewed-on: http://gerrit.cloudera.org:8080/10024
Reviewed-by: Philip Zeyliger 
Tested-by: Philip Zeyliger 
---
M shell/impala_shell.py
M tests/shell/test_shell_interactive.py
2 files changed, 50 insertions(+), 7 deletions(-)

Approvals:
  Philip Zeyliger: Looks good to me, approved; Verified

--
To view, visit http://gerrit.cloudera.org:8080/10024
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: 2.x
Gerrit-MessageType: merged
Gerrit-Change-Id: Ifb0ae58507321e426e5f0f16518671420974a3fc
Gerrit-Change-Number: 10024
Gerrit-PatchSet: 3
Gerrit-Owner: Philip Zeyliger 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Michael Brown 
Gerrit-Reviewer: Philip Zeyliger

[Impala-ASF-CR](2.x) IMPALA-6805: Show current database in Impala shell prompt

Philip Zeyliger has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10024 )

Change subject: IMPALA-6805: Show current database in Impala shell prompt
..


Patch Set 2: Verified+1 Code-Review+2

I ran gerrit-verify-dryrun with this change (and equivalent parents) already, 
so carrying +2 and applying +1 from gerrit-verify-dryrun-external/112.


--
To view, visit http://gerrit.cloudera.org:8080/10024
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: 2.x
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifb0ae58507321e426e5f0f16518671420974a3fc
Gerrit-Change-Number: 10024
Gerrit-PatchSet: 2
Gerrit-Owner: Philip Zeyliger 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Michael Brown 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Comment-Date: Thu, 12 Apr 2018 02:59:14 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6480: [DOCS] DESCRIBE respects column-level privilege

Alex Rodoni has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9996 )

Change subject: IMPALA-6480: [DOCS] DESCRIBE respects column-level privilege
..


Patch Set 1:

I need to add this for the 2.12 / 3.0 release. Could you please 
review/comment/approve?


--
To view, visit http://gerrit.cloudera.org:8080/9996
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I094e00c2a4e8b19226e06afd8cf67968265edc4d
Gerrit-Change-Number: 9996
Gerrit-PatchSet: 1
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Adam Holley 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Thu, 12 Apr 2018 02:42:46 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6837: allow multiple networks for distcc server

Philip Zeyliger has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10029 )

Change subject: IMPALA-6837: allow multiple networks for distcc server
..


Patch Set 1: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/10029
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I6d49786b4f6e9c3bfab244e63620a62b2ae4dd64
Gerrit-Change-Number: 10029
Gerrit-PatchSet: 1
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Comment-Date: Thu, 12 Apr 2018 02:43:03 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6451: AuthorizationException in CTAS for Kudu tables

Fredy Wijaya has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/10030


Change subject: IMPALA-6451: AuthorizationException in CTAS for Kudu tables
..

IMPALA-6451: AuthorizationException in CTAS for Kudu tables

CREATE TABLE on EXTERNAL Kudu tables or with TBLPROPERTIES
('kudu.master_addresses') require ALL privileges on SERVER.
See IMPALA-4000.

The patch fixes a bug where the CTAS statement is rewritten
and during the initial analysis phase, 'kudu.master_addresses'
property is added into TBLPROPERTIES which causes the next
analysis phase to assume the statement was executed with
TBLPROPERTIES ('kudu.master_addresses'). Hence, causing the
ALL privilege request on SERVER to be registered.

Testing:
- Added a new test
- Ran all front-end tests

Change-Id: Iac3bbe4dceb80dfefd9756bc322f8c10ceab9f49
---
M fe/src/main/java/org/apache/impala/analysis/CreateTableStmt.java
M fe/src/test/java/org/apache/impala/analysis/AuthorizationTest.java
2 files changed, 22 insertions(+), 1 deletion(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/30/10030/1
--
To view, visit http://gerrit.cloudera.org:8080/10030
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Iac3bbe4dceb80dfefd9756bc322f8c10ceab9f49
Gerrit-Change-Number: 10030
Gerrit-PatchSet: 1
Gerrit-Owner: Fredy Wijaya

[Impala-ASF-CR] IMPALA-6822: Add a query option to control shuffling by distinct exprs

2018-04-11 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9949 )

Change subject: IMPALA-6822: Add a query option to control shuffling by 
distinct exprs
..


Patch Set 6: Verified-1

Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/2287/


--
To view, visit http://gerrit.cloudera.org:8080/9949
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Icb4b4576fb29edd62cf4b4ba0719c0e0a2a5a8dc
Gerrit-Change-Number: 9949
Gerrit-PatchSet: 6
Gerrit-Owner: Tianyi Wang 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tianyi Wang 
Gerrit-Comment-Date: Thu, 12 Apr 2018 01:39:41 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6793: Fix empty metadata after statestore restarts

2018-04-11 Thread Tianyi Wang (Code Review)

Tianyi Wang has uploaded a new patch set (#8). ( 
http://gerrit.cloudera.org:8080/9921 )

Change subject: IMPALA-6793: Fix empty metadata after statestore restarts
..

IMPALA-6793: Fix empty metadata after statestore restarts

IMPALA-5990 introduced a bug where restarting the statestore
deterministically clears the metadata without ever coming back. The
cause of the bug is a wrong condition used by catalog to detect the
restart of statestore.

A custom cluster regression test is added. The process restarting
utility function in the custom cluster test is changed into using
shell=True in popen.

Change-Id: I332a60e172af84b93b3544373fe363cdced5e8d0
---
M be/src/catalog/catalog-server.cc
M be/src/catalog/catalog-server.h
M tests/common/impala_cluster.py
A tests/custom_cluster/test_restart_services.py
M tests/util/shell_util.py
5 files changed, 69 insertions(+), 13 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/21/9921/8
--
To view, visit http://gerrit.cloudera.org:8080/9921
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I332a60e172af84b93b3544373fe363cdced5e8d0
Gerrit-Change-Number: 9921
Gerrit-PatchSet: 8
Gerrit-Owner: Tianyi Wang 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Dimitris Tsirogiannis 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Michael Brown 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Tianyi Wang 
Gerrit-Reviewer: Vuk Ercegovac

[Impala-ASF-CR] Move some test spilling debug actions to exhaustive

2018-04-11 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/9976 )

Change subject: Move some test_spilling debug actions to exhaustive
..

Move some test_spilling debug actions to exhaustive

Phil Z pointed out that these tests take a long time to run.
Based on experience, we only need the always/never variants
to catch the vast majority of spilling-related bugs, so
relegating the other variants to exhaustive should not cause any major
problems.

People making spilling-related changes to the spilling code should run
this test with the exhaustive dimension to flush out any problems in
advance, e.g.:

  impala-py.test tests/query_test/test_spilling.py -n2 --verbose \
--workload_exploration_strategy=functional-query:exhaustive

Change-Id: I9ea9f6c299480f8dfc943635342e4473499cc8ad
Reviewed-on: http://gerrit.cloudera.org:8080/9976
Reviewed-by: Tim Armstrong 
Reviewed-by: Philip Zeyliger 
Tested-by: Impala Public Jenkins 
---
M tests/query_test/test_spilling.py
1 file changed, 13 insertions(+), 4 deletions(-)

Approvals:
  Tim Armstrong: Looks good to me, approved
  Philip Zeyliger: Looks good to me, approved
  Impala Public Jenkins: Verified

--
To view, visit http://gerrit.cloudera.org:8080/9976
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I9ea9f6c299480f8dfc943635342e4473499cc8ad
Gerrit-Change-Number: 9976
Gerrit-PatchSet: 4
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Tim Armstrong

[Impala-ASF-CR] Move some test spilling debug actions to exhaustive

2018-04-11 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9976 )

Change subject: Move some test_spilling debug actions to exhaustive
..


Patch Set 3: Verified+1


--
To view, visit http://gerrit.cloudera.org:8080/9976
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9ea9f6c299480f8dfc943635342e4473499cc8ad
Gerrit-Change-Number: 9976
Gerrit-PatchSet: 3
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Thu, 12 Apr 2018 01:25:35 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6120: Add thread timers for reporting codegen time

Bikramjeet Vig has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9960 )

Change subject: IMPALA-6120: Add thread timers for reporting codegen time
..


Patch Set 6:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/9960/6/be/src/runtime/fragment-instance-state.cc
File be/src/runtime/fragment-instance-state.cc:

http://gerrit.cloudera.org:8080/#/c/9960/6/be/src/runtime/fragment-instance-state.cc@256
PS6, Line 256:  RETURN_IF_ERROR(runtime_state_->CodegenScalarFns());
missed to include this



--
To view, visit http://gerrit.cloudera.org:8080/9960
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I24d5a46b8870bc959b89045432d2e86af72b30e5
Gerrit-Change-Number: 9960
Gerrit-PatchSet: 6
Gerrit-Owner: Bikramjeet Vig 
Gerrit-Reviewer: Bikramjeet Vig 
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Thu, 12 Apr 2018 01:18:13 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6120: Add thread timers for reporting codegen time

Hello Michael Ho, Tim Armstrong,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/9960

to look at the new patch set (#6).

Change subject: IMPALA-6120: Add thread timers for reporting codegen time
..

IMPALA-6120: Add thread timers for reporting codegen time

Add thread times for accurate reporting of codegen time.
Also cleaned up a few places where time elapsed was being counted twice.

Sample Profile:

Query: SELECT count(*) FROM tpch_parquet.lineitem
WHERE l_partkey in (1,6,11,16,21,26,31,36,41);

CodeGen:(Total: 37.948ms, non-child: 37.948ms, % non-child: 100.00%)
   - CodegenInvoluntaryContextSwitches: 0 (0)
   - CodegenTotalWallClockTime: 37.942ms
 - CodegenSysTime: 0.000ns
 - CodegenUserTime: 36.938ms
   - CodegenVoluntaryContextSwitches: 0 (0)
   - CompileTime: 2.065ms
   - IrGenerationTime: 392.351us
   - LoadTime: 0.000ns
   - ModuleBitcodeSize: 2.26 MB (2373148)
   - NumFunctions: 22 (22)
   - NumInstructions: 381 (381)
   - OptimizationTime: 21.416ms
   - PeakMemoryUsage: 190.50 KB (195072)
   - PrepareTime: 13.496ms

Sample Profile with an added 2 sec sleep time to "OptimizationTime":

CodeGen:(Total: 2s037ms, non-child: 2s037ms, % non-child: 100.00%)
   - CodegenInvoluntaryContextSwitches: 0 (0)
   - CodegenTotalWallClockTime: 2s037ms
 - CodegenSysTime: 0.000ns
 - CodegenUserTime: 37.672ms
   - CodegenVoluntaryContextSwitches: 1 (1)
   - CompileTime: 2.032ms
   - IrGenerationTime: 386.948us
   - LoadTime: 0.000ns
   - ModuleBitcodeSize: 2.26 MB (2373148)
   - NumFunctions: 22 (22)
   - NumInstructions: 381 (381)
   - OptimizationTime: 2s023ms
   - PeakMemoryUsage: 190.50 KB (195072)
   - PrepareTime: 11.598ms

Change-Id: I24d5a46b8870bc959b89045432d2e86af72b30e5
---
M be/src/codegen/llvm-codegen.cc
M be/src/codegen/llvm-codegen.h
M be/src/exec/hdfs-avro-scanner.cc
M be/src/exec/hdfs-parquet-scanner.cc
M be/src/exec/hdfs-scanner.cc
M be/src/exec/partitioned-aggregation-node.cc
M be/src/exec/select-node.cc
M be/src/exec/text-converter.cc
M be/src/runtime/fragment-instance-state.cc
M be/src/runtime/tuple.cc
M be/src/util/tuple-row-compare.cc
11 files changed, 30 insertions(+), 39 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/60/9960/6
--
To view, visit http://gerrit.cloudera.org:8080/9960
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I24d5a46b8870bc959b89045432d2e86af72b30e5
Gerrit-Change-Number: 9960
Gerrit-PatchSet: 6
Gerrit-Owner: Bikramjeet Vig 
Gerrit-Reviewer: Bikramjeet Vig 
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Tim Armstrong

[Impala-ASF-CR](2.x) IMPALA-5717: Support for reading ORC data files

Tim Armstrong has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9988 )

Change subject: IMPALA-5717: Support for reading ORC data files
..


Patch Set 1:

Here's the ORC patch backported to 2.x with a minor conflict resolved. Joe, I 
think you were looking at the backed up commits on 2.x, so I'll leave you to 
decide when to submit this.


--
To view, visit http://gerrit.cloudera.org:8080/9988
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: 2.x
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia7b6ae4ce3b9ee8125b21993702faa87537790a4
Gerrit-Change-Number: 9988
Gerrit-PatchSet: 1
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Thu, 12 Apr 2018 01:14:25 +
Gerrit-HasComments: No

[Impala-ASF-CR](2.x) IMPALA-5717: Support for reading ORC data files

Hello Quanlong Huang, Impala Public Jenkins,

I'd like you to do a code review. Please visit

http://gerrit.cloudera.org:8080/9988

to review the following change.


Change subject: IMPALA-5717: Support for reading ORC data files
..

IMPALA-5717: Support for reading ORC data files

This patch integrates the orc library into Impala and implements
HdfsOrcScanner as a middle layer between them. The HdfsOrcScanner
supplies input needed from the orc-reader, tracks memory consumption of
the reader and transfers the reader's output (orc::ColumnVectorBatch)
into impala::RowBatch. The ORC version we used is release-1.4.3.

A startup option --enable_orc_scanner is added for this feature. It's
set to true by default. Setting it to false will fail queries on ORC
tables.

Currently, we only support reading primitive types. Writing into ORC
table has not been supported neither.

Tests
 - Most of the end-to-end tests can run on ORC format.
 - Add tpcds, tpch tests for ORC.
 - Add some ORC specific tests.
 - Haven't enabled test_scanner_fuzz for ORC yet, since the ORC library
   is not robust for corrupt files (ORC-315).

Change-Id: Ia7b6ae4ce3b9ee8125b21993702faa87537790a4
Reviewed-on: http://gerrit.cloudera.org:8080/9134
Reviewed-by: Quanlong Huang 
Reviewed-by: Tim Armstrong 
Tested-by: Impala Public Jenkins 
---
M CMakeLists.txt
M be/CMakeLists.txt
M be/src/codegen/gen_ir_descriptions.py
M be/src/exec/CMakeLists.txt
A be/src/exec/hdfs-orc-scanner.cc
A be/src/exec/hdfs-orc-scanner.h
M be/src/exec/hdfs-parquet-scanner-ir.cc
M be/src/exec/hdfs-parquet-scanner.cc
M be/src/exec/hdfs-parquet-scanner.h
M be/src/exec/hdfs-scan-node-base.cc
M be/src/exec/hdfs-scan-node-mt.cc
M be/src/exec/hdfs-scanner-ir.cc
M be/src/exec/hdfs-scanner.cc
M be/src/exec/hdfs-scanner.h
M be/src/util/backend-gflag-util.cc
M bin/bootstrap_toolchain.py
M bin/impala-config.sh
A cmake_modules/FindOrc.cmake
M common/thrift/BackendGflags.thrift
M common/thrift/CatalogObjects.thrift
M fe/src/main/cup/sql-parser.cup
M fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java
M fe/src/main/java/org/apache/impala/catalog/HdfsFileFormat.java
M fe/src/main/java/org/apache/impala/catalog/HdfsStorageDescriptor.java
M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java
M fe/src/main/java/org/apache/impala/service/BackendConfig.java
M fe/src/main/java/org/apache/impala/service/Frontend.java
M fe/src/main/jflex/sql-scanner.flex
M testdata/LineItemMultiBlock/README.dox
A testdata/LineItemMultiBlock/lineitem_orc_multiblock_one_stripe.orc
A testdata/LineItemMultiBlock/lineitem_sixblocks.orc
A testdata/LineItemMultiBlock/lineitem_threeblocks.orc
M testdata/bin/create-load-data.sh
M testdata/bin/generate-schema-statements.py
M testdata/bin/run-hive-server.sh
M testdata/cluster/node_templates/common/etc/hadoop/conf/hdfs-site.xml.tmpl
A testdata/data/chars-formats.orc
M testdata/datasets/functional/functional_schema_template.sql
M testdata/datasets/functional/schema_constraints.csv
M 
testdata/workloads/functional-planner/queries/PlannerTest/complex-types-file-formats.test
M testdata/workloads/functional-query/functional-query_core.csv
M testdata/workloads/functional-query/functional-query_dimensions.csv
M testdata/workloads/functional-query/functional-query_exhaustive.csv
M testdata/workloads/functional-query/functional-query_pairwise.csv
A 
testdata/workloads/functional-query/queries/DataErrorsTest/orc-type-checks.test
M testdata/workloads/tpcds/tpcds_core.csv
M testdata/workloads/tpcds/tpcds_dimensions.csv
M testdata/workloads/tpcds/tpcds_exhaustive.csv
M testdata/workloads/tpcds/tpcds_pairwise.csv
M testdata/workloads/tpch/tpch_core.csv
M testdata/workloads/tpch/tpch_dimensions.csv
M testdata/workloads/tpch/tpch_exhaustive.csv
M testdata/workloads/tpch/tpch_pairwise.csv
M tests/common/impala_test_suite.py
M tests/common/test_dimensions.py
M tests/common/test_vector.py
M tests/comparison/cli_options.py
M tests/query_test/test_chars.py
M tests/query_test/test_decimal_queries.py
M tests/query_test/test_scanners.py
M tests/query_test/test_scanners_fuzz.py
M tests/query_test/test_tpch_queries.py
62 files changed, 1,745 insertions(+), 307 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/88/9988/1
--
To view, visit http://gerrit.cloudera.org:8080/9988
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: 2.x
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ia7b6ae4ce3b9ee8125b21993702faa87537790a4
Gerrit-Change-Number: 9988
Gerrit-PatchSet: 1
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Quanlong Huang

[Impala-ASF-CR](2.x) IMPALA-5717: Support for reading ORC data files

Tim Armstrong has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9988 )

Change subject: IMPALA-5717: Support for reading ORC data files
..


Patch Set 1: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/9988
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: 2.x
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia7b6ae4ce3b9ee8125b21993702faa87537790a4
Gerrit-Change-Number: 9988
Gerrit-PatchSet: 1
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Thu, 12 Apr 2018 01:13:47 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6120: Add thread timers for reporting codegen time

Bikramjeet Vig has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9960 )

Change subject: IMPALA-6120: Add thread timers for reporting codegen time
..


Patch Set 5: Code-Review+2

(1 comment)

Carrying over Tim's +2

http://gerrit.cloudera.org:8080/#/c/9960/4/be/src/codegen/llvm-codegen.h
File be/src/codegen/llvm-codegen.h:

http://gerrit.cloudera.org:8080/#/c/9960/4/be/src/codegen/llvm-codegen.h@758
PS4, Line 758: N_STA
> Is OPEN a phase or a state? Not sure what terminology we use, or if we're c
Turns out we do define states and this corresponds to "CODEGEN_START" state. 
See 
https://github.com/apache/impala/blob/818cd8fa2721cd8205b304563b728952bffc8b2f/be/src/runtime/fragment-instance-state.h#L186



--
To view, visit http://gerrit.cloudera.org:8080/9960
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I24d5a46b8870bc959b89045432d2e86af72b30e5
Gerrit-Change-Number: 9960
Gerrit-PatchSet: 5
Gerrit-Owner: Bikramjeet Vig 
Gerrit-Reviewer: Bikramjeet Vig 
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Thu, 12 Apr 2018 01:11:57 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6120: Add thread timers for reporting codegen time

Hello Michael Ho, Tim Armstrong,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/9960

to look at the new patch set (#5).

Change subject: IMPALA-6120: Add thread timers for reporting codegen time
..

IMPALA-6120: Add thread timers for reporting codegen time

Add thread times for accurate reporting of codegen time.
Also cleaned up a few places where time elapsed was being counted twice.

Sample Profile:

Query: SELECT count(*) FROM tpch_parquet.lineitem
WHERE l_partkey in (1,6,11,16,21,26,31,36,41);

CodeGen:(Total: 37.948ms, non-child: 37.948ms, % non-child: 100.00%)
   - CodegenInvoluntaryContextSwitches: 0 (0)
   - CodegenTotalWallClockTime: 37.942ms
 - CodegenSysTime: 0.000ns
 - CodegenUserTime: 36.938ms
   - CodegenVoluntaryContextSwitches: 0 (0)
   - CompileTime: 2.065ms
   - IrGenerationTime: 392.351us
   - LoadTime: 0.000ns
   - ModuleBitcodeSize: 2.26 MB (2373148)
   - NumFunctions: 22 (22)
   - NumInstructions: 381 (381)
   - OptimizationTime: 21.416ms
   - PeakMemoryUsage: 190.50 KB (195072)
   - PrepareTime: 13.496ms

Sample Profile with an added 2 sec sleep time to "OptimizationTime":

CodeGen:(Total: 2s037ms, non-child: 2s037ms, % non-child: 100.00%)
   - CodegenInvoluntaryContextSwitches: 0 (0)
   - CodegenTotalWallClockTime: 2s037ms
 - CodegenSysTime: 0.000ns
 - CodegenUserTime: 37.672ms
   - CodegenVoluntaryContextSwitches: 1 (1)
   - CompileTime: 2.032ms
   - IrGenerationTime: 386.948us
   - LoadTime: 0.000ns
   - ModuleBitcodeSize: 2.26 MB (2373148)
   - NumFunctions: 22 (22)
   - NumInstructions: 381 (381)
   - OptimizationTime: 2s023ms
   - PeakMemoryUsage: 190.50 KB (195072)
   - PrepareTime: 11.598ms

Change-Id: I24d5a46b8870bc959b89045432d2e86af72b30e5
---
M be/src/codegen/llvm-codegen.cc
M be/src/codegen/llvm-codegen.h
M be/src/exec/hdfs-avro-scanner.cc
M be/src/exec/hdfs-parquet-scanner.cc
M be/src/exec/hdfs-scanner.cc
M be/src/exec/partitioned-aggregation-node.cc
M be/src/exec/select-node.cc
M be/src/exec/text-converter.cc
M be/src/runtime/fragment-instance-state.cc
M be/src/runtime/tuple.cc
M be/src/util/tuple-row-compare.cc
11 files changed, 26 insertions(+), 35 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/60/9960/5
--
To view, visit http://gerrit.cloudera.org:8080/9960
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I24d5a46b8870bc959b89045432d2e86af72b30e5
Gerrit-Change-Number: 9960
Gerrit-PatchSet: 5
Gerrit-Owner: Bikramjeet Vig 
Gerrit-Reviewer: Bikramjeet Vig 
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Tim Armstrong

[Impala-ASF-CR] IMPALA-6837: allow multiple networks for distcc server

Tim Armstrong has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/10029


Change subject: IMPALA-6837: allow multiple networks for distcc server
..

IMPALA-6837: allow multiple networks for distcc server

Testing:
Tested the script on Ubuntu 14.04 and CentOS 6 servers to confirm
that it resulted in a working configuration.

Change-Id: I6d49786b4f6e9c3bfab244e63620a62b2ae4dd64
---
M bin/distcc/distcc_server_setup.sh
1 file changed, 9 insertions(+), 1 deletion(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/29/10029/1
--
To view, visit http://gerrit.cloudera.org:8080/10029
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I6d49786b4f6e9c3bfab244e63620a62b2ae4dd64
Gerrit-Change-Number: 10029
Gerrit-PatchSet: 1
Gerrit-Owner: Tim Armstrong

[Impala-ASF-CR] IMPALA-5842: Write page index in Parquet files

Tim Armstrong has posted comments on this change. (
http://gerrit.cloudera.org:8080/9693 )

Change subject: IMPALA-5842: Write page index in Parquet files
..

Patch Set 10:

(9 comments)

Overall this is looking good. I had some specific concerns about some of the
nitty-gritty details.

http://gerrit.cloudera.org:8080/#/c/9693/10/be/src/exec/hdfs-parquet-table-writer.cc
File be/src/exec/hdfs-parquet-table-writer.cc:

http://gerrit.cloudera.org:8080/#/c/9693/10/be/src/exec/hdfs-parquet-table-writer.cc@301
PS10, Line 301: std::vector min_values_;
I'm still concerned about the amount of untracked memory from min_values_ and
max_values_, even if we truncate the string values to 1KB or similar - it seems
like could end up with multiple MB of untracked memory. We could probably live
with it since it's smaller than the actual data, but it's a step in the wrong
direction.

Maybe we could store min_values_ and max_values_ as StringValues backed by
memory per_file_mem_pool_ and then only convert to strings when writing out
each column to the page index?

http://gerrit.cloudera.org:8080/#/c/9693/10/be/src/exec/hdfs-parquet-table-writer.cc@735
PS10, Line 735: min_values_.push_back(std::string(""));
I don't know if we need the call to std::string() here, I think it should work
if we just emplace_back() to instantiate an empty string.

http://gerrit.cloudera.org:8080/#/c/9693/10/be/src/exec/hdfs-parquet-table-writer.cc@1227
PS10, Line 1227: for (auto& column : columns_) {
nit: can fit loop on one line.

http://gerrit.cloudera.org:8080/#/c/9693/10/be/src/exec/parquet-column-stats.h
File be/src/exec/parquet-column-stats.h:

http://gerrit.cloudera.org:8080/#/c/9693/10/be/src/exec/parquet-column-stats.h@159
PS10, Line 159: // If true, min/max values are ascending.
Maybe briefly mention why they both start off true? And both can be true at the
same time? It's slightly subtle.

http://gerrit.cloudera.org:8080/#/c/9693/10/tests/query_test/test_parquet_page_index.py
File tests/query_test/test_parquet_page_index.py:

http://gerrit.cloudera.org:8080/#/c/9693/10/tests/query_test/test_parquet_page_index.py@37
PS10, Line 37: class TestHdfsParquetTableIndexWriter(ImpalaTestSuite):
We've got a lot of good coverage in this test.

I'm wondering if we're missing some basic tests that confirm that the values in
the page match the min/max values in the page index. It seems like these
validations might not catch some kinds of bugs. E.g. min/max values in the
index are somehow out-of-sync with the pages. Most bugs that I can imagine
would get caught by one validation or another but it would be nice to have a
sanity test where we confirm that the values in each page match the values in
the page index.

http://gerrit.cloudera.org:8080/#/c/9693/10/tests/query_test/test_parquet_page_index.py@177
PS10, Line 177: previouse_value
typo in variable name

http://gerrit.cloudera.org:8080/#/c/9693/10/tests/query_test/test_parquet_page_index.py@205
PS10, Line 205: falied
nit: failed

http://gerrit.cloudera.org:8080/#/c/9693/10/tests/query_test/test_parquet_page_index.py@205
PS10, Line 205: column_info_schema
this variable isn't defined - did you mean column_info?

http://gerrit.cloudera.org:8080/#/c/9693/10/tests/query_test/test_parquet_page_index.py@244
PS10, Line 244: chars_formats
chars_formats is weird in that it's created by a different test -
TestCharsFormats. I.e. it's not present unless that test ran before this one.
Maybe we should change it so that the table is loaded during normal data
loading?

--
To view, visit http://gerrit.cloudera.org:8080/9693
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Icbacf7fe3b7672e3ce719261ecef445b16f8dec9
Gerrit-Change-Number: 9693
Gerrit-PatchSet: 10
Gerrit-Owner: Zoltan Borok-Nagy
Gerrit-Reviewer: Anonymous Coward #248
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Lars Volker
Gerrit-Reviewer: Tim Armstrong
Gerrit-Reviewer: Zoltan Borok-Nagy
Gerrit-Comment-Date: Thu, 12 Apr 2018 00:23:08 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6518,IMPALA-6340: Check that decimal types are compatible in FE

Alex Behm has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9930 )

Change subject: IMPALA-6518,IMPALA-6340: Check that decimal types are 
compatible in FE
..


Patch Set 1:

(4 comments)

High-level questions/comments. I'm still going through the details.

http://gerrit.cloudera.org:8080/#/c/9930/1/fe/src/main/java/org/apache/impala/analysis/AnalyticExpr.java
File fe/src/main/java/org/apache/impala/analysis/AnalyticExpr.java:

http://gerrit.cloudera.org:8080/#/c/9930/1/fe/src/main/java/org/apache/impala/analysis/AnalyticExpr.java@385
PS1, Line 385: rangeExpr.getType(), 
orderByElements_.get(0).getExpr().getType(),
Let's change this to be strict regardless of decimal_v2. My understanding is 
that we don't support RANGE offset boundaries today, so this code change is not 
incompatible.

See AnalyzeExprsTest.TestAnalyticExprs()


http://gerrit.cloudera.org:8080/#/c/9930/1/fe/src/main/java/org/apache/impala/analysis/ArithmeticExpr.java
File fe/src/main/java/org/apache/impala/analysis/ArithmeticExpr.java:

http://gerrit.cloudera.org:8080/#/c/9930/1/fe/src/main/java/org/apache/impala/analysis/ArithmeticExpr.java@218
PS1, Line 218: false, analyzer.getQueryOptions().isDecimal_v2());
Since this decimalV2 check is so common, let's add a function in Analyzer 
directly:

boolean isDecimalV2() { return getQueryOptions().isDecimal_v2(); }


http://gerrit.cloudera.org:8080/#/c/9930/1/fe/src/main/java/org/apache/impala/catalog/Type.java
File fe/src/main/java/org/apache/impala/catalog/Type.java:

http://gerrit.cloudera.org:8080/#/c/9930/1/fe/src/main/java/org/apache/impala/catalog/Type.java@292
PS1, Line 292:* If strict is true, only consider casts that result in no 
loss of precision.
Why is the decimalV2 case different than the strict case?


http://gerrit.cloudera.org:8080/#/c/9930/1/fe/src/main/java/org/apache/impala/catalog/Type.java@296
PS1, Line 296:   Type t1, Type t2, boolean strict, boolean decimal_v2) {
decimalV2

(camel-case in Java land)



--
To view, visit http://gerrit.cloudera.org:8080/9930
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id406f4189e01a909152985fabd5cca7a1527a568
Gerrit-Change-Number: 9930
Gerrit-PatchSet: 1
Gerrit-Owner: Taras Bobrovytsky 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Comment-Date: Thu, 12 Apr 2018 00:19:22 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-5717: Support for reading ORC data files

Alex Behm has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9134 )

Change subject: IMPALA-5717: Support for reading ORC data files
..


Patch Set 21:

Awesome work, Quanlong! Thanks for your contribution.


--
To view, visit http://gerrit.cloudera.org:8080/9134
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia7b6ae4ce3b9ee8125b21993702faa87537790a4
Gerrit-Change-Number: 9134
Gerrit-PatchSet: 21
Gerrit-Owner: Quanlong Huang 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Thu, 12 Apr 2018 00:05:42 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6809: Allow bootstrap system.sh in non ~/Impala directory

Fredy Wijaya has uploaded a new patch set (#5). ( 
http://gerrit.cloudera.org:8080/9994 )

Change subject: IMPALA-6809: Allow bootstrap_system.sh in non ~/Impala directory
..

IMPALA-6809: Allow bootstrap_system.sh in non ~/Impala directory

Testing:
- Ran bootstrap_development.sh with IMPALA_HOME set to non ~/Impala
  directory
- Ran bootstrap_development.sh with IMPALA_HOME not set

Change-Id: I3241c180b5fb28f1b5f939200f72461ad6fd7d7a
---
M bin/bootstrap_system.sh
1 file changed, 14 insertions(+), 8 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/94/9994/5
--
To view, visit http://gerrit.cloudera.org:8080/9994
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I3241c180b5fb28f1b5f939200f72461ad6fd7d7a
Gerrit-Change-Number: 9994
Gerrit-PatchSet: 5
Gerrit-Owner: Fredy Wijaya 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Philip Zeyliger

[Impala-ASF-CR] IMPALA-6809: Allow bootstrap system.sh in non ~/Impala directory

Fredy Wijaya has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9994 )

Change subject: IMPALA-6809: Allow bootstrap_system.sh in non ~/Impala directory
..


Patch Set 5:

(3 comments)

http://gerrit.cloudera.org:8080/#/c/9994/3/bin/bootstrap_system.sh
File bin/bootstrap_system.sh:

http://gerrit.cloudera.org:8080/#/c/9994/3/bin/bootstrap_system.sh@24
PS3, Line 24: # variables to make Impala compile and run after this script is 
complete.
> Add a note here that it respects IMPALA_HOME
Done


http://gerrit.cloudera.org:8080/#/c/9994/3/bin/bootstrap_system.sh@102
PS3, Line 102: # If there is no Impala git repo, get one now
> Impala house style for this is:
Done


http://gerrit.cloudera.org:8080/#/c/9994/3/bin/bootstrap_system.sh@208
PS3, Line 208: then
> Define IMPALA_LZO_HOME next to where you define IMPALA_HOME and use that?
Done



--
To view, visit http://gerrit.cloudera.org:8080/9994
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3241c180b5fb28f1b5f939200f72461ad6fd7d7a
Gerrit-Change-Number: 9994
Gerrit-PatchSet: 5
Gerrit-Owner: Fredy Wijaya 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Comment-Date: Thu, 12 Apr 2018 00:03:32 +
Gerrit-HasComments: Yes

[native-toolchain-CR] Bump Kudu version to a954418

2018-04-11 Thread Thomas Tauber-Marshall (Code Review)

Thomas Tauber-Marshall has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9981 )

Change subject: Bump Kudu version to a954418
..


Patch Set 1: Verified+1


--
To view, visit http://gerrit.cloudera.org:8080/9981
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic66c8e492dcd648cfb2c184f2d64449095ccf726
Gerrit-Change-Number: 9981
Gerrit-PatchSet: 1
Gerrit-Owner: Thomas Tauber-Marshall 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Thomas Tauber-Marshall 
Gerrit-Comment-Date: Wed, 11 Apr 2018 23:43:10 +
Gerrit-HasComments: No

[native-toolchain-CR] Bump Kudu version to a954418

2018-04-11 Thread Thomas Tauber-Marshall (Code Review)

Thomas Tauber-Marshall has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/9981 )

Change subject: Bump Kudu version to a954418
..

Bump Kudu version to a954418

Change-Id: Ic66c8e492dcd648cfb2c184f2d64449095ccf726
Reviewed-on: http://gerrit.cloudera.org:8080/9981
Reviewed-by: Alex Behm 
Tested-by: Thomas Tauber-Marshall 
---
M buildall.sh
1 file changed, 1 insertion(+), 1 deletion(-)

Approvals:
  Alex Behm: Looks good to me, approved
  Thomas Tauber-Marshall: Verified

--
To view, visit http://gerrit.cloudera.org:8080/9981
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ic66c8e492dcd648cfb2c184f2d64449095ccf726
Gerrit-Change-Number: 9981
Gerrit-PatchSet: 2
Gerrit-Owner: Thomas Tauber-Marshall 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Thomas Tauber-Marshall

[native-toolchain-CR] Bump Kudu version to a954418

Alex Behm has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9981 )

Change subject: Bump Kudu version to a954418
..


Patch Set 1: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/9981
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic66c8e492dcd648cfb2c184f2d64449095ccf726
Gerrit-Change-Number: 9981
Gerrit-PatchSet: 1
Gerrit-Owner: Thomas Tauber-Marshall 
Gerrit-Reviewer: Alex Behm 
Gerrit-Comment-Date: Wed, 11 Apr 2018 23:34:28 +
Gerrit-HasComments: No

[Impala-ASF-CR] Bump Kudu version to a954418

Alex Behm has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9982 )

Change subject: Bump Kudu version to a954418
..


Patch Set 1: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/9982
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib06c0fb3c24a8cee1dd4f34a221cf41a711a5359
Gerrit-Change-Number: 9982
Gerrit-PatchSet: 1
Gerrit-Owner: Thomas Tauber-Marshall 
Gerrit-Reviewer: Alex Behm 
Gerrit-Comment-Date: Wed, 11 Apr 2018 23:34:44 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6811: Increase timeout on S3 for test exchange delays

2018-04-11 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9995 )

Change subject: IMPALA-6811: Increase timeout on S3 for test_exchange_delays
..


Patch Set 4:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2290/


--
To view, visit http://gerrit.cloudera.org:8080/9995
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I967e6eb336c801219c77d657655c42984910b479
Gerrit-Change-Number: 9995
Gerrit-PatchSet: 4
Gerrit-Owner: Joe McDonnell 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Michael Brown 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Vuk Ercegovac 
Gerrit-Comment-Date: Wed, 11 Apr 2018 23:31:54 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-3833: Fix invalid data handling in Sequence and RCFile scanners

Tim Armstrong has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/8936 )

Change subject: IMPALA-3833: Fix invalid data handling in Sequence and RCFile 
scanners
..


Patch Set 13:

(9 comments)

http://gerrit.cloudera.org:8080/#/c/8936/13//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/8936/13//COMMIT_MSG@14
PS13, Line 14:   with 100 iteration without failure.
We should run it for more than 100 iterations before committing,  I think 
there's still some chance of a rare failure slipping through otherwise.


http://gerrit.cloudera.org:8080/#/c/8936/13/be/src/exec/hdfs-rcfile-scanner.cc
File be/src/exec/hdfs-rcfile-scanner.cc:

http://gerrit.cloudera.org:8080/#/c/8936/13/be/src/exec/hdfs-rcfile-scanner.cc@86
PS13, Line 86:   const int max_ncols = 800;
Let's make this consistent with other constant - should be all caps and defined 
up the top of the file.


http://gerrit.cloudera.org:8080/#/c/8936/13/be/src/exec/hdfs-rcfile-scanner.cc@90
PS13, Line 90: ss << stream_->filename() << " Invalid file-header-metadata, 
number of columns "
I think this error message is confusing. The metadata isn't invalid, the 
problem is that we've imposed a limit on the maximum number of columns that 
Impala can handle. I think the error message needs to be rewritten to explain 
the limit. It should also include the value of the limit.


http://gerrit.cloudera.org:8080/#/c/8936/13/be/src/exec/hdfs-rcfile-scanner.cc@363
PS13, Line 363:   ss << stream_->filename() << " Bad rowcolumn index: " << 
col_idx;
Is there meant to be a space between row and column? It seems like we should 
also say something relating to corruption so that it's less cryptic. E.g. 
"Corrupt column at index: "


http://gerrit.cloudera.org:8080/#/c/8936/13/be/src/exec/hdfs-sequence-scanner.cc
File be/src/exec/hdfs-sequence-scanner.cc:

http://gerrit.cloudera.org:8080/#/c/8936/13/be/src/exec/hdfs-sequence-scanner.cc@56
PS13, Line 56:   if (codegen == nullptr) {
This DCHECK should be valid - why did this need to be changed? If we're hitting 
the DCHECK it indicates a bug with how we set up codegen and we need to fix the 
bug rather than remove the DCHECK.


http://gerrit.cloudera.org:8080/#/c/8936/11/be/src/exec/read-write-util.h
File be/src/exec/read-write-util.h:

http://gerrit.cloudera.org:8080/#/c/8936/11/be/src/exec/read-write-util.h@198
PS11, Line 198:   // Buffer access out of bounds.
> Shouldn't this be >=? Since buf[size] is one past the end of the buffer?
Looks like you missed this comment.


http://gerrit.cloudera.org:8080/#/c/8936/11/be/src/util/decompress.cc
File be/src/util/decompress.cc:

http://gerrit.cloudera.org:8080/#/c/8936/11/be/src/util/decompress.cc@452
PS11, Line 452:   if (input_len < 0) {
> Shouldn't we be checking if it's <= sizeof(uint32_t), since that's the amou
Looks like you missed this comment.


http://gerrit.cloudera.org:8080/#/c/8936/11/be/src/util/decompress.cc@454
PS11, Line 454: ss << " Corruption snappy decomp input_len " << 
input_len;
> Can we make this consistent with the other error handling in this function
Looks like you missed this comment.


http://gerrit.cloudera.org:8080/#/c/8936/13/tests/query_test/test_scanners_fuzz.py
File tests/query_test/test_scanners_fuzz.py:

http://gerrit.cloudera.org:8080/#/c/8936/13/tests/query_test/test_scanners_fuzz.py@198
PS13, Line 198: elif 'corrupt' in str(e).lower():
In theory we should just be skipping over corrupt files with abort_on_error=1. 
This doesn't always happen for a lot of file formats. We should just add rc and 
sequence file to the list of formats below on l207 that don't implement this 
skipping behaviour, rather than trying to guess based on the error message.



--
To view, visit http://gerrit.cloudera.org:8080/8936
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic9cfc38af3f30c65ada9734eb471dbfa6ecdd74a
Gerrit-Change-Number: 8936
Gerrit-PatchSet: 13
Gerrit-Owner: Pranay Singh
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Pranay Singh
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Reviewer: anujphadke 
Gerrit-Comment-Date: Wed, 11 Apr 2018 23:27:36 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6809: Allow bootstrap system.sh in non ~/Impala directory

Philip Zeyliger has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9994 )

Change subject: IMPALA-6809: Allow bootstrap_system.sh in non ~/Impala directory
..


Patch Set 3:

(3 comments)

Seems fine. I've got minor suggestions.

http://gerrit.cloudera.org:8080/#/c/9994/3/bin/bootstrap_system.sh
File bin/bootstrap_system.sh:

http://gerrit.cloudera.org:8080/#/c/9994/3/bin/bootstrap_system.sh@24
PS3, Line 24: # variables to make Impala compile and run after this script is 
complete.
Add a note here that it respects IMPALA_HOME


http://gerrit.cloudera.org:8080/#/c/9994/3/bin/bootstrap_system.sh@102
PS3, Line 102: if ! [[ -v IMPALA_HOME ]]
Impala house style for this is:

: ${IMPALA_HOME:=~/Impala}

instead of lines 102-105.


http://gerrit.cloudera.org:8080/#/c/9994/3/bin/bootstrap_system.sh@208
PS3, Line 208: if ! [[ -d $IMPALA_HOME/../Impala-lzo ]]
Define IMPALA_LZO_HOME next to where you define IMPALA_HOME and use that?



--
To view, visit http://gerrit.cloudera.org:8080/9994
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3241c180b5fb28f1b5f939200f72461ad6fd7d7a
Gerrit-Change-Number: 9994
Gerrit-PatchSet: 3
Gerrit-Owner: Fredy Wijaya 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Comment-Date: Wed, 11 Apr 2018 23:22:13 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-4631: loosen monotonic clock DCHECK

2018-04-11 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10026 )

Change subject: IMPALA-4631: loosen monotonic clock DCHECK
..


Patch Set 2:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2289/


--
To view, visit http://gerrit.cloudera.org:8080/10026
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I72d314518087aede16e8d702c2f904b679a55f6d
Gerrit-Change-Number: 10026
Gerrit-PatchSet: 2
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Wed, 11 Apr 2018 23:16:37 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-4631: loosen monotonic clock DCHECK

Tim Armstrong has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10026 )

Change subject: IMPALA-4631: loosen monotonic clock DCHECK
..


Patch Set 1:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/10026/1/be/src/runtime/fragment-instance-state.cc
File be/src/runtime/fragment-instance-state.cc:

http://gerrit.cloudera.org:8080/#/c/10026/1/be/src/runtime/fragment-instance-state.cc@323
PS1, Line 323: // TODO: IMPALA-4631: Occasionally we see other_time = 
total_time + ε where ε is 1,
> I'm worried that the epsilon will kill some editor or compiler somewhere. I
You're probably right that we can't have nice things :)


http://gerrit.cloudera.org:8080/#/c/10026/1/be/src/util/runtime-profile-counters.h
File be/src/util/runtime-profile-counters.h:

http://gerrit.cloudera.org:8080/#/c/10026/1/be/src/util/runtime-profile-counters.h@306
PS1, Line 306: // (start_time_ns - ε), where ε is 1, 2 or 3 even though 
'start_time_ns' was
> same here
Done



--
To view, visit http://gerrit.cloudera.org:8080/10026
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I72d314518087aede16e8d702c2f904b679a55f6d
Gerrit-Change-Number: 10026
Gerrit-PatchSet: 1
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Wed, 11 Apr 2018 23:16:28 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-4631: loosen monotonic clock DCHECK

Hello Philip Zeyliger,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/10026

to look at the new patch set (#2).

Change subject: IMPALA-4631: loosen monotonic clock DCHECK
..

IMPALA-4631: loosen monotonic clock DCHECK

We saw another build failure due to hitting one of these DCHECKs.
Let's further loosen the check to avoid the failures on misbehaving
systems.

Change-Id: I72d314518087aede16e8d702c2f904b679a55f6d
---
M be/src/runtime/fragment-instance-state.cc
M be/src/util/runtime-profile-counters.h
2 files changed, 7 insertions(+), 6 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/26/10026/2
--
To view, visit http://gerrit.cloudera.org:8080/10026
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I72d314518087aede16e8d702c2f904b679a55f6d
Gerrit-Change-Number: 10026
Gerrit-PatchSet: 2
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Philip Zeyliger

[Impala-ASF-CR] Add --partial ok to compare branches.py.

Philip Zeyliger has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10025 )

Change subject: Add --partial_ok to compare_branches.py.
..


Patch Set 1: Verified+1

GVD is not helpful with this script, so I'm self-verifying. I'm also about to 
cherrypick it to 2.x.


--
To view, visit http://gerrit.cloudera.org:8080/10025
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2a8b34577f9cb74565adf90a2b7d5328bc555f85
Gerrit-Change-Number: 10025
Gerrit-PatchSet: 1
Gerrit-Owner: Philip Zeyliger 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Comment-Date: Wed, 11 Apr 2018 22:54:48 +
Gerrit-HasComments: No

[Impala-ASF-CR](2.x) Add --partial ok to compare branches.py.

Philip Zeyliger has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/10027


Change subject: Add --partial_ok to compare_branches.py.
..

Add --partial_ok to compare_branches.py.

This change lets compare_branches.py succeed with a partial set of
cherry-picks. In our case, we sometimes get baacked up with several
commits needing to be cherry-picked. The first few cherry-pick fine, and
there's a problematic commit sometime down the line. By accepting the
first few that cherry-pick fine, someone resolving the conflict can
start at where that conflict begins, rather than having to wrangle more
commits unnecessarily.

I tested this with and without the flag, and confirmed that if the
first commit is problematic for cherry-picks, the command does
fail.

Change-Id: I2a8b34577f9cb74565adf90a2b7d5328bc555f85
Reviewed-on: http://gerrit.cloudera.org:8080/10025
Reviewed-by: Joe McDonnell 
Tested-by: Philip Zeyliger 
(cherry picked from commit a1babea113d46fb69bd4f38b005f9ee02eada200)
---
M bin/compare_branches.py
1 file changed, 16 insertions(+), 5 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/27/10027/1
--
To view, visit http://gerrit.cloudera.org:8080/10027
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: 2.x
Gerrit-MessageType: newchange
Gerrit-Change-Id: I2a8b34577f9cb74565adf90a2b7d5328bc555f85
Gerrit-Change-Number: 10027
Gerrit-PatchSet: 1
Gerrit-Owner: Philip Zeyliger

[Impala-ASF-CR](2.x) Add --partial ok to compare branches.py.

Philip Zeyliger has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/10027 )

Change subject: Add --partial_ok to compare_branches.py.
..

Add --partial_ok to compare_branches.py.

This change lets compare_branches.py succeed with a partial set of
cherry-picks. In our case, we sometimes get baacked up with several
commits needing to be cherry-picked. The first few cherry-pick fine, and
there's a problematic commit sometime down the line. By accepting the
first few that cherry-pick fine, someone resolving the conflict can
start at where that conflict begins, rather than having to wrangle more
commits unnecessarily.

I tested this with and without the flag, and confirmed that if the
first commit is problematic for cherry-picks, the command does
fail.

Change-Id: I2a8b34577f9cb74565adf90a2b7d5328bc555f85
Reviewed-on: http://gerrit.cloudera.org:8080/10025
Reviewed-by: Joe McDonnell 
Tested-by: Philip Zeyliger 
(cherry picked from commit a1babea113d46fb69bd4f38b005f9ee02eada200)
Reviewed-on: http://gerrit.cloudera.org:8080/10027
Reviewed-by: Philip Zeyliger 
---
M bin/compare_branches.py
1 file changed, 16 insertions(+), 5 deletions(-)

Approvals:
  Philip Zeyliger: Looks good to me, approved; Verified

--
To view, visit http://gerrit.cloudera.org:8080/10027
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: 2.x
Gerrit-MessageType: merged
Gerrit-Change-Id: I2a8b34577f9cb74565adf90a2b7d5328bc555f85
Gerrit-Change-Number: 10027
Gerrit-PatchSet: 2
Gerrit-Owner: Philip Zeyliger 
Gerrit-Reviewer: Philip Zeyliger

[Impala-ASF-CR](2.x) Add --partial ok to compare branches.py.

Philip Zeyliger has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10027 )

Change subject: Add --partial_ok to compare_branches.py.
..


Patch Set 1: Verified+1 Code-Review+2

Clean pick.


--
To view, visit http://gerrit.cloudera.org:8080/10027
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: 2.x
Gerrit-MessageType: comment
Gerrit-Change-Id: I2a8b34577f9cb74565adf90a2b7d5328bc555f85
Gerrit-Change-Number: 10027
Gerrit-PatchSet: 1
Gerrit-Owner: Philip Zeyliger 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Comment-Date: Wed, 11 Apr 2018 22:55:07 +
Gerrit-HasComments: No

[Impala-ASF-CR] Add --partial ok to compare branches.py.

Philip Zeyliger has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/10025 )

Change subject: Add --partial_ok to compare_branches.py.
..

Add --partial_ok to compare_branches.py.

This change lets compare_branches.py succeed with a partial set of
cherry-picks. In our case, we sometimes get baacked up with several
commits needing to be cherry-picked. The first few cherry-pick fine, and
there's a problematic commit sometime down the line. By accepting the
first few that cherry-pick fine, someone resolving the conflict can
start at where that conflict begins, rather than having to wrangle more
commits unnecessarily.

I tested this with and without the flag, and confirmed that if the
first commit is problematic for cherry-picks, the command does
fail.

Change-Id: I2a8b34577f9cb74565adf90a2b7d5328bc555f85
Reviewed-on: http://gerrit.cloudera.org:8080/10025
Reviewed-by: Joe McDonnell 
Tested-by: Philip Zeyliger 
---
M bin/compare_branches.py
1 file changed, 16 insertions(+), 5 deletions(-)

Approvals:
  Joe McDonnell: Looks good to me, approved
  Philip Zeyliger: Verified

--
To view, visit http://gerrit.cloudera.org:8080/10025
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I2a8b34577f9cb74565adf90a2b7d5328bc555f85
Gerrit-Change-Number: 10025
Gerrit-PatchSet: 2
Gerrit-Owner: Philip Zeyliger 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Philip Zeyliger

[Impala-ASF-CR] IMPALA-4631: loosen monotonic clock DCHECK

Philip Zeyliger has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10026 )

Change subject: IMPALA-4631: loosen monotonic clock DCHECK
..


Patch Set 1: Code-Review+2

(2 comments)

Weird. +2 but see my s/e/eps/ suggestion.

http://gerrit.cloudera.org:8080/#/c/10026/1/be/src/runtime/fragment-instance-state.cc
File be/src/runtime/fragment-instance-state.cc:

http://gerrit.cloudera.org:8080/#/c/10026/1/be/src/runtime/fragment-instance-state.cc@323
PS1, Line 323: // TODO: IMPALA-4631: Occasionally we see other_time = 
total_time + ε where ε is 1,
I'm worried that the epsilon will kill some editor or compiler somewhere. If 
you know we do this in .cc files already, sure, but if this is the first time, 
I'd replace with "eps" or "epsilon".


http://gerrit.cloudera.org:8080/#/c/10026/1/be/src/util/runtime-profile-counters.h
File be/src/util/runtime-profile-counters.h:

http://gerrit.cloudera.org:8080/#/c/10026/1/be/src/util/runtime-profile-counters.h@306
PS1, Line 306: // (start_time_ns - ε), where ε is 1, 2 or 3 even though 
'start_time_ns' was
same here



--
To view, visit http://gerrit.cloudera.org:8080/10026
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I72d314518087aede16e8d702c2f904b679a55f6d
Gerrit-Change-Number: 10026
Gerrit-PatchSet: 1
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Comment-Date: Wed, 11 Apr 2018 22:53:57 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-2717: fix output of formatted unicode to non-TTY

Tim Armstrong has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9928 )

Change subject: IMPALA-2717: fix output of formatted unicode to non-TTY
..


Patch Set 4:

Could you have a look at this Mike? Would be good to validate that our use of 
Python strings is correct.


--
To view, visit http://gerrit.cloudera.org:8080/9928
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9de641ecf767a2feef3b9f48b344ef2d55e17a7f
Gerrit-Change-Number: 9928
Gerrit-PatchSet: 4
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Michael Brown 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Wed, 11 Apr 2018 22:50:51 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-2717: fix output of formatted unicode to non-TTY

Tim Armstrong has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9928 )

Change subject: IMPALA-2717: fix output of formatted unicode to non-TTY
..


Patch Set 4: Code-Review+1

carry


--
To view, visit http://gerrit.cloudera.org:8080/9928
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9de641ecf767a2feef3b9f48b344ef2d55e17a7f
Gerrit-Change-Number: 9928
Gerrit-PatchSet: 4
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Wed, 11 Apr 2018 22:50:13 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-2717: fix output of formatted unicode to non-TTY

Hello Lars Volker,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/9928

to look at the new patch set (#4).

Change subject: IMPALA-2717: fix output of formatted unicode to non-TTY
..

IMPALA-2717: fix output of formatted unicode to non-TTY

The bug is that PrettyOutputFormatter.format() returned a unicode
object, and Python cannot automatically write unicode objects to
output streams where there is no default encoding.

The fix is to convert to UTF-8 encoded in a regular string, which
can be output to any output device. This makes the output type
consistent with DelimitedOutputFormatter.format().

Based on code by Marcell Szabo.

Testing:
Added a basic test.

Played around in an interactive shell to make sure that unicode
characters still work in interactive mode.

Change-Id: I9de641ecf767a2feef3b9f48b344ef2d55e17a7f
---
M shell/impala_shell.py
M shell/shell_output.py
M tests/shell/test_shell_commandline.py
3 files changed, 31 insertions(+), 6 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/28/9928/4
--
To view, visit http://gerrit.cloudera.org:8080/9928
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I9de641ecf767a2feef3b9f48b344ef2d55e17a7f
Gerrit-Change-Number: 9928
Gerrit-PatchSet: 4
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Tim Armstrong

[Impala-ASF-CR] IMPALA-2717: fix output of formatted unicode to non-TTY

Tim Armstrong has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9928 )

Change subject: IMPALA-2717: fix output of formatted unicode to non-TTY
..


Patch Set 3:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/9928/3/shell/shell_output.py
File shell/shell_output.py:

http://gerrit.cloudera.org:8080/#/c/9928/3/shell/shell_output.py@61
PS3, Line 61:
> nit: remove space. Apologies for missing this one earlier.
Done



--
To view, visit http://gerrit.cloudera.org:8080/9928
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9de641ecf767a2feef3b9f48b344ef2d55e17a7f
Gerrit-Change-Number: 9928
Gerrit-PatchSet: 3
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Wed, 11 Apr 2018 22:50:05 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-2717: fix output of formatted unicode to non-TTY

2018-04-11 Thread Lars Volker (Code Review)

Lars Volker has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9928 )

Change subject: IMPALA-2717: fix output of formatted unicode to non-TTY
..


Patch Set 3: Code-Review+1

(1 comment)

Thank you for fixing the open TODO, too!

http://gerrit.cloudera.org:8080/#/c/9928/3/shell/shell_output.py
File shell/shell_output.py:

http://gerrit.cloudera.org:8080/#/c/9928/3/shell/shell_output.py@61
PS3, Line 61:
nit: remove space. Apologies for missing this one earlier.



--
To view, visit http://gerrit.cloudera.org:8080/9928
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9de641ecf767a2feef3b9f48b344ef2d55e17a7f
Gerrit-Change-Number: 9928
Gerrit-PatchSet: 3
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Wed, 11 Apr 2018 22:48:02 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-4631: loosen monotonic clock DCHECK

Tim Armstrong has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/10026


Change subject: IMPALA-4631: loosen monotonic clock DCHECK
..

IMPALA-4631: loosen monotonic clock DCHECK

We saw another build failure due to hitting one of these DCHECKs.
Let's further loosen the check to avoid the failures on misbehaving
systems.

Change-Id: I72d314518087aede16e8d702c2f904b679a55f6d
---
M be/src/runtime/fragment-instance-state.cc
M be/src/util/runtime-profile-counters.h
2 files changed, 7 insertions(+), 6 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/26/10026/1
--
To view, visit http://gerrit.cloudera.org:8080/10026
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I72d314518087aede16e8d702c2f904b679a55f6d
Gerrit-Change-Number: 10026
Gerrit-PatchSet: 1
Gerrit-Owner: Tim Armstrong

[Impala-ASF-CR] Add --partial ok to compare branches.py.

Joe McDonnell has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10025 )

Change subject: Add --partial_ok to compare_branches.py.
..


Patch Set 1: Code-Review+2

Thanks for doing this!


--
To view, visit http://gerrit.cloudera.org:8080/10025
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2a8b34577f9cb74565adf90a2b7d5328bc555f85
Gerrit-Change-Number: 10025
Gerrit-PatchSet: 1
Gerrit-Owner: Philip Zeyliger 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Comment-Date: Wed, 11 Apr 2018 22:43:36 +
Gerrit-HasComments: No

[Impala-ASF-CR] Add --partial ok to compare branches.py.

Philip Zeyliger has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/10025


Change subject: Add --partial_ok to compare_branches.py.
..

Add --partial_ok to compare_branches.py.

This change lets compare_branches.py succeed with a partial set of
cherry-picks. In our case, we sometimes get baacked up with several
commits needing to be cherry-picked. The first few cherry-pick fine, and
there's a problematic commit sometime down the line. By accepting the
first few that cherry-pick fine, someone resolving the conflict can
start at where that conflict begins, rather than having to wrangle more
commits unnecessarily.

I tested this with and without the flag, and confirmed that if the
first commit is problematic for cherry-picks, the command does
fail.

Change-Id: I2a8b34577f9cb74565adf90a2b7d5328bc555f85
---
M bin/compare_branches.py
1 file changed, 16 insertions(+), 5 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/25/10025/1
--
To view, visit http://gerrit.cloudera.org:8080/10025
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I2a8b34577f9cb74565adf90a2b7d5328bc555f85
Gerrit-Change-Number: 10025
Gerrit-PatchSet: 1
Gerrit-Owner: Philip Zeyliger

[Impala-ASF-CR] IMPALA-6120: Add thread timers for reporting codegen time

Tim Armstrong has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9960 )

Change subject: IMPALA-6120: Add thread timers for reporting codegen time
..


Patch Set 4: Code-Review+2

(1 comment)

http://gerrit.cloudera.org:8080/#/c/9960/4/be/src/codegen/llvm-codegen.h
File be/src/codegen/llvm-codegen.h:

http://gerrit.cloudera.org:8080/#/c/9960/4/be/src/codegen/llvm-codegen.h@758
PS4, Line 758: state
Is OPEN a phase or a state? Not sure what terminology we use, or if we're 
consistent.



--
To view, visit http://gerrit.cloudera.org:8080/9960
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I24d5a46b8870bc959b89045432d2e86af72b30e5
Gerrit-Change-Number: 9960
Gerrit-PatchSet: 4
Gerrit-Owner: Bikramjeet Vig 
Gerrit-Reviewer: Bikramjeet Vig 
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Wed, 11 Apr 2018 22:27:25 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6314: Add run time scalar subquery check for uncorrelated subqueries

Alex Behm has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9005 )

Change subject: IMPALA-6314: Add run time scalar subquery check for 
uncorrelated subqueries
..


Patch Set 21:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/9005/21/fe/src/main/java/org/apache/impala/analysis/StmtRewriter.java
File fe/src/main/java/org/apache/impala/analysis/StmtRewriter.java:

http://gerrit.cloudera.org:8080/#/c/9005/21/fe/src/main/java/org/apache/impala/analysis/StmtRewriter.java@1025
PS21, Line 1025: if (!subquery.getType().isScalarType()) {
> Right. That could be an indication that this check is useless or that we ha
I looked into this and concluded that we should convert the type check to a 
Precondition.

However, I also found some weirdness in our error checking, maybe you can fix 
that also. The error message for the following query doesn't quite make sense:

select * from functional.alltypes t1 where id < (select id from 
functional.alltypes t2 where t1.int_col = t2.int_col)

UDAs are not supported in the select list of correlated subqueries: (SELECT id 
FROM functional.alltypes t2 WHERE t1.int_col = t2.int_col)



--
To view, visit http://gerrit.cloudera.org:8080/9005
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I0f52b93a60eeacedd242a2f17fa6b99c4fc38e06
Gerrit-Change-Number: 9005
Gerrit-PatchSet: 21
Gerrit-Owner: Zoltan Borok-Nagy 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Attila Jeges 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Dimitris Tsirogiannis 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Reviewer: Vuk Ercegovac 
Gerrit-Reviewer: Zoltan Borok-Nagy 
Gerrit-Comment-Date: Wed, 11 Apr 2018 22:15:34 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6120: Add thread timers for reporting codegen time

Hello Michael Ho, Tim Armstrong,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/9960

to look at the new patch set (#4).

Change subject: IMPALA-6120: Add thread timers for reporting codegen time
..

IMPALA-6120: Add thread timers for reporting codegen time

Add thread times for accurate reporting of codegen time.
Also cleaned up a few places where time elapsed was being counted twice.

Sample Profile:

Query: SELECT count(*) FROM tpch_parquet.lineitem
WHERE l_partkey in (1,6,11,16,21,26,31,36,41);

CodeGen:(Total: 37.948ms, non-child: 37.948ms, % non-child: 100.00%)
   - CodegenInvoluntaryContextSwitches: 0 (0)
   - CodegenTotalWallClockTime: 37.942ms
 - CodegenSysTime: 0.000ns
 - CodegenUserTime: 36.938ms
   - CodegenVoluntaryContextSwitches: 0 (0)
   - CompileTime: 2.065ms
   - IrGenerationTime: 392.351us
   - LoadTime: 0.000ns
   - ModuleBitcodeSize: 2.26 MB (2373148)
   - NumFunctions: 22 (22)
   - NumInstructions: 381 (381)
   - OptimizationTime: 21.416ms
   - PeakMemoryUsage: 190.50 KB (195072)
   - PrepareTime: 13.496ms

Sample Profile with an added 2 sec sleep time to "OptimizationTime":

CodeGen:(Total: 2s037ms, non-child: 2s037ms, % non-child: 100.00%)
   - CodegenInvoluntaryContextSwitches: 0 (0)
   - CodegenTotalWallClockTime: 2s037ms
 - CodegenSysTime: 0.000ns
 - CodegenUserTime: 37.672ms
   - CodegenVoluntaryContextSwitches: 1 (1)
   - CompileTime: 2.032ms
   - IrGenerationTime: 386.948us
   - LoadTime: 0.000ns
   - ModuleBitcodeSize: 2.26 MB (2373148)
   - NumFunctions: 22 (22)
   - NumInstructions: 381 (381)
   - OptimizationTime: 2s023ms
   - PeakMemoryUsage: 190.50 KB (195072)
   - PrepareTime: 11.598ms

Change-Id: I24d5a46b8870bc959b89045432d2e86af72b30e5
---
M be/src/codegen/llvm-codegen.cc
M be/src/codegen/llvm-codegen.h
M be/src/exec/hdfs-avro-scanner.cc
M be/src/exec/hdfs-parquet-scanner.cc
M be/src/exec/hdfs-scanner.cc
M be/src/exec/partitioned-aggregation-node.cc
M be/src/exec/select-node.cc
M be/src/exec/text-converter.cc
M be/src/runtime/fragment-instance-state.cc
M be/src/runtime/tuple.cc
M be/src/util/tuple-row-compare.cc
11 files changed, 26 insertions(+), 35 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/60/9960/4
--
To view, visit http://gerrit.cloudera.org:8080/9960
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I24d5a46b8870bc959b89045432d2e86af72b30e5
Gerrit-Change-Number: 9960
Gerrit-PatchSet: 4
Gerrit-Owner: Bikramjeet Vig 
Gerrit-Reviewer: Bikramjeet Vig 
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Tim Armstrong

[Impala-ASF-CR] IMPALA-6120: Add thread timers for reporting codegen time

Bikramjeet Vig has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9960 )

Change subject: IMPALA-6120: Add thread timers for reporting codegen time
..


Patch Set 2:

(4 comments)

http://gerrit.cloudera.org:8080/#/c/9960/1/be/src/codegen/llvm-codegen.h
File be/src/codegen/llvm-codegen.h:

http://gerrit.cloudera.org:8080/#/c/9960/1/be/src/codegen/llvm-codegen.h@761
PS1, Line 761:   RuntimeProfile::Counter* optimization_timer_;
> If we switch to having the counter updated outside of this class, lets docu
Done


http://gerrit.cloudera.org:8080/#/c/9960/1/be/src/codegen/llvm-codegen.h@779
PS1, Line 779:
> If we switch to having the counter updated outside of this class, lets docu
Done


http://gerrit.cloudera.org:8080/#/c/9960/1/be/src/codegen/llvm-codegen.cc
File be/src/codegen/llvm-codegen.cc:

http://gerrit.cloudera.org:8080/#/c/9960/1/be/src/codegen/llvm-codegen.cc@204
PS1, Line 204: CodegenTime
> Yeah IrGenerationTime or something like that would be more accurate
Done


http://gerrit.cloudera.org:8080/#/c/9960/1/be/src/codegen/llvm-codegen.cc@216
PS1, Line 216:   SCOPED_TIMER((*codegen)->profile_->total_time_counter());
 :   
SCOPED_THREAD_COUNTER_MEASUREMENT((*codegen)->llvm_thread_counters());
> I think we should keep this consistent with CreateFromMemory() to minimise
got it. I ll leave this in and remove the timer from LinkModuleFromLocalFs()



--
To view, visit http://gerrit.cloudera.org:8080/9960
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I24d5a46b8870bc959b89045432d2e86af72b30e5
Gerrit-Change-Number: 9960
Gerrit-PatchSet: 2
Gerrit-Owner: Bikramjeet Vig 
Gerrit-Reviewer: Bikramjeet Vig 
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Wed, 11 Apr 2018 22:12:42 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6120: Add thread timers for reporting codegen time

Hello Michael Ho, Tim Armstrong,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/9960

to look at the new patch set (#3).

Change subject: IMPALA-6120: Add thread timers for reporting codegen time
..

IMPALA-6120: Add thread timers for reporting codegen time

Add thread times for accurate reporting of codegen time.
Also cleaned up a few places where time elapsed was being counted twice.

Sample Profile:

Query: SELECT count(*) FROM tpch_parquet.lineitem
WHERE l_partkey in (1,6,11,16,21,26,31,36,41);

CodeGen:(Total: 37.948ms, non-child: 37.948ms, % non-child: 100.00%)
   - CodegenInvoluntaryContextSwitches: 0 (0)
   - CodegenTotalWallClockTime: 37.942ms
 - CodegenSysTime: 0.000ns
 - CodegenUserTime: 36.938ms
   - CodegenVoluntaryContextSwitches: 0 (0)
   - CompileTime: 2.065ms
   - IrGenerationTime: 392.351us
   - LoadTime: 0.000ns
   - ModuleBitcodeSize: 2.26 MB (2373148)
   - NumFunctions: 22 (22)
   - NumInstructions: 381 (381)
   - OptimizationTime: 21.416ms
   - PeakMemoryUsage: 190.50 KB (195072)
   - PrepareTime: 13.496ms

Sample Profile with an added 2 sec sleep time to "OptimizationTime":

CodeGen:(Total: 2s037ms, non-child: 2s037ms, % non-child: 100.00%)
   - CodegenInvoluntaryContextSwitches: 0 (0)
   - CodegenTotalWallClockTime: 2s037ms
 - CodegenSysTime: 0.000ns
 - CodegenUserTime: 37.672ms
   - CodegenVoluntaryContextSwitches: 1 (1)
   - CompileTime: 2.032ms
   - IrGenerationTime: 386.948us
   - LoadTime: 0.000ns
   - ModuleBitcodeSize: 2.26 MB (2373148)
   - NumFunctions: 22 (22)
   - NumInstructions: 381 (381)
   - OptimizationTime: 2s023ms
   - PeakMemoryUsage: 190.50 KB (195072)
   - PrepareTime: 11.598ms

Change-Id: I24d5a46b8870bc959b89045432d2e86af72b30e5
---
M be/src/codegen/llvm-codegen.cc
M be/src/codegen/llvm-codegen.h
M be/src/exec/hdfs-avro-scanner.cc
M be/src/exec/hdfs-parquet-scanner.cc
M be/src/exec/hdfs-scanner.cc
M be/src/exec/partitioned-aggregation-node.cc
M be/src/exec/select-node.cc
M be/src/exec/text-converter.cc
M be/src/runtime/fragment-instance-state.cc
M be/src/runtime/tuple.cc
M be/src/util/tuple-row-compare.cc
11 files changed, 27 insertions(+), 34 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/60/9960/3
--
To view, visit http://gerrit.cloudera.org:8080/9960
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I24d5a46b8870bc959b89045432d2e86af72b30e5
Gerrit-Change-Number: 9960
Gerrit-PatchSet: 3
Gerrit-Owner: Bikramjeet Vig 
Gerrit-Reviewer: Bikramjeet Vig 
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Tim Armstrong

[Impala-ASF-CR] IMPALA-5717: Support for reading ORC data files

2018-04-11 Thread Quanlong Huang (Code Review)

Quanlong Huang has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9134 )

Change subject: IMPALA-5717: Support for reading ORC data files
..


Patch Set 21:

Thank you, Tim, Dan, and Joe! I do appreciate your review. Hopes we can work 
together to move forward the orc-support feature!


--
To view, visit http://gerrit.cloudera.org:8080/9134
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia7b6ae4ce3b9ee8125b21993702faa87537790a4
Gerrit-Change-Number: 9134
Gerrit-PatchSet: 21
Gerrit-Owner: Quanlong Huang 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Wed, 11 Apr 2018 22:06:05 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6623: [DOCS] ltrim and rtrim docs updated

2018-04-11 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/9984 )

Change subject: IMPALA-6623: [DOCS] ltrim and rtrim docs updated
..

IMPALA-6623: [DOCS] ltrim and rtrim docs updated

Change-Id: If4f7a04e3c64eade7a23cded21de5ff91c9c8c8c
Reviewed-on: http://gerrit.cloudera.org:8080/9984
Reviewed-by: Zoram Thanga 
Reviewed-by: Alex Rodoni 
Tested-by: Impala Public Jenkins 
---
M docs/topics/impala_string_functions.xml
1 file changed, 8 insertions(+), 4 deletions(-)

Approvals:
  Zoram Thanga: Looks good to me, but someone else must approve
  Alex Rodoni: Looks good to me, approved
  Impala Public Jenkins: Verified

--
To view, visit http://gerrit.cloudera.org:8080/9984
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: If4f7a04e3c64eade7a23cded21de5ff91c9c8c8c
Gerrit-Change-Number: 9984
Gerrit-PatchSet: 2
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Zoram Thanga

[Impala-ASF-CR] IMPALA-6623: [DOCS] ltrim and rtrim docs updated

2018-04-11 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9984 )

Change subject: IMPALA-6623: [DOCS] ltrim and rtrim docs updated
..


Patch Set 1: Verified+1


--
To view, visit http://gerrit.cloudera.org:8080/9984
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If4f7a04e3c64eade7a23cded21de5ff91c9c8c8c
Gerrit-Change-Number: 9984
Gerrit-PatchSet: 1
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Zoram Thanga 
Gerrit-Comment-Date: Wed, 11 Apr 2018 22:02:15 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6793: Fix empty metadata after statestore restarts

2018-04-11 Thread Michael Brown (Code Review)

Michael Brown has posted comments on this change. (
http://gerrit.cloudera.org:8080/9921 )

Change subject: IMPALA-6793: Fix empty metadata after statestore restarts
..

Patch Set 7:

(5 comments)

I'm kind of in the same mode as Alex, and I'll to avoid rehashing his concerns.

http://gerrit.cloudera.org:8080/#/c/9921/7/tests/custom_cluster/test_restart_services.py
File tests/custom_cluster/test_restart_services.py:

http://gerrit.cloudera.org:8080/#/c/9921/7/tests/custom_cluster/test_restart_services.py@38
PS7, Line 38: sleep(60)
If this is expected to differ greatly in ASAN builds, consider using
specific_build_type_timeout() .

It's also unfortunate you're not able to poll here. Can you at least add a
comment that says why you can't do so? From my reading of the long discussion
about this test, it seems that there's no metric suitable.

http://gerrit.cloudera.org:8080/#/c/9921/7/tests/custom_cluster/test_restart_services.py@39
PS7, Line 39: range(0, 10)
Python trivia that's for your info that you do or don't have to fix, up to you.

1. This can just be range(10).

2. You can also switch to using xrange(10) which does lazy evaluation of each
element. In this case, it's a micro-optimization.

http://gerrit.cloudera.org:8080/#/c/9921/7/tests/custom_cluster/test_restart_services.py@43
PS7, Line 43: except HiveServer2Error:
Is any HiveServer2Error acceptable, or just an AnalysisException in which
functional does not exist? If the latter, maybe inspect the exception to make
sure that's what you got, and fail the test if you got something else.

http://gerrit.cloudera.org:8080/#/c/9921/7/tests/custom_cluster/test_restart_services.py@44
PS7, Line 44: sleep(10)
Where does 10s come from? Why not 1?

http://gerrit.cloudera.org:8080/#/c/9921/7/tests/custom_cluster/test_restart_services.py@45
PS7, Line 45: assert False
How about something a tad more descriptive, like:

assert False, "Coordinator never received metadata from restarted statestore
after {0} seconds or
iterations".format(calculation_seconds_or_iterations_or_similar)

--
To view, visit http://gerrit.cloudera.org:8080/9921
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I332a60e172af84b93b3544373fe363cdced5e8d0
Gerrit-Change-Number: 9921
Gerrit-PatchSet: 7
Gerrit-Owner: Tianyi Wang
Gerrit-Reviewer: Alex Behm
Gerrit-Reviewer: Dimitris Tsirogiannis
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Michael Brown
Gerrit-Reviewer: Philip Zeyliger
Gerrit-Reviewer: Tianyi Wang
Gerrit-Reviewer: Vuk Ercegovac
Gerrit-Comment-Date: Wed, 11 Apr 2018 21:58:17 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-2717: fix output of formatted unicode to non-TTY

Tim Armstrong has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9928 )

Change subject: IMPALA-2717: fix output of formatted unicode to non-TTY
..


Patch Set 2:

(4 comments)

http://gerrit.cloudera.org:8080/#/c/9928/1/shell/impala_shell.py
File shell/impala_shell.py:

http://gerrit.cloudera.org:8080/#/c/9928/1/shell/impala_shell.py@73
PS1, Line 73:   """Patched version of PrettyTable that TODO"""
> Not your change, but would you mind filling in the TODO? The difference see
Hah oh dear.


http://gerrit.cloudera.org:8080/#/c/9928/1/shell/shell_output.py
File shell/shell_output.py:

http://gerrit.cloudera.org:8080/#/c/9928/1/shell/shell_output.py@45
PS1, Line 45: row
> I looked around a bit and tried to figure out how passing a str into add_ro
I extended the current comment to explain the two-way conversion.


http://gerrit.cloudera.org:8080/#/c/9928/2/shell/shell_output.py
File shell/shell_output.py:

http://gerrit.cloudera.org:8080/#/c/9928/2/shell/shell_output.py@31
PS2, Line 31: contained
> nit: containing?
Done


http://gerrit.cloudera.org:8080/#/c/9928/2/shell/shell_output.py@58
PS2, Line 58: contained
> here too?
Done



--
To view, visit http://gerrit.cloudera.org:8080/9928
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9de641ecf767a2feef3b9f48b344ef2d55e17a7f
Gerrit-Change-Number: 9928
Gerrit-PatchSet: 2
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Wed, 11 Apr 2018 21:57:20 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-2717: fix output of formatted unicode to non-TTY

Hello Lars Volker,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/9928

to look at the new patch set (#3).

Change subject: IMPALA-2717: fix output of formatted unicode to non-TTY
..

IMPALA-2717: fix output of formatted unicode to non-TTY

The bug is that PrettyOutputFormatter.format() returned a unicode
object, and Python cannot automatically write unicode objects to
output streams where there is no default encoding.

The fix is to convert to UTF-8 encoded in a regular string, which
can be output to any output device. This makes the output type
consistent with DelimitedOutputFormatter.format().

Based on code by Marcell Szabo.

Testing:
Added a basic test.

Played around in an interactive shell to make sure that unicode
characters still work in interactive mode.

Change-Id: I9de641ecf767a2feef3b9f48b344ef2d55e17a7f
---
M shell/impala_shell.py
M shell/shell_output.py
M tests/shell/test_shell_commandline.py
3 files changed, 31 insertions(+), 6 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/28/9928/3
--
To view, visit http://gerrit.cloudera.org:8080/9928
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I9de641ecf767a2feef3b9f48b344ef2d55e17a7f
Gerrit-Change-Number: 9928
Gerrit-PatchSet: 3
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Tim Armstrong

[Impala-ASF-CR] IMPALA-2195: Improper handling of comments in queries

2018-04-11 Thread Bharath Vissapragada (Code Review)

Bharath Vissapragada has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9933 )

Change subject: IMPALA-2195: Improper handling of comments in queries
..


Patch Set 5:

Apologies for the delay on this and blocking your progress. I'm a little busy 
this week and didn't get enough time to dig into the latest round of comments. 
I'm ok if some other reviewer wants to drive this, if not I can get back by 
Friday or so.


--
To view, visit http://gerrit.cloudera.org:8080/9933
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7ac7cb5a30e6dda73ebe761d9f0eb9ba038e14a7
Gerrit-Change-Number: 9933
Gerrit-PatchSet: 5
Gerrit-Owner: Fredy Wijaya 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Bharath Vissapragada 
Gerrit-Reviewer: David Knupp 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Taras Bobrovytsky 
Gerrit-Comment-Date: Wed, 11 Apr 2018 21:52:54 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6710: [DOCS] Update the Partition Insert content

2018-04-11 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/9977 )

Change subject: IMPALA-6710: [DOCS] Update the Partition Insert content
..

IMPALA-6710: [DOCS] Update the Partition Insert content

Added a section at the end for inserting into partitioned tables.

Change-Id: I4ccc8227579dabc321a949da95e8a59158528f20
Reviewed-on: http://gerrit.cloudera.org:8080/9977
Reviewed-by: Thomas Tauber-Marshall 
Tested-by: Impala Public Jenkins 
---
M docs/topics/impala_insert.xml
1 file changed, 173 insertions(+), 84 deletions(-)

Approvals:
  Thomas Tauber-Marshall: Looks good to me, approved
  Impala Public Jenkins: Verified

--
To view, visit http://gerrit.cloudera.org:8080/9977
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I4ccc8227579dabc321a949da95e8a59158528f20
Gerrit-Change-Number: 9977
Gerrit-PatchSet: 2
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Thomas Tauber-Marshall

[Impala-ASF-CR] IMPALA-6710: [DOCS] Update the Partition Insert content

2018-04-11 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9977 )

Change subject: IMPALA-6710: [DOCS] Update the Partition Insert content
..


Patch Set 1: Verified+1


--
To view, visit http://gerrit.cloudera.org:8080/9977
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4ccc8227579dabc321a949da95e8a59158528f20
Gerrit-Change-Number: 9977
Gerrit-PatchSet: 1
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Thomas Tauber-Marshall 
Gerrit-Comment-Date: Wed, 11 Apr 2018 21:51:56 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6811: Increase timeout on S3 for test exchange delays

2018-04-11 Thread Lars Volker (Code Review)

Lars Volker has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9995 )

Change subject: IMPALA-6811: Increase timeout on S3 for test_exchange_delays
..


Patch Set 3: Code-Review+1


--
To view, visit http://gerrit.cloudera.org:8080/9995
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I967e6eb336c801219c77d657655c42984910b479
Gerrit-Change-Number: 9995
Gerrit-PatchSet: 3
Gerrit-Owner: Joe McDonnell 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Michael Brown 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Vuk Ercegovac 
Gerrit-Comment-Date: Wed, 11 Apr 2018 21:45:10 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6314: Add run time scalar subquery check for uncorrelated subqueries

Alex Behm has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9005 )

Change subject: IMPALA-6314: Add run time scalar subquery check for 
uncorrelated subqueries
..


Patch Set 23:

(3 comments)

Code looks pretty good now. I'll do another pass over everything focusing on 
the tests.

http://gerrit.cloudera.org:8080/#/c/9005/21/fe/src/main/java/org/apache/impala/analysis/StmtRewriter.java
File fe/src/main/java/org/apache/impala/analysis/StmtRewriter.java:

http://gerrit.cloudera.org:8080/#/c/9005/21/fe/src/main/java/org/apache/impala/analysis/StmtRewriter.java@1025
PS21, Line 1025: if (isCorrelated && 
!item.getExpr().contains(Expr.IS_BUILTIN_AGG_FN)) {
> This allows runtime scalars here, because their type is scalar now and not
Right. That could be an indication that this check is useless or that we have a 
gap in test coverage. I'm worried about the latter. One of two outcomes is 
acceptable:
* add a test that hits this branch
* convince ourselves that the type is always scalar here, and then add a 
Preconditions check


http://gerrit.cloudera.org:8080/#/c/9005/23/fe/src/test/java/org/apache/impala/analysis/ToSqlTest.java
File fe/src/test/java/org/apache/impala/analysis/ToSqlTest.java:

http://gerrit.cloudera.org:8080/#/c/9005/23/fe/src/test/java/org/apache/impala/analysis/ToSqlTest.java@82
PS23, Line 82:   if (node instanceof StatementBase) {
No need for this. We're always going to have a StatementBase


http://gerrit.cloudera.org:8080/#/c/9005/23/fe/src/test/java/org/apache/impala/analysis/ToSqlTest.java@917
PS23, Line 917: //TODO: make it work again or delete
  : //testToSql("select * from functional.alltypes where not 
(id < 10 and " +
  : //"(int_col in (select int_col from 
functional.alltypestiny)) and " +
  : //"(string_col = (select max(string_col) from 
functional.alltypestiny)))",
  : //"SELECT * FROM functional.alltypes WHERE NOT (id < 10 
AND " +
  : //"(int_col IN (SELECT int_col FROM 
functional.alltypestiny)) AND " +
  : //"(string_col = (SELECT max(string_col) FROM 
functional.alltypestiny)))");
> So, is it OK if I just leave this TODO here?
Please file an improvement JIRA for this case, and remove the test.

The issue here is that we push the NOT into the condition, so the ANDs turn 
into ORs and then we no longer support the subquery.



--
To view, visit http://gerrit.cloudera.org:8080/9005
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I0f52b93a60eeacedd242a2f17fa6b99c4fc38e06
Gerrit-Change-Number: 9005
Gerrit-PatchSet: 23
Gerrit-Owner: Zoltan Borok-Nagy 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Attila Jeges 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Dimitris Tsirogiannis 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Reviewer: Vuk Ercegovac 
Gerrit-Reviewer: Zoltan Borok-Nagy 
Gerrit-Comment-Date: Wed, 11 Apr 2018 21:44:28 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-2195: Improper handling of comments in queries

Fredy Wijaya has uploaded a new patch set (#5). ( 
http://gerrit.cloudera.org:8080/9933 )

Change subject: IMPALA-2195: Improper handling of comments in queries
..

IMPALA-2195: Improper handling of comments in queries

This patch fixes an issue where parseline is unable to determine the
correct command when a statement has a leading comment.

Before:
> -- comment
> insert into table t values(100);
Fetched 1 row(s) in 0.01s

After:
> -- comment
> insert into table t values(100);
Modified 1 row(s) in 0.01s

Testing:
- Added shell tests
- Ran end-to-end shell tests

Change-Id: I7ac7cb5a30e6dda73ebe761d9f0eb9ba038e14a7
---
M shell/impala_shell.py
M tests/shell/test_shell_interactive.py
2 files changed, 67 insertions(+), 6 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/33/9933/5
--
To view, visit http://gerrit.cloudera.org:8080/9933
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I7ac7cb5a30e6dda73ebe761d9f0eb9ba038e14a7
Gerrit-Change-Number: 9933
Gerrit-PatchSet: 5
Gerrit-Owner: Fredy Wijaya 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Bharath Vissapragada 
Gerrit-Reviewer: David Knupp 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Taras Bobrovytsky

[Impala-ASF-CR] IMPALA-2717: fix output of formatted unicode to non-TTY

2018-04-11 Thread Lars Volker (Code Review)

Lars Volker has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9928 )

Change subject: IMPALA-2717: fix output of formatted unicode to non-TTY
..


Patch Set 1:

(4 comments)

http://gerrit.cloudera.org:8080/#/c/9928/1/shell/impala_shell.py
File shell/impala_shell.py:

http://gerrit.cloudera.org:8080/#/c/9928/1/shell/impala_shell.py@73
PS1, Line 73:   """Patched version of PrettyTable that TODO"""
Not your change, but would you mind filling in the TODO? The difference seems 
to be that we're passing "replace" instead of "strict" in 
/usr/local/lib/python2.7/dist-packages/prettytable.py:181 .


http://gerrit.cloudera.org:8080/#/c/9928/1/shell/shell_output.py
File shell/shell_output.py:

http://gerrit.cloudera.org:8080/#/c/9928/1/shell/shell_output.py@45
PS1, Line 45: row
> It's already a str (verified by printing type(rows[0][0])).
I looked around a bit and tried to figure out how passing a str into add_row() 
can result in unicode coming out of get_string(). PrettyTable calls _unicode() 
on the row's fields, which will encode the str() back into unicode.

Do you want to add a brief comment mentioning this conversion inside add_row, 
maybe before L35?


http://gerrit.cloudera.org:8080/#/c/9928/2/shell/shell_output.py
File shell/shell_output.py:

http://gerrit.cloudera.org:8080/#/c/9928/2/shell/shell_output.py@31
PS2, Line 31:  containe
nit: containing?


http://gerrit.cloudera.org:8080/#/c/9928/2/shell/shell_output.py@58
PS2, Line 58: contained
here too?



--
To view, visit http://gerrit.cloudera.org:8080/9928
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9de641ecf767a2feef3b9f48b344ef2d55e17a7f
Gerrit-Change-Number: 9928
Gerrit-PatchSet: 1
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Wed, 11 Apr 2018 21:37:35 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6811: Increase timeout on S3 for test exchange delays

Hello Lars Volker, Michael Brown, Philip Zeyliger, Impala Public Jenkins, Vuk 
Ercegovac,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/9995

to look at the new patch set (#3).

Change subject: IMPALA-6811: Increase timeout on S3 for test_exchange_delays
..

IMPALA-6811: Increase timeout on S3 for test_exchange_delays

The exchange-delays-zero-rows portion of test_exchange_delays
tests a RPC timeout when sending an EOS with no rows. In
order to send the EOS, the executor must have completed
the scan (which returns zero rows). On S3, the scan has
been slow and can exceed the current receiver delay of
10 seconds, leading to a failure to timeout. In manual
testing, the sender timeout is being applied quickly
once the scan finishes.

This increases the receiver delay for S3 and ADLS to
20 seconds. None of the existing symptoms show the
scan taking that long. Most failures have a scan that
barely exceeds 10 seconds, the current receiver delay.
This has only been seen on S3, but ADLS is included as a
precaution.

Change-Id: I967e6eb336c801219c77d657655c42984910b479
---
M tests/custom_cluster/test_exchange_delays.py
1 file changed, 7 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/95/9995/3
--
To view, visit http://gerrit.cloudera.org:8080/9995
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I967e6eb336c801219c77d657655c42984910b479
Gerrit-Change-Number: 9995
Gerrit-PatchSet: 3
Gerrit-Owner: Joe McDonnell 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Michael Brown 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Vuk Ercegovac

[Impala-ASF-CR] IMPALA-6811: Increase timeout on S3 for test exchange delays

Joe McDonnell has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9995 )

Change subject: IMPALA-6811: Increase timeout on S3 for test_exchange_delays
..


Patch Set 2:

> Somewhere along the way, you mentioned that a log message helped
 > you confirm this. Is there a story we want to include in the
 > comments of this python file in case we ever run into it again?

Added a comment next to the exchange-delays-zero-rows part of the test 
explaining that slow scans can cause the test to fail.


--
To view, visit http://gerrit.cloudera.org:8080/9995
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I967e6eb336c801219c77d657655c42984910b479
Gerrit-Change-Number: 9995
Gerrit-PatchSet: 2
Gerrit-Owner: Joe McDonnell 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Michael Brown 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Vuk Ercegovac 
Gerrit-Comment-Date: Wed, 11 Apr 2018 21:36:19 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6811: Increase timeout on S3 for test exchange delays

Joe McDonnell has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9995 )

Change subject: IMPALA-6811: Increase timeout on S3 for test_exchange_delays
..


Patch Set 2:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/9995/1/tests/custom_cluster/test_exchange_delays.py
File tests/custom_cluster/test_exchange_delays.py:

http://gerrit.cloudera.org:8080/#/c/9995/1/tests/custom_cluster/test_exchange_delays.py@43
PS1, Line 43:   + " --datastream_sender_timeout_ms=5000")
> Are the semantics of this test and its comment still valid? With the increa
>From testing this, the semantics of this test aren't valid even without this 
>change. When the sender times out, that aborts the query. There is no 
>subsequent batch. The results don't exist.

I will file a JIRA. I think we should fix that separately.


http://gerrit.cloudera.org:8080/#/c/9995/2/tests/custom_cluster/test_exchange_delays.py
File tests/custom_cluster/test_exchange_delays.py:

http://gerrit.cloudera.org:8080/#/c/9995/2/tests/custom_cluster/test_exchange_delays.py@29
PS2, Line 29: IS_S3 or IS_ADLS
> These aren't being imported from what I can tell.
Done



--
To view, visit http://gerrit.cloudera.org:8080/9995
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I967e6eb336c801219c77d657655c42984910b479
Gerrit-Change-Number: 9995
Gerrit-PatchSet: 2
Gerrit-Owner: Joe McDonnell 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Michael Brown 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Vuk Ercegovac 
Gerrit-Comment-Date: Wed, 11 Apr 2018 21:35:29 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-5814: Remove startup flag to disable admission control

Tim Armstrong has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9964 )

Change subject: IMPALA-5814: Remove startup flag to disable admission control
..


Patch Set 3: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/9964
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9bf4087ce03ca63f82fd27c6d94b578881b85d42
Gerrit-Change-Number: 9964
Gerrit-PatchSet: 3
Gerrit-Owner: Bikramjeet Vig 
Gerrit-Reviewer: Bikramjeet Vig 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Wed, 11 Apr 2018 21:34:00 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-5814: Remove startup flag to disable admission control

Hello Philip Zeyliger, Tim Armstrong,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/9964

to look at the new patch set (#3).

Change subject: IMPALA-5814: Remove startup flag to disable admission control
..

IMPALA-5814: Remove startup flag to disable admission control

Remove "--disable admission control" startup flag and its related
functionality and usage.

Cherry-picks: not for 2.x

Change-Id: I9bf4087ce03ca63f82fd27c6d94b578881b85d42
---
M be/src/common/global-flags.cc
M be/src/runtime/coordinator.cc
M be/src/runtime/exec-env.cc
M be/src/service/client-request-state.cc
M bin/generate_minidump_collection_testdata.py
M tests/custom_cluster/test_admission_controller.py
6 files changed, 13 insertions(+), 20 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/64/9964/3
--
To view, visit http://gerrit.cloudera.org:8080/9964
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I9bf4087ce03ca63f82fd27c6d94b578881b85d42
Gerrit-Change-Number: 9964
Gerrit-PatchSet: 3
Gerrit-Owner: Bikramjeet Vig 
Gerrit-Reviewer: Bikramjeet Vig 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Tim Armstrong

[Impala-ASF-CR] IMPALA-5814: Remove startup flag to disable admission control

Bikramjeet Vig has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9964 )

Change subject: IMPALA-5814: Remove startup flag to disable admission control
..


Patch Set 2:

(1 comment)

> Actually we should separate out the doc component into a separate
 > commit, since we don't run the docs build on code commits.

Done

http://gerrit.cloudera.org:8080/#/c/9964/2/be/src/runtime/coordinator.cc
File be/src/runtime/coordinator.cc:

http://gerrit.cloudera.org:8080/#/c/9964/2/be/src/runtime/coordinator.cc@787
PS2, Line 787:   DCHECK(admission_controller);
> nit: != nullptr
Done. Here and in client-request-state



--
To view, visit http://gerrit.cloudera.org:8080/9964
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9bf4087ce03ca63f82fd27c6d94b578881b85d42
Gerrit-Change-Number: 9964
Gerrit-PatchSet: 2
Gerrit-Owner: Bikramjeet Vig 
Gerrit-Reviewer: Bikramjeet Vig 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Wed, 11 Apr 2018 21:32:19 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6710: [DOCS] Update the Partition Insert content

2018-04-11 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9977 )

Change subject: IMPALA-6710: [DOCS] Update the Partition Insert content
..


Patch Set 1:

Build started: https://jenkins.impala.io/job/gerrit-docs-submit/231/


--
To view, visit http://gerrit.cloudera.org:8080/9977
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4ccc8227579dabc321a949da95e8a59158528f20
Gerrit-Change-Number: 9977
Gerrit-PatchSet: 1
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Thomas Tauber-Marshall 
Gerrit-Comment-Date: Wed, 11 Apr 2018 21:28:08 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6623: [DOCS] ltrim and rtrim docs updated

2018-04-11 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9984 )

Change subject: IMPALA-6623: [DOCS] ltrim and rtrim docs updated
..


Patch Set 1:

Build started: https://jenkins.impala.io/job/gerrit-docs-submit/232/


--
To view, visit http://gerrit.cloudera.org:8080/9984
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If4f7a04e3c64eade7a23cded21de5ff91c9c8c8c
Gerrit-Change-Number: 9984
Gerrit-PatchSet: 1
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Zoram Thanga 
Gerrit-Comment-Date: Wed, 11 Apr 2018 21:28:27 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6822: Add a query option to control shuffling by distinct exprs

2018-04-11 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9949 )

Change subject: IMPALA-6822: Add a query option to control shuffling by 
distinct exprs
..


Patch Set 6:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2287/


--
To view, visit http://gerrit.cloudera.org:8080/9949
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Icb4b4576fb29edd62cf4b4ba0719c0e0a2a5a8dc
Gerrit-Change-Number: 9949
Gerrit-PatchSet: 6
Gerrit-Owner: Tianyi Wang 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tianyi Wang 
Gerrit-Comment-Date: Wed, 11 Apr 2018 21:13:55 +
Gerrit-HasComments: No

[Impala-ASF-CR] Move some test spilling debug actions to exhaustive

2018-04-11 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9976 )

Change subject: Move some test_spilling debug actions to exhaustive
..


Patch Set 3:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2286/


--
To view, visit http://gerrit.cloudera.org:8080/9976
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9ea9f6c299480f8dfc943635342e4473499cc8ad
Gerrit-Change-Number: 9976
Gerrit-PatchSet: 3
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Wed, 11 Apr 2018 21:08:33 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6811: Increase timeout on S3 for test exchange delays

2018-04-11 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9995 )

Change subject: IMPALA-6811: Increase timeout on S3 for test_exchange_delays
..


Patch Set 2:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2284/


--
To view, visit http://gerrit.cloudera.org:8080/9995
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I967e6eb336c801219c77d657655c42984910b479
Gerrit-Change-Number: 9995
Gerrit-PatchSet: 2
Gerrit-Owner: Joe McDonnell 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Michael Brown 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Vuk Ercegovac 
Gerrit-Comment-Date: Wed, 11 Apr 2018 21:07:55 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6822: Add a query option to control shuffling by distinct exprs

Alex Behm has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9949 )

Change subject: IMPALA-6822: Add a query option to control shuffling by 
distinct exprs
..


Patch Set 6: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/9949
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Icb4b4576fb29edd62cf4b4ba0719c0e0a2a5a8dc
Gerrit-Change-Number: 9949
Gerrit-PatchSet: 6
Gerrit-Owner: Tianyi Wang 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Tianyi Wang 
Gerrit-Comment-Date: Wed, 11 Apr 2018 20:54:17 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6793: Fix empty metadata after statestore restarts

2018-04-11 Thread Vuk Ercegovac (Code Review)

Vuk Ercegovac has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9921 )

Change subject: IMPALA-6793: Fix empty metadata after statestore restarts
..


Patch Set 7:

Michael, pls have a look at the python tests.


--
To view, visit http://gerrit.cloudera.org:8080/9921
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I332a60e172af84b93b3544373fe363cdced5e8d0
Gerrit-Change-Number: 9921
Gerrit-PatchSet: 7
Gerrit-Owner: Tianyi Wang 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Dimitris Tsirogiannis 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Michael Brown 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Tianyi Wang 
Gerrit-Reviewer: Vuk Ercegovac 
Gerrit-Comment-Date: Wed, 11 Apr 2018 20:49:58 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-5814: Remove startup flag to disable admission control

Tim Armstrong has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9964 )

Change subject: IMPALA-5814: Remove startup flag to disable admission control
..


Patch Set 2: -Code-Review

Actually we should separate out the doc component into a separate commit, since 
we don't run the docs build on code commits.


--
To view, visit http://gerrit.cloudera.org:8080/9964
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9bf4087ce03ca63f82fd27c6d94b578881b85d42
Gerrit-Change-Number: 9964
Gerrit-PatchSet: 2
Gerrit-Owner: Bikramjeet Vig 
Gerrit-Reviewer: Bikramjeet Vig 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Wed, 11 Apr 2018 20:46:31 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6811: Increase timeout on S3 for test exchange delays

Philip Zeyliger has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9995 )

Change subject: IMPALA-6811: Increase timeout on S3 for test_exchange_delays
..


Patch Set 2:

Somewhere along the way, you mentioned that a log message helped you confirm 
this. Is there a story we want to include in the comments of this python file 
in case we ever run into it again?


--
To view, visit http://gerrit.cloudera.org:8080/9995
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I967e6eb336c801219c77d657655c42984910b479
Gerrit-Change-Number: 9995
Gerrit-PatchSet: 2
Gerrit-Owner: Joe McDonnell 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Michael Brown 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Vuk Ercegovac 
Gerrit-Comment-Date: Wed, 11 Apr 2018 20:44:33 +
Gerrit-HasComments: No

[Impala-ASF-CR] Add tpcds-unmodified performance workload.

Tim Armstrong has abandoned this change. ( http://gerrit.cloudera.org:8080/5813 
)

Change subject: Add tpcds-unmodified performance workload.
..


Abandoned

Let's just abandon until the new version is ready for review. We can still look 
at the abandoned patch but it won't show in any lists of patches that need to 
be reviewed.
--
To view, visit http://gerrit.cloudera.org:8080/5813
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: abandon
Gerrit-Change-Id: Iea8f3b4e20a30553f1a5a321b2053ff9ca967e92
Gerrit-Change-Number: 5813
Gerrit-PatchSet: 1
Gerrit-Owner: Matthew Mulder 
Gerrit-Reviewer: David Knupp 
Gerrit-Reviewer: Michael Brown 
Gerrit-Reviewer: Mostafa Mokhtar 
Gerrit-Reviewer: Nithya Janarthanan 
Gerrit-Reviewer: Tim Armstrong

[Impala-ASF-CR] Move some test spilling debug actions to exhaustive

2018-04-11 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9976 )

Change subject: Move some test_spilling debug actions to exhaustive
..


Patch Set 3: Verified-1

Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/2281/


--
To view, visit http://gerrit.cloudera.org:8080/9976
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9ea9f6c299480f8dfc943635342e4473499cc8ad
Gerrit-Change-Number: 9976
Gerrit-PatchSet: 3
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Wed, 11 Apr 2018 20:38:44 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-5814: Remove startup flag to disable admission control

Tim Armstrong has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9964 )

Change subject: IMPALA-5814: Remove startup flag to disable admission control
..


Patch Set 2: Code-Review+2

(1 comment)

This looks good. I don't think the change will be user-visible unless they have 
disabled admission control but provided admission control config files (since 
the default even with AC enabled is to not apply any limits).

http://gerrit.cloudera.org:8080/#/c/9964/2/be/src/runtime/coordinator.cc
File be/src/runtime/coordinator.cc:

http://gerrit.cloudera.org:8080/#/c/9964/2/be/src/runtime/coordinator.cc@787
PS2, Line 787:   DCHECK(admission_controller);
nit: != nullptr



--
To view, visit http://gerrit.cloudera.org:8080/9964
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9bf4087ce03ca63f82fd27c6d94b578881b85d42
Gerrit-Change-Number: 9964
Gerrit-PatchSet: 2
Gerrit-Owner: Bikramjeet Vig 
Gerrit-Reviewer: Bikramjeet Vig 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Wed, 11 Apr 2018 20:39:52 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6480: [DOCS] DESCRIBE respects column-level privilege