[Impala-ASF-CR] IMPALA-6805: Show current database in Impala shell prompt

[Taras Bobrovytsky (Code Review)]

Taras Bobrovytsky has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9927 )

Change subject: IMPALA-6805: Show current database in Impala shell prompt
..


Patch Set 6: Code-Review+1

Looks good to me. David, do you want to give the final +2?


--
To view, visit http://gerrit.cloudera.org:8080/9927
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifb0ae58507321e426e5f0f16518671420974a3fc
Gerrit-Change-Number: 9927
Gerrit-PatchSet: 6
Gerrit-Owner: Fredy Wijaya 
Gerrit-Reviewer: David Knupp 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Taras Bobrovytsky 
Gerrit-Comment-Date: Fri, 06 Apr 2018 05:17:26 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-5903: Inconsistent specification of result set and result set metadata

[Alex Behm (Code Review)]

Alex Behm has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9090 )

Change subject: IMPALA-5903: Inconsistent specification of result set and 
result set metadata
..


Patch Set 6:

(1 comment)

Nice work! I'll +2 after this last comment has been addressed.

http://gerrit.cloudera.org:8080/#/c/9090/6/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
File fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java:

http://gerrit.cloudera.org:8080/#/c/9090/6/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@1178
PS6, Line 1178:   addSummary(resp, "Stats has been dropped.");
Stats have been dropped.



--
To view, visit http://gerrit.cloudera.org:8080/9090
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic542fb8e49e850052416ac663ee329ee3974e3b9
Gerrit-Change-Number: 9090
Gerrit-PatchSet: 6
Gerrit-Owner: Zoltan Borok-Nagy 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Dimitris Tsirogiannis 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Reviewer: Zoltan Borok-Nagy 
Gerrit-Comment-Date: Fri, 06 Apr 2018 05:12:43 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] KUDU-2401: External TLS certificate with Intermediate CA in server cert file fails

[Impala Public Jenkins (Code Review)]

Impala Public Jenkins has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/9940 )

Change subject: KUDU-2401: External TLS certificate with Intermediate CA in 
server cert file fails
..

KUDU-2401: External TLS certificate with Intermediate CA in server cert file 
fails

Take 2 certificate files: cert.pem and truststore.pem

cert.pem has 2 certificates in it:
A cert for that node (with CN="hostname", and signed by CN=CertToolkitIntCA)
And the intermediate CA cert (with CN=CertToolkitIntCA, and signed by 
CN=CertToolkitRootCA)

truststore.pem has 1 certificate in it:
A cert which is the root CA (with CN=CertToolkitRootCA, self-signed)

This previously would not work with KRPC because in
TlsContext::VerifyCertChainUnlocked(), we would only verify X509_verify_cert()
with the top certificate in the server certificate chain.

With this change, we pass the chain to X509_STORE_CTX_init() as well to make
sure that the entire chain gets checked against the CA.

A test is added that uses the specific certificate format mentioned above
and added to rpc-test.

TODO: Add a test case that has multiple intermediate CAs. Right now we're 
testing
with only one intermediate CA.

Change-Id: If4af35e97ec6f91c1d9ed902128bd7f4e260f0f4
Reviewed-on: http://gerrit.cloudera.org:8080/9940
Reviewed-by: Lars Volker 
Tested-by: Impala Public Jenkins 
---
M be/src/kudu/rpc/rpc-test.cc
M be/src/kudu/security/test/test_certs.cc
M be/src/kudu/security/test/test_certs.h
M be/src/kudu/security/tls_context.cc
4 files changed, 263 insertions(+), 4 deletions(-)

Approvals:
  Lars Volker: Looks good to me, approved
  Impala Public Jenkins: Verified

--
To view, visit http://gerrit.cloudera.org:8080/9940
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: If4af35e97ec6f91c1d9ed902128bd7f4e260f0f4
Gerrit-Change-Number: 9940
Gerrit-PatchSet: 2
Gerrit-Owner: Sailesh Mukil 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Sailesh Mukil 

[Impala-ASF-CR] IMPALA-6817: Clean up Impala privilege model

[Fredy Wijaya (Code Review)]

Fredy Wijaya has uploaded a new patch set (#7). ( 
http://gerrit.cloudera.org:8080/9942 )

Change subject: IMPALA-6817: Clean up Impala privilege model
..

IMPALA-6817: Clean up Impala privilege model

When we added support for multiple versions of Hadoop, the Impala
privilege model was copied from Sentry's Hive privilege model with
some modification. This patch aims to clean up that code.

Testing:
- Added new tests
- Ran all front-end tests

Cherry-picks: not for 2.x

Change-Id: I65f3f9b4d06f3b03bfa2f484737bb746ec598a6b
---
M 
fe/src/compat-minicluster-profile-3/java/org/apache/impala/authorization/ImpalaActionFactory.java
M 
fe/src/compat-minicluster-profile-3/java/org/apache/impala/authorization/ImpalaPrivilegeModel.java
M fe/src/main/java/org/apache/impala/authorization/AuthorizationChecker.java
M fe/src/main/java/org/apache/impala/authorization/Privilege.java
A fe/src/test/java/org/apache/impala/authorization/ImpalaActionFactoryTest.java
5 files changed, 156 insertions(+), 80 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/42/9942/7
--
To view, visit http://gerrit.cloudera.org:8080/9942
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I65f3f9b4d06f3b03bfa2f484737bb746ec598a6b
Gerrit-Change-Number: 9942
Gerrit-PatchSet: 7
Gerrit-Owner: Fredy Wijaya 

[Impala-ASF-CR] IMPALA-6817: Clean up Impala privilege model

[Fredy Wijaya (Code Review)]

Fredy Wijaya has uploaded a new patch set (#6). ( 
http://gerrit.cloudera.org:8080/9942 )

Change subject: IMPALA-6817: Clean up Impala privilege model
..

IMPALA-6817: Clean up Impala privilege model

When we added support for multiple versions of Hadoop, the Impala
privilege model was copied from Sentry's Hive privilege model with
some modification. This patch aims to clean up that code.

Testing:
- Added new tests
- Ran all front-end tests

Cherry-picks: not for 2.x

Change-Id: I65f3f9b4d06f3b03bfa2f484737bb746ec598a6b
---
M 
fe/src/compat-minicluster-profile-3/java/org/apache/impala/authorization/ImpalaActionFactory.java
M 
fe/src/compat-minicluster-profile-3/java/org/apache/impala/authorization/ImpalaPrivilegeModel.java
M fe/src/main/java/org/apache/impala/authorization/AuthorizationChecker.java
M fe/src/main/java/org/apache/impala/authorization/Privilege.java
A fe/src/test/java/org/apache/impala/authorization/ImpalaActionFactoryTest.java
5 files changed, 157 insertions(+), 79 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/42/9942/6
--
To view, visit http://gerrit.cloudera.org:8080/9942
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I65f3f9b4d06f3b03bfa2f484737bb746ec598a6b
Gerrit-Change-Number: 9942
Gerrit-PatchSet: 6
Gerrit-Owner: Fredy Wijaya 

[Impala-ASF-CR] IMPALA-6817: Clean up Impala privilege model

[Fredy Wijaya (Code Review)]

Fredy Wijaya has uploaded a new patch set (#5). ( 
http://gerrit.cloudera.org:8080/9942 )

Change subject: IMPALA-6817: Clean up Impala privilege model
..

IMPALA-6817: Clean up Impala privilege model

When we added support for multiple versions of Hadoop, the Impala
privilege model was copied from Sentry's Hive privilege model with
some modification. This patch aims to clean up that code.

Testing:
- Added new tests
- Ran all front-end tests

Change-Id: I65f3f9b4d06f3b03bfa2f484737bb746ec598a6b
---
M 
fe/src/compat-minicluster-profile-3/java/org/apache/impala/authorization/ImpalaActionFactory.java
M 
fe/src/compat-minicluster-profile-3/java/org/apache/impala/authorization/ImpalaPrivilegeModel.java
M fe/src/main/java/org/apache/impala/authorization/AuthorizationChecker.java
M fe/src/main/java/org/apache/impala/authorization/Privilege.java
A fe/src/test/java/org/apache/impala/authorization/ImpalaActionFactoryTest.java
5 files changed, 157 insertions(+), 79 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/42/9942/5
--
To view, visit http://gerrit.cloudera.org:8080/9942
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I65f3f9b4d06f3b03bfa2f484737bb746ec598a6b
Gerrit-Change-Number: 9942
Gerrit-PatchSet: 5
Gerrit-Owner: Fredy Wijaya 

[Impala-ASF-CR] IMPALA-6817: Clean up Impala authorization model

[Fredy Wijaya (Code Review)]

Fredy Wijaya has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/9942


Change subject: IMPALA-6817: Clean up Impala authorization model
..

IMPALA-6817: Clean up Impala authorization model

When we added support for multiple versions of Hadoop, the Impala
authorization model was copied from Sentry's Hive authorization model
with some modification. This patch aims to clean up that code.

Testing:
- Added new tests
- Ran all front-end tests

Change-Id: I65f3f9b4d06f3b03bfa2f484737bb746ec598a6b
---
M 
fe/src/compat-minicluster-profile-3/java/org/apache/impala/authorization/ImpalaActionFactory.java
M 
fe/src/compat-minicluster-profile-3/java/org/apache/impala/authorization/ImpalaPrivilegeModel.java
M fe/src/main/java/org/apache/impala/authorization/AuthorizationChecker.java
M fe/src/main/java/org/apache/impala/authorization/Privilege.java
A fe/src/test/java/org/apache/impala/authorization/ImpalaActionFactoryTest.java
5 files changed, 157 insertions(+), 79 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/42/9942/3
--
To view, visit http://gerrit.cloudera.org:8080/9942
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I65f3f9b4d06f3b03bfa2f484737bb746ec598a6b
Gerrit-Change-Number: 9942
Gerrit-PatchSet: 3
Gerrit-Owner: Fredy Wijaya 

[Impala-ASF-CR] IMPALA-6070: Expose using Docker to run tests faster.

[Philip Zeyliger (Code Review)]

Philip Zeyliger has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9085 )

Change subject: IMPALA-6070: Expose using Docker to run tests faster.
..


Patch Set 8: Code-Review+2

Carry +2. Rat build failed on docker/README.md. We seem to exclude other 
READMEs from rat, so I did the same here.

(Um, rat is going to be added to test-with-docker.py sooner rather than later.)


--
To view, visit http://gerrit.cloudera.org:8080/9085
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I82052ef31979564968effef13a3c6af0d5c62767
Gerrit-Change-Number: 9085
Gerrit-PatchSet: 8
Gerrit-Owner: Philip Zeyliger 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Jim Apple 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Taras Bobrovytsky 
Gerrit-Comment-Date: Fri, 06 Apr 2018 02:47:44 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6070: Expose using Docker to run tests faster.

[Impala Public Jenkins (Code Review)]

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9085 )

Change subject: IMPALA-6070: Expose using Docker to run tests faster.
..


Patch Set 8:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2249/


--
To view, visit http://gerrit.cloudera.org:8080/9085
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I82052ef31979564968effef13a3c6af0d5c62767
Gerrit-Change-Number: 9085
Gerrit-PatchSet: 8
Gerrit-Owner: Philip Zeyliger 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Jim Apple 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Taras Bobrovytsky 
Gerrit-Comment-Date: Fri, 06 Apr 2018 02:47:52 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6070: Expose using Docker to run tests faster.

[Philip Zeyliger (Code Review)]

Hello Lars Volker, Taras Bobrovytsky, Jim Apple, Impala Public Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/9085

to look at the new patch set (#8).

Change subject: IMPALA-6070: Expose using Docker to run tests faster.
..

IMPALA-6070: Expose using Docker to run tests faster.

Allows running the tests that make up the "core" suite in about 2 hours.
By comparison, 
https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/buildTimeTrend
tends to run in about 3.5 hours.

This commit:
* Adds "echo" statements in a few places, to facilitate timing.
* Adds --skip-parallel/--skip-serial flags to run-tests.py,
  and exposes them in run-all-tests.sh.
* Marks TestRuntimeFilters as a serial test. This test runs
  queries that need > 1GB of memory, and, combined with
  other tests running in parallel, can kill the parallel test
  suite.
* Adds "test-with-docker.py", which runs a full build, data load,
  and executes tests inside of Docker containers, generating
  a timeline at the end. In short, one container is used
  to do the build and data load, and then this container is
  re-used to run various tests in parallel. All logs are
  left on the host system.

Besides the obvious win of getting test results more quickly, this
commit serves as an example of how to get various bits of Impala
development working inside of Docker containers. For example, Kudu
relies on atomic rename of directories, which isn't available in most
Docker filesystems, and entrypoint.sh works around it.

In addition, the timeline generated by the build suggests where further
optimizations can be made. Most obviously, dataload eats up a precious
~30-50 minutes, on a largely idle machine.

This work is significantly CPU and memory hungry. It was developed on a
32-core, 120GB RAM Google Compute Engine machine. I've worked out
parallelism configurations such that it runs nicely on 60GB of RAM
(c4.8xlarge) and over 100GB (eg., m4.10xlarge, which has 160GB). There is
some simple logic to guess at some knobs, and there are knobs.  By and
large, EC2 and GCE price machines linearly, so, if CPU usage can be kept
up, it's not wasteful to run on bigger machines.

Change-Id: I82052ef31979564968effef13a3c6af0d5c62767
---
M bin/bootstrap_system.sh
M bin/rat_exclude_files.txt
M bin/run-all-tests.sh
M buildall.sh
A docker/README.md
A docker/annotate.py
A docker/entrypoint.sh
A docker/monitor.py
A docker/test-with-docker.py
A docker/timeline.html.template
M testdata/bin/run-all.sh
M tests/query_test/test_runtime_filters.py
M tests/run-tests.py
13 files changed, 1,467 insertions(+), 13 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/85/9085/8
--
To view, visit http://gerrit.cloudera.org:8080/9085
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I82052ef31979564968effef13a3c6af0d5c62767
Gerrit-Change-Number: 9085
Gerrit-PatchSet: 8
Gerrit-Owner: Philip Zeyliger 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Jim Apple 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Taras Bobrovytsky 

[Impala-ASF-CR] IMPALA-5717: Support for reading ORC data files

[Quanlong Huang (Code Review)]

Quanlong Huang has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9134 )

Change subject: IMPALA-5717: Support for reading ORC data files
..


Patch Set 14:

Fixed some compiler warnings. I can build with -asan. Running tests locally too.


--
To view, visit http://gerrit.cloudera.org:8080/9134
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia7b6ae4ce3b9ee8125b21993702faa87537790a4
Gerrit-Change-Number: 9134
Gerrit-PatchSet: 14
Gerrit-Owner: Quanlong Huang 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Fri, 06 Apr 2018 01:52:55 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-5717: Support for reading ORC data files

[Quanlong Huang (Code Review)]

Hello Tim Armstrong, Joe McDonnell, Dan Hecht,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/9134

to look at the new patch set (#14).

Change subject: IMPALA-5717: Support for reading ORC data files
..

IMPALA-5717: Support for reading ORC data files

This patch integrates the orc library into Impala and implements
HdfsOrcScanner as a middle layer between them. The HdfsOrcScanner
supplies input needed from the orc-reader, tracks memory consumption of
the reader and transfers the reader's output (orc::ColumnVectorBatch)
into impala::RowBatch. The ORC version we used is release-1.4.3.

A startup option --enable_orc_scanner is added for this feature. It's
set to true by default. Setting it to false will fail queries on ORC
tables.

Currently, we only support reading primitive types. Writing into ORC
table has not been supported neither.

Tests
 - Most of the end-to-end tests can run on ORC format.
 - Add tpcds, tpch tests for ORC.
 - Add some ORC specific tests.
 - Haven't enabled test_scanner_fuzz for ORC yet, since the ORC library
   is not robust for corrupt files (ORC-315).

Change-Id: Ia7b6ae4ce3b9ee8125b21993702faa87537790a4
---
M CMakeLists.txt
M be/CMakeLists.txt
M be/src/codegen/gen_ir_descriptions.py
M be/src/exec/CMakeLists.txt
A be/src/exec/hdfs-orc-scanner.cc
A be/src/exec/hdfs-orc-scanner.h
M be/src/exec/hdfs-parquet-scanner-ir.cc
M be/src/exec/hdfs-parquet-scanner.cc
M be/src/exec/hdfs-parquet-scanner.h
M be/src/exec/hdfs-scan-node-base.cc
M be/src/exec/hdfs-scan-node-mt.cc
M be/src/exec/hdfs-scanner-ir.cc
M be/src/exec/hdfs-scanner.cc
M be/src/exec/hdfs-scanner.h
M be/src/util/backend-gflag-util.cc
M bin/bootstrap_toolchain.py
M bin/impala-config.sh
A cmake_modules/FindOrc.cmake
M common/thrift/BackendGflags.thrift
M common/thrift/CatalogObjects.thrift
M fe/src/main/cup/sql-parser.cup
M fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java
M fe/src/main/java/org/apache/impala/catalog/HdfsFileFormat.java
M fe/src/main/java/org/apache/impala/catalog/HdfsStorageDescriptor.java
M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java
M fe/src/main/java/org/apache/impala/service/BackendConfig.java
M fe/src/main/java/org/apache/impala/service/Frontend.java
M fe/src/main/jflex/sql-scanner.flex
M testdata/LineItemMultiBlock/README.dox
A testdata/LineItemMultiBlock/lineitem_orc_multiblock_one_stripe.orc
A testdata/LineItemMultiBlock/lineitem_sixblocks.orc
A testdata/LineItemMultiBlock/lineitem_threeblocks.orc
M testdata/bin/create-load-data.sh
M testdata/bin/generate-schema-statements.py
M testdata/bin/run-hive-server.sh
M testdata/cluster/node_templates/common/etc/hadoop/conf/hdfs-site.xml.tmpl
A testdata/data/chars-formats.orc
M testdata/datasets/functional/functional_schema_template.sql
M testdata/datasets/functional/schema_constraints.csv
M 
testdata/workloads/functional-planner/queries/PlannerTest/complex-types-file-formats.test
M testdata/workloads/functional-query/functional-query_core.csv
M testdata/workloads/functional-query/functional-query_dimensions.csv
M testdata/workloads/functional-query/functional-query_exhaustive.csv
M testdata/workloads/functional-query/functional-query_pairwise.csv
A 
testdata/workloads/functional-query/queries/DataErrorsTest/orc-type-checks.test
M testdata/workloads/tpcds/tpcds_core.csv
M testdata/workloads/tpcds/tpcds_dimensions.csv
M testdata/workloads/tpcds/tpcds_exhaustive.csv
M testdata/workloads/tpcds/tpcds_pairwise.csv
M testdata/workloads/tpch/tpch_core.csv
M testdata/workloads/tpch/tpch_dimensions.csv
M testdata/workloads/tpch/tpch_exhaustive.csv
M testdata/workloads/tpch/tpch_pairwise.csv
M tests/common/impala_test_suite.py
M tests/common/test_dimensions.py
M tests/common/test_vector.py
M tests/comparison/cli_options.py
M tests/query_test/test_chars.py
M tests/query_test/test_decimal_queries.py
M tests/query_test/test_scanners.py
M tests/query_test/test_scanners_fuzz.py
M tests/query_test/test_tpch_queries.py
62 files changed, 1,753 insertions(+), 307 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/34/9134/14
--
To view, visit http://gerrit.cloudera.org:8080/9134
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia7b6ae4ce3b9ee8125b21993702faa87537790a4
Gerrit-Change-Number: 9134
Gerrit-PatchSet: 14
Gerrit-Owner: Quanlong Huang 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Reviewer: Tim Armstrong 

[Impala-ASF-CR] IMPALA-5893: Remove old kinit code for Impala 3

[Sailesh Mukil (Code Review)]

Sailesh Mukil has uploaded a new patch set (#2). ( 
http://gerrit.cloudera.org:8080/9941 )

Change subject: IMPALA-5893: Remove old kinit code for Impala 3
..

IMPALA-5893: Remove old kinit code for Impala 3

We've gone through a couple of releases with Kudu's kinit as the
default way to use kerberos and we've not come across any major issues.

Since we're going to have a major release soon, it's time to get rid of
the old Kinit code that's largely unused for a while now.

Testing: Made sure that our current kerberos tests continue to work
without the old code.

Cherry-picks: not for 2.x

Change-Id: Ic78de10f3fb9ec36537de7a090916e4be123234b
---
M be/src/rpc/auth-provider.h
M be/src/rpc/authentication.cc
M be/src/rpc/rpc-mgr-kerberized-test.cc
M be/src/rpc/thrift-server-test.cc
M be/src/testutil/mini-kdc-wrapper.h
5 files changed, 9 insertions(+), 102 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/41/9941/2
--
To view, visit http://gerrit.cloudera.org:8080/9941
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ic78de10f3fb9ec36537de7a090916e4be123234b
Gerrit-Change-Number: 9941
Gerrit-PatchSet: 2
Gerrit-Owner: Sailesh Mukil 

[Impala-ASF-CR] IMPALA-5893: Remove old kinit code for Impala 3

[Sailesh Mukil (Code Review)]

Sailesh Mukil has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/9941


Change subject: IMPALA-5893: Remove old kinit code for Impala 3
..

IMPALA-5893: Remove old kinit code for Impala 3

We've gone through a couple of releases with Kudu's kinit as the
default way to use kerberos and we've not come across any major issues.

Since we're going to have a major release soon, it's time to get rid of
the old Kinit code that's largely unused for a while now.

Testing: Made sure that our current kerberos tests continue to work
without the old code.

Change-Id: Ic78de10f3fb9ec36537de7a090916e4be123234b
---
M be/src/rpc/auth-provider.h
M be/src/rpc/authentication.cc
M be/src/rpc/rpc-mgr-kerberized-test.cc
M be/src/rpc/thrift-server-test.cc
M be/src/testutil/mini-kdc-wrapper.h
5 files changed, 9 insertions(+), 102 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/41/9941/1
--
To view, visit http://gerrit.cloudera.org:8080/9941
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ic78de10f3fb9ec36537de7a090916e4be123234b
Gerrit-Change-Number: 9941
Gerrit-PatchSet: 1
Gerrit-Owner: Sailesh Mukil 

[Impala-ASF-CR] IMPALA-6805: Show current database in Impala shell prompt

[Fredy Wijaya (Code Review)]

Fredy Wijaya has uploaded a new patch set (#6). ( 
http://gerrit.cloudera.org:8080/9927 )

Change subject: IMPALA-6805: Show current database in Impala shell prompt
..

IMPALA-6805: Show current database in Impala shell prompt

Prompt format:
[host:port] db_name>

Testing:
- Added new shell tests
- Ran end-to-end shell tests

Change-Id: Ifb0ae58507321e426e5f0f16518671420974a3fc
---
M shell/impala_shell.py
M tests/shell/test_shell_interactive.py
2 files changed, 39 insertions(+), 7 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/27/9927/6
--
To view, visit http://gerrit.cloudera.org:8080/9927
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ifb0ae58507321e426e5f0f16518671420974a3fc
Gerrit-Change-Number: 9927
Gerrit-PatchSet: 6
Gerrit-Owner: Fredy Wijaya 
Gerrit-Reviewer: David Knupp 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Taras Bobrovytsky 

[Impala-ASF-CR] KUDU-2401: External TLS certificate with Intermediate CA in server cert file fails

[Impala Public Jenkins (Code Review)]

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9940 )

Change subject: KUDU-2401: External TLS certificate with Intermediate CA in 
server cert file fails
..


Patch Set 1:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2248/


--
To view, visit http://gerrit.cloudera.org:8080/9940
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If4af35e97ec6f91c1d9ed902128bd7f4e260f0f4
Gerrit-Change-Number: 9940
Gerrit-PatchSet: 1
Gerrit-Owner: Sailesh Mukil 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Comment-Date: Fri, 06 Apr 2018 01:10:25 +
Gerrit-HasComments: No

[Impala-ASF-CR] KUDU-2401: External TLS certificate with Intermediate CA in server cert file fails

[Lars Volker (Code Review)]

Lars Volker has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9940 )

Change subject: KUDU-2401: External TLS certificate with Intermediate CA in 
server cert file fails
..


Patch Set 1: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/9940
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If4af35e97ec6f91c1d9ed902128bd7f4e260f0f4
Gerrit-Change-Number: 9940
Gerrit-PatchSet: 1
Gerrit-Owner: Sailesh Mukil 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Comment-Date: Fri, 06 Apr 2018 01:09:47 +
Gerrit-HasComments: No

[Impala-ASF-CR] KUDU-2401: External TLS certificate with Intermediate CA in server cert file fails

[Impala Public Jenkins (Code Review)]

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9940 )

Change subject: KUDU-2401: External TLS certificate with Intermediate CA in 
server cert file fails
..


Patch Set 1:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2247/


--
To view, visit http://gerrit.cloudera.org:8080/9940
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If4af35e97ec6f91c1d9ed902128bd7f4e260f0f4
Gerrit-Change-Number: 9940
Gerrit-PatchSet: 1
Gerrit-Owner: Sailesh Mukil 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Comment-Date: Fri, 06 Apr 2018 01:10:16 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6070: Expose using Docker to run tests faster.

[Impala Public Jenkins (Code Review)]

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9085 )

Change subject: IMPALA-6070: Expose using Docker to run tests faster.
..


Patch Set 7: Verified-1

Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/2246/


--
To view, visit http://gerrit.cloudera.org:8080/9085
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I82052ef31979564968effef13a3c6af0d5c62767
Gerrit-Change-Number: 9085
Gerrit-PatchSet: 7
Gerrit-Owner: Philip Zeyliger 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Jim Apple 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Taras Bobrovytsky 
Gerrit-Comment-Date: Fri, 06 Apr 2018 00:58:03 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6810, IMPALA-6814: Skip remote tests that expect a local env

[Bikramjeet Vig (Code Review)]

Bikramjeet Vig has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9937 )

Change subject: IMPALA-6810, IMPALA-6814: Skip remote tests that expect a local 
env
..


Patch Set 1:

There is only one test case that seems to contain a local env specific string. 
If you remove

"ImpalaBeeswaxException: INNER EXCEPTION: 
 MESSAGE: Rejected query from pool default-pool: "

this part from the error string here

https://github.com/apache/impala/blob/4d6b07f0e2d18c8f5284d27b82bcee3aa9f1fe77/testdata/workloads/functional-query/queries/QueryTest/runtime_row_filters.test#L404

the test would work on all env and still fulfill its purpose.


--
To view, visit http://gerrit.cloudera.org:8080/9937
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Icfc1c6b56c9721a4641c939d4bebabfc0600281f
Gerrit-Change-Number: 9937
Gerrit-PatchSet: 1
Gerrit-Owner: David Knupp 
Gerrit-Reviewer: Bikramjeet Vig 
Gerrit-Reviewer: David Knupp 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Comment-Date: Fri, 06 Apr 2018 00:33:50 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6805: Show current database in Impala shell prompt

[Taras Bobrovytsky (Code Review)]

Taras Bobrovytsky has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9927 )

Change subject: IMPALA-6805: Show current database in Impala shell prompt
..


Patch Set 5:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/9927/5/shell/impala_shell.py
File shell/impala_shell.py:

http://gerrit.cloudera.org:8080/#/c/9927/5/shell/impala_shell.py@1150
PS5, Line 1150: # args == current_db means -d option was passed but the 
"use [db]" operation failed.
This comments needs to be updated. Move this comment under elif.



--
To view, visit http://gerrit.cloudera.org:8080/9927
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifb0ae58507321e426e5f0f16518671420974a3fc
Gerrit-Change-Number: 9927
Gerrit-PatchSet: 5
Gerrit-Owner: Fredy Wijaya 
Gerrit-Reviewer: David Knupp 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Taras Bobrovytsky 
Gerrit-Comment-Date: Fri, 06 Apr 2018 00:21:19 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-5717: Support for reading ORC data files

[Tim Armstrong (Code Review)]

Tim Armstrong has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9134 )

Change subject: IMPALA-5717: Support for reading ORC data files
..


Patch Set 13:

Actually I think the ASAN build is just slow - not sure if the LLVM 5 upgrade 
is related.


--
To view, visit http://gerrit.cloudera.org:8080/9134
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia7b6ae4ce3b9ee8125b21993702faa87537790a4
Gerrit-Change-Number: 9134
Gerrit-PatchSet: 13
Gerrit-Owner: Quanlong Huang 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Fri, 06 Apr 2018 00:20:13 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-5717: Support for reading ORC data files

[Tim Armstrong (Code Review)]

Tim Armstrong has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9134 )

Change subject: IMPALA-5717: Support for reading ORC data files
..


Patch Set 13: Code-Review+2

I'm still in the process of running tests. I'm a little concerned that the 
AddressSanitizer test is hung, but I may not have time to investigate before 
the weekend (I'm taking Friday off).

AddressSanitizer is easy enough to run locally - you just pass the -asan option 
to buildall.sh.

S3 is tricky to set up - you need a data snapshot and an S3 bucket. I'm not 
sure that I'm actually going to be able to run that precommit because it's not 
trivial to get it set up correctly with new data.


--
To view, visit http://gerrit.cloudera.org:8080/9134
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia7b6ae4ce3b9ee8125b21993702faa87537790a4
Gerrit-Change-Number: 9134
Gerrit-PatchSet: 13
Gerrit-Owner: Quanlong Huang 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Fri, 06 Apr 2018 00:16:29 +
Gerrit-HasComments: No

[Impala-ASF-CR] [DOCS] Removed old files no longer in use

[Impala Public Jenkins (Code Review)]

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9938 )

Change subject: [DOCS] Removed old files no longer in use
..


Patch Set 1: Verified+1


--
To view, visit http://gerrit.cloudera.org:8080/9938
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0aaa756bf0fc0092186ebc293543408b403aa5e
Gerrit-Change-Number: 9938
Gerrit-PatchSet: 1
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: John Russell 
Gerrit-Comment-Date: Thu, 05 Apr 2018 23:49:55 +
Gerrit-HasComments: No

[Impala-ASF-CR] KUDU-2401: External TLS certificate with Intermediate CA in server cert file fails

[Sailesh Mukil (Code Review)]

Sailesh Mukil has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9940 )

Change subject: KUDU-2401: External TLS certificate with Intermediate CA in 
server cert file fails
..


Patch Set 1:

This is the fix for the blocker IMPALA-6806.
The cherry-pick was clean.


--
To view, visit http://gerrit.cloudera.org:8080/9940
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If4af35e97ec6f91c1d9ed902128bd7f4e260f0f4
Gerrit-Change-Number: 9940
Gerrit-PatchSet: 1
Gerrit-Owner: Sailesh Mukil 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Comment-Date: Thu, 05 Apr 2018 23:44:55 +
Gerrit-HasComments: No

[Impala-ASF-CR] KUDU-2401: External TLS certificate with Intermediate CA in server cert file fails

[Sailesh Mukil (Code Review)]

Sailesh Mukil has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/9940


Change subject: KUDU-2401: External TLS certificate with Intermediate CA in 
server cert file fails
..

KUDU-2401: External TLS certificate with Intermediate CA in server cert file 
fails

Take 2 certificate files: cert.pem and truststore.pem

cert.pem has 2 certificates in it:
A cert for that node (with CN="hostname", and signed by CN=CertToolkitIntCA)
And the intermediate CA cert (with CN=CertToolkitIntCA, and signed by 
CN=CertToolkitRootCA)

truststore.pem has 1 certificate in it:
A cert which is the root CA (with CN=CertToolkitRootCA, self-signed)

This previously would not work with KRPC because in
TlsContext::VerifyCertChainUnlocked(), we would only verify X509_verify_cert()
with the top certificate in the server certificate chain.

With this change, we pass the chain to X509_STORE_CTX_init() as well to make
sure that the entire chain gets checked against the CA.

A test is added that uses the specific certificate format mentioned above
and added to rpc-test.

TODO: Add a test case that has multiple intermediate CAs. Right now we're 
testing
with only one intermediate CA.

Change-Id: If4af35e97ec6f91c1d9ed902128bd7f4e260f0f4
---
M be/src/kudu/rpc/rpc-test.cc
M be/src/kudu/security/test/test_certs.cc
M be/src/kudu/security/test/test_certs.h
M be/src/kudu/security/tls_context.cc
4 files changed, 263 insertions(+), 4 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/40/9940/1
--
To view, visit http://gerrit.cloudera.org:8080/9940
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: If4af35e97ec6f91c1d9ed902128bd7f4e260f0f4
Gerrit-Change-Number: 9940
Gerrit-PatchSet: 1
Gerrit-Owner: Sailesh Mukil 

[Impala-ASF-CR] [DOCS] Removed old files no longer in use

[John Russell (Code Review)]

John Russell has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9938 )

Change subject: [DOCS] Removed old files no longer in use
..


Patch Set 1: Code-Review+2

I verified the changes and did a clean test build.


--
To view, visit http://gerrit.cloudera.org:8080/9938
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0aaa756bf0fc0092186ebc293543408b403aa5e
Gerrit-Change-Number: 9938
Gerrit-PatchSet: 1
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: John Russell 
Gerrit-Comment-Date: Thu, 05 Apr 2018 23:41:12 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6807: [DOCS] Update the known issue for HDFS-12528

[Impala Public Jenkins (Code Review)]

Impala Public Jenkins has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/9929 )

Change subject: IMPALA-6807: [DOCS] Update the known issue for HDFS-12528
..

IMPALA-6807: [DOCS] Update the known issue for HDFS-12528

Added a new recommendation for the new setting with the fix version
of HDFS, 2.10 and higher.

Change-Id: If51cb111a9ddc67be4a1cf42502a8a021486b7e4
Reviewed-on: http://gerrit.cloudera.org:8080/9929
Reviewed-by: Joe McDonnell 
Tested-by: Impala Public Jenkins 
---
M docs/topics/impala_known_issues.xml
1 file changed, 45 insertions(+), 16 deletions(-)

Approvals:
  Joe McDonnell: Looks good to me, approved
  Impala Public Jenkins: Verified

--
To view, visit http://gerrit.cloudera.org:8080/9929
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: If51cb111a9ddc67be4a1cf42502a8a021486b7e4
Gerrit-Change-Number: 9929
Gerrit-PatchSet: 4
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 

[Impala-ASF-CR] IMPALA-6807: [DOCS] Update the known issue for HDFS-12528

[Impala Public Jenkins (Code Review)]

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9929 )

Change subject: IMPALA-6807: [DOCS] Update the known issue for HDFS-12528
..


Patch Set 3: Verified+1


--
To view, visit http://gerrit.cloudera.org:8080/9929
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If51cb111a9ddc67be4a1cf42502a8a021486b7e4
Gerrit-Change-Number: 9929
Gerrit-PatchSet: 3
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Comment-Date: Thu, 05 Apr 2018 23:35:59 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6807: [DOCS] Update the known issue for HDFS-12528

[Impala Public Jenkins (Code Review)]

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9929 )

Change subject: IMPALA-6807: [DOCS] Update the known issue for HDFS-12528
..


Patch Set 3:

Build started: https://jenkins.impala.io/job/gerrit-docs-submit/225/


--
To view, visit http://gerrit.cloudera.org:8080/9929
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If51cb111a9ddc67be4a1cf42502a8a021486b7e4
Gerrit-Change-Number: 9929
Gerrit-PatchSet: 3
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Comment-Date: Thu, 05 Apr 2018 23:27:14 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6807: [DOCS] Update the known issue for HDFS-12528

[Joe McDonnell (Code Review)]

Joe McDonnell has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9929 )

Change subject: IMPALA-6807: [DOCS] Update the known issue for HDFS-12528
..


Patch Set 3: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/9929
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If51cb111a9ddc67be4a1cf42502a8a021486b7e4
Gerrit-Change-Number: 9929
Gerrit-PatchSet: 3
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Comment-Date: Thu, 05 Apr 2018 23:17:31 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-2195: Improper handling of comments in queries

[Fredy Wijaya (Code Review)]

Fredy Wijaya has uploaded a new patch set (#3). ( 
http://gerrit.cloudera.org:8080/9933 )

Change subject: IMPALA-2195: Improper handling of comments in queries
..

IMPALA-2195: Improper handling of comments in queries

This patch fixes an issue where parseline is unable to determine the
correct command when a statement has a trailing comment.

Before:
> -- comment
> insert into table t values(100);
Fetched 1 row(s) in 0.01s

After:
> -- comment
> insert into table t values(100);
Modified 1 row(s) in 0.01s

Testing:
- Added shell tests
- Ran end-to-end shell tests

Change-Id: I7ac7cb5a30e6dda73ebe761d9f0eb9ba038e14a7
---
M shell/impala_shell.py
M tests/shell/test_shell_interactive.py
2 files changed, 67 insertions(+), 6 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/33/9933/3
--
To view, visit http://gerrit.cloudera.org:8080/9933
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I7ac7cb5a30e6dda73ebe761d9f0eb9ba038e14a7
Gerrit-Change-Number: 9933
Gerrit-PatchSet: 3
Gerrit-Owner: Fredy Wijaya 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Bharath Vissapragada 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Taras Bobrovytsky 

[Impala-ASF-CR] [DOCS] Removed old files no longer in use

[Alex Rodoni (Code Review)]

Alex Rodoni has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/9938


Change subject: [DOCS] Removed old files no longer in use
..

[DOCS] Removed old files no longer in use

Change-Id: Ia0aaa756bf0fc0092186ebc293543408b403aa5e
---
M docs/impala.ditamap
R docs/impala_release_notes.ditamap
D docs/topics/impala_alter_function.xml
D docs/topics/impala_window_functions.xml
4 files changed, 2 insertions(+), 82 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/38/9938/1
--
To view, visit http://gerrit.cloudera.org:8080/9938
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ia0aaa756bf0fc0092186ebc293543408b403aa5e
Gerrit-Change-Number: 9938
Gerrit-PatchSet: 1
Gerrit-Owner: Alex Rodoni 

[Impala-ASF-CR] IMPALA-6810, IMPALA-6814: Skip remote tests that expect a local env

[David Knupp (Code Review)]

David Knupp has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/9937


Change subject: IMPALA-6810, IMPALA-6814: Skip remote tests that expect a local 
env
..

IMPALA-6810, IMPALA-6814: Skip remote tests that expect a local env

Change-Id: Icfc1c6b56c9721a4641c939d4bebabfc0600281f
---
M tests/query_test/test_queries.py
M tests/query_test/test_runtime_filters.py
2 files changed, 2 insertions(+), 0 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/37/9937/1
--
To view, visit http://gerrit.cloudera.org:8080/9937
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Icfc1c6b56c9721a4641c939d4bebabfc0600281f
Gerrit-Change-Number: 9937
Gerrit-PatchSet: 1
Gerrit-Owner: David Knupp 

[Impala-ASF-CR] IMPALA-6070: Expose using Docker to run tests faster.

[Impala Public Jenkins (Code Review)]

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9085 )

Change subject: IMPALA-6070: Expose using Docker to run tests faster.
..


Patch Set 7:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2246/


--
To view, visit http://gerrit.cloudera.org:8080/9085
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I82052ef31979564968effef13a3c6af0d5c62767
Gerrit-Change-Number: 9085
Gerrit-PatchSet: 7
Gerrit-Owner: Philip Zeyliger 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Jim Apple 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Taras Bobrovytsky 
Gerrit-Comment-Date: Thu, 05 Apr 2018 20:59:44 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6070: Expose using Docker to run tests faster.

[Philip Zeyliger (Code Review)]

Philip Zeyliger has uploaded a new patch set (#7). ( 
http://gerrit.cloudera.org:8080/9085 )

Change subject: IMPALA-6070: Expose using Docker to run tests faster.
..

IMPALA-6070: Expose using Docker to run tests faster.

Allows running the tests that make up the "core" suite in about 2 hours.
By comparison, 
https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/buildTimeTrend
tends to run in about 3.5 hours.

This commit:
* Adds "echo" statements in a few places, to facilitate timing.
* Adds --skip-parallel/--skip-serial flags to run-tests.py,
  and exposes them in run-all-tests.sh.
* Marks TestRuntimeFilters as a serial test. This test runs
  queries that need > 1GB of memory, and, combined with
  other tests running in parallel, can kill the parallel test
  suite.
* Adds "test-with-docker.py", which runs a full build, data load,
  and executes tests inside of Docker containers, generating
  a timeline at the end. In short, one container is used
  to do the build and data load, and then this container is
  re-used to run various tests in parallel. All logs are
  left on the host system.

Besides the obvious win of getting test results more quickly, this
commit serves as an example of how to get various bits of Impala
development working inside of Docker containers. For example, Kudu
relies on atomic rename of directories, which isn't available in most
Docker filesystems, and entrypoint.sh works around it.

In addition, the timeline generated by the build suggests where further
optimizations can be made. Most obviously, dataload eats up a precious
~30-50 minutes, on a largely idle machine.

This work is significantly CPU and memory hungry. It was developed on a
32-core, 120GB RAM Google Compute Engine machine. I've worked out
parallelism configurations such that it runs nicely on 60GB of RAM
(c4.8xlarge) and over 100GB (eg., m4.10xlarge, which has 160GB). There is
some simple logic to guess at some knobs, and there are knobs.  By and
large, EC2 and GCE price machines linearly, so, if CPU usage can be kept
up, it's not wasteful to run on bigger machines.

Change-Id: I82052ef31979564968effef13a3c6af0d5c62767
---
M bin/bootstrap_system.sh
M bin/run-all-tests.sh
M buildall.sh
A docker/README.md
A docker/annotate.py
A docker/entrypoint.sh
A docker/monitor.py
A docker/test-with-docker.py
A docker/timeline.html.template
M testdata/bin/run-all.sh
M tests/query_test/test_runtime_filters.py
M tests/run-tests.py
12 files changed, 1,466 insertions(+), 13 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/85/9085/7
--
To view, visit http://gerrit.cloudera.org:8080/9085
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I82052ef31979564968effef13a3c6af0d5c62767
Gerrit-Change-Number: 9085
Gerrit-PatchSet: 7
Gerrit-Owner: Philip Zeyliger 
Gerrit-Reviewer: Jim Apple 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Taras Bobrovytsky 

[Impala-ASF-CR] IMPALA-6389: Make '\0' delimited text files work

[Zach Amsden (Code Review)]

Zach Amsden has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/9857 )

Change subject: IMPALA-6389: Make '\0' delimited text files work
..

IMPALA-6389: Make '\0' delimited text files work

Initially I didn't want to fully implement this, as the metadata
for these tables can't even be fully stored in Postgres; however
after digging into some older documentation, it appears that the
ASCII NUL character actually has been used as a field separator
in various vendors CSV implementation.

Therefore, this patch attempts to make things as non-broken as
possible and allows \0 as a field or tuple delimiter.  Collection
column delimiters are not allowed to be \0, as they genuinly may
not exist and we don't want to force special escaping on an
arbitrary character.  Note that the field delimiter must be distinct
from the tuple delimiter when they both exist; if it is not, the
effect will be that there is no field delimiter (this is actually
possible with single column tables).

Testing: Created a zero delimited table as described in the JIRA,
using MySQL backed Hive metastore; ran select * from tab_separated
on the table, updated the unit test.  Additionally, build ASAN
and ran the unit test.

Change-Id: I2190c57681f29f34ee1eb393e30dfdda5839098c
Reviewed-on: http://gerrit.cloudera.org:8080/9857
Tested-by: Impala Public Jenkins
Reviewed-by: Zach Amsden 
---
M be/src/exec/delimited-text-parser-test.cc
M be/src/exec/delimited-text-parser.cc
M be/src/exec/delimited-text-parser.h
M be/src/exec/delimited-text-parser.inline.h
M be/src/exec/hdfs-sequence-scanner.cc
M be/src/exec/hdfs-sequence-scanner.h
M be/src/exec/hdfs-text-scanner.cc
M be/src/exec/hdfs-text-scanner.h
8 files changed, 172 insertions(+), 86 deletions(-)

Approvals:
  Impala Public Jenkins: Verified
  Zach Amsden: Looks good to me, approved

--
To view, visit http://gerrit.cloudera.org:8080/9857
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I2190c57681f29f34ee1eb393e30dfdda5839098c
Gerrit-Change-Number: 9857
Gerrit-PatchSet: 4
Gerrit-Owner: Zach Amsden 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Reviewer: Zach Amsden 

[Impala-ASF-CR] IMPALA-2717: fix output of formatted unicode to non-TTY

[Tim Armstrong (Code Review)]

Tim Armstrong has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9928 )

Change subject: IMPALA-2717: fix output of formatted unicode to non-TTY
..


Patch Set 1:

(3 comments)

http://gerrit.cloudera.org:8080/#/c/9928/1/shell/shell_output.py
File shell/shell_output.py:

http://gerrit.cloudera.org:8080/#/c/9928/1/shell/shell_output.py@31
PS1, Line 31:
> nit: remove space
Done


http://gerrit.cloudera.org:8080/#/c/9928/1/shell/shell_output.py@45
PS1, Line 45: row
> should this be encoded in the same way?
It's already a str (verified by printing type(rows[0][0])).

I added a test for this code path too, since it is different.


http://gerrit.cloudera.org:8080/#/c/9928/1/shell/shell_output.py@64
PS1, Line 64: rows
> should this be "rows.encode(...)"?
I checked that the returned rows is already a str instead of a unicode, so 
re-encoding it is not necessary. We already have test coverage for this case.



--
To view, visit http://gerrit.cloudera.org:8080/9928
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9de641ecf767a2feef3b9f48b344ef2d55e17a7f
Gerrit-Change-Number: 9928
Gerrit-PatchSet: 1
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Thu, 05 Apr 2018 18:31:09 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-2717: fix output of formatted unicode to non-TTY

[Tim Armstrong (Code Review)]

Hello Lars Volker,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/9928

to look at the new patch set (#2).

Change subject: IMPALA-2717: fix output of formatted unicode to non-TTY
..

IMPALA-2717: fix output of formatted unicode to non-TTY

The bug is that PrettyOutputFormatter.format() returned a unicode
object, and Python cannot automatically write unicode objects to
output streams where there is no default encoding.

The fix is to convert to UTF-8 encoded in a regular string, which
can be output to any output device. This makes the output type
consistent with DelimitedOutputFormatter.format().

Based on code by Marcell Szabo.

Testing:
Added a basic test.

Played around in an interactive shell to make sure that unicode
characters still work in interactive mode.

Change-Id: I9de641ecf767a2feef3b9f48b344ef2d55e17a7f
---
M shell/shell_output.py
M tests/shell/test_shell_commandline.py
2 files changed, 25 insertions(+), 5 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/28/9928/2
--
To view, visit http://gerrit.cloudera.org:8080/9928
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I9de641ecf767a2feef3b9f48b344ef2d55e17a7f
Gerrit-Change-Number: 9928
Gerrit-PatchSet: 2
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Lars Volker 

[Impala-ASF-CR] IMPALA-6389: Make '\0' delimited text files work

[Zach Amsden (Code Review)]

Zach Amsden has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9857 )

Change subject: IMPALA-6389: Make '\0' delimited text files work
..


Patch Set 3: Code-Review+2

Yes, I did - forgot to carry the +2 after rebase


--
To view, visit http://gerrit.cloudera.org:8080/9857
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2190c57681f29f34ee1eb393e30dfdda5839098c
Gerrit-Change-Number: 9857
Gerrit-PatchSet: 3
Gerrit-Owner: Zach Amsden 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Reviewer: Zach Amsden 
Gerrit-Comment-Date: Thu, 05 Apr 2018 18:27:35 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6807: [DOCS] Update the known issue for HDFS-12528

[Alex Rodoni (Code Review)]

Hello Joe McDonnell, Impala Public Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/9929

to look at the new patch set (#3).

Change subject: IMPALA-6807: [DOCS] Update the known issue for HDFS-12528
..

IMPALA-6807: [DOCS] Update the known issue for HDFS-12528

Added a new recommendation for the new setting with the fix version
of HDFS, 2.10 and higher.

Change-Id: If51cb111a9ddc67be4a1cf42502a8a021486b7e4
---
M docs/topics/impala_known_issues.xml
1 file changed, 45 insertions(+), 16 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/29/9929/3
--
To view, visit http://gerrit.cloudera.org:8080/9929
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: If51cb111a9ddc67be4a1cf42502a8a021486b7e4
Gerrit-Change-Number: 9929
Gerrit-PatchSet: 3
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Joe McDonnell 

[Impala-ASF-CR] IMPALA-6549: [DOCS] Update the known issue for HDFS-12528

[Alex Rodoni (Code Review)]

Alex Rodoni has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9929 )

Change subject: IMPALA-6549: [DOCS] Update the known issue for HDFS-12528
..


Patch Set 2:

(3 comments)

http://gerrit.cloudera.org:8080/#/c/9929/2/docs/topics/impala_known_issues.xml
File docs/topics/impala_known_issues.xml:

http://gerrit.cloudera.org:8080/#/c/9929/2/docs/topics/impala_known_issues.xml@450
PS2, Line 450: to be
> "to be" -> "are"
Done


http://gerrit.cloudera.org:8080/#/c/9929/2/docs/topics/impala_known_issues.xml@453
PS2, Line 453:
> Nit: stray space
Done


http://gerrit.cloudera.org:8080/#/c/9929/2/docs/topics/impala_known_issues.xml@453
PS2, Line 453:  0
 :   is not recommended.
> Setting dfs.domain.socket.disable.interval.seconds to 0 is not recommended,
Done



--
To view, visit http://gerrit.cloudera.org:8080/9929
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If51cb111a9ddc67be4a1cf42502a8a021486b7e4
Gerrit-Change-Number: 9929
Gerrit-PatchSet: 2
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Comment-Date: Thu, 05 Apr 2018 16:47:59 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6549: [DOCS] Update the known issue for HDFS-12528

[Joe McDonnell (Code Review)]

Joe McDonnell has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9929 )

Change subject: IMPALA-6549: [DOCS] Update the known issue for HDFS-12528
..


Patch Set 2:

(3 comments)

http://gerrit.cloudera.org:8080/#/c/9929/2/docs/topics/impala_known_issues.xml
File docs/topics/impala_known_issues.xml:

http://gerrit.cloudera.org:8080/#/c/9929/2/docs/topics/impala_known_issues.xml@450
PS2, Line 450: to be
"to be" -> "are"

I'd like this sentence to be clear about when short circuit reads are disabled. 
Something like:

Use the new HDFS parameter blah to specify the amount of time short circuit 
reads are disabled on encountering an error.


http://gerrit.cloudera.org:8080/#/c/9929/2/docs/topics/impala_known_issues.xml@453
PS2, Line 453:
Nit: stray space


http://gerrit.cloudera.org:8080/#/c/9929/2/docs/topics/impala_known_issues.xml@453
PS2, Line 453:  0
 :   is not recommended.
Setting dfs.domain.socket.disable.interval.seconds to 0 is not recommended, as 
a non-zero interval protects the system if there is a persistent problem with 
short circuit reads.



--
To view, visit http://gerrit.cloudera.org:8080/9929
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If51cb111a9ddc67be4a1cf42502a8a021486b7e4
Gerrit-Change-Number: 9929
Gerrit-PatchSet: 2
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Comment-Date: Thu, 05 Apr 2018 16:25:03 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-5903: Inconsistent specification of result set and result set metadata

[Tim Armstrong (Code Review)]

Tim Armstrong has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9090 )

Change subject: IMPALA-5903: Inconsistent specification of result set and 
result set metadata
..


Patch Set 6: Code-Review+1


--
To view, visit http://gerrit.cloudera.org:8080/9090
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic542fb8e49e850052416ac663ee329ee3974e3b9
Gerrit-Change-Number: 9090
Gerrit-PatchSet: 6
Gerrit-Owner: Zoltan Borok-Nagy 
Gerrit-Reviewer: Dimitris Tsirogiannis 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Reviewer: Zoltan Borok-Nagy 
Gerrit-Comment-Date: Thu, 05 Apr 2018 16:13:01 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6793: Fix empty metadata after statestore restarts

[Alex Behm (Code Review)]

Alex Behm has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9921 )

Change subject: IMPALA-6793: Fix empty metadata after statestore restarts
..


Patch Set 3:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/9921/3/be/src/catalog/catalog-server.cc
File be/src/catalog/catalog-server.cc:

http://gerrit.cloudera.org:8080/#/c/9921/3/be/src/catalog/catalog-server.cc@256
PS3, Line 256:   }
> If statestore restarts, branch "if (delta.from_version == 0 && last_sent_ca
It also used to be the case that heartbeats were piggy-backed onto topic 
updates, so a long topic could lead the statestore to believe the catalogd was 
down. So I agree that it makes sense to revisit the async collection process, 
but we need to be careful.



--
To view, visit http://gerrit.cloudera.org:8080/9921
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I332a60e172af84b93b3544373fe363cdced5e8d0
Gerrit-Change-Number: 9921
Gerrit-PatchSet: 3
Gerrit-Owner: Tianyi Wang 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Dimitris Tsirogiannis 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Tianyi Wang 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Reviewer: Vuk Ercegovac 
Gerrit-Comment-Date: Thu, 05 Apr 2018 16:07:48 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-2195: Improper handling of comments in queries

[Alex Behm (Code Review)]

Alex Behm has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9933 )

Change subject: IMPALA-2195: Improper handling of comments in queries
..


Patch Set 2:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/9933/2/shell/impala_shell.py
File shell/impala_shell.py:

http://gerrit.cloudera.org:8080/#/c/9933/2/shell/impala_shell.py@1279
PS2, Line 1279: line = sqlparse.format(line.strip(), 
strip_comments=True).encode('utf-8')
> looks like this strips the query hints too? (Users can provide query hints
Would also be good to look how often we parse this string. IIRC, we invoke 
sqlparse quite a few times, and it's not exactly cheap. Perhaps we can parse it 
once and reuse the result in the various relevant places.



--
To view, visit http://gerrit.cloudera.org:8080/9933
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7ac7cb5a30e6dda73ebe761d9f0eb9ba038e14a7
Gerrit-Change-Number: 9933
Gerrit-PatchSet: 2
Gerrit-Owner: Fredy Wijaya 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Bharath Vissapragada 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Taras Bobrovytsky 
Gerrit-Comment-Date: Thu, 05 Apr 2018 15:51:13 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-5903: Inconsistent specification of result set and result set metadata

[Zoltan Borok-Nagy (Code Review)]

Zoltan Borok-Nagy has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9090 )

Change subject: IMPALA-5903: Inconsistent specification of result set and 
result set metadata
..


Patch Set 5:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/9090/5/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
File fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java:

http://gerrit.cloudera.org:8080/#/c/9090/5/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@506
PS5, Line 506:   addSummary(response, "New location has been set.");
> I think we need to tweak this case to behave more like SET_ROW_FORMAT and o
Looking at the definition of alterTableSetLocation() it seems to me that if a 
partition spec is provided, then only one partition will be affected. Or, there 
can be subpartitions I guess, but I'm not sure how to count them.

Anyway, I added an if-else with different summaries.



--
To view, visit http://gerrit.cloudera.org:8080/9090
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic542fb8e49e850052416ac663ee329ee3974e3b9
Gerrit-Change-Number: 9090
Gerrit-PatchSet: 5
Gerrit-Owner: Zoltan Borok-Nagy 
Gerrit-Reviewer: Dimitris Tsirogiannis 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Reviewer: Zoltan Borok-Nagy 
Gerrit-Comment-Date: Thu, 05 Apr 2018 15:36:52 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-5903: Inconsistent specification of result set and result set metadata

[Zoltan Borok-Nagy (Code Review)]

Hello Fredy Wijaya, Dimitris Tsirogiannis, Tim Armstrong,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/9090

to look at the new patch set (#6).

Change subject: IMPALA-5903: Inconsistent specification of result set and 
result set metadata
..

IMPALA-5903: Inconsistent specification of result set and result set metadata

Before this commit it was quite random which DDL oprations
returned a result set and which didn't.

With this commit, every DDL operations return a summary of
its execution. They declare their result set schema in
Frontend.java, and provide the summary in CalatogOpExecutor.java.

Updated the tests according to the new behavior.

Change-Id: Ic542fb8e49e850052416ac663ee329ee3974e3b9
---
M be/src/service/client-request-state.cc
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
M fe/src/main/java/org/apache/impala/service/Frontend.java
M shell/impala_client.py
M testdata/workloads/functional-query/queries/QueryTest/alter-table.test
M testdata/workloads/functional-query/queries/QueryTest/chars-tmp-tables.test
M 
testdata/workloads/functional-query/queries/QueryTest/compute-stats-incremental.test
M testdata/workloads/functional-query/queries/QueryTest/compute-stats.test
M testdata/workloads/functional-query/queries/QueryTest/create-database.test
M 
testdata/workloads/functional-query/queries/QueryTest/create-table-like-file.test
M 
testdata/workloads/functional-query/queries/QueryTest/create-table-like-table.test
M testdata/workloads/functional-query/queries/QueryTest/create-table.test
M 
testdata/workloads/functional-query/queries/QueryTest/delimited-latin-text.test
M testdata/workloads/functional-query/queries/QueryTest/delimited-text.test
M testdata/workloads/functional-query/queries/QueryTest/describe-path.test
M testdata/workloads/functional-query/queries/QueryTest/functions-ddl.test
M 
testdata/workloads/functional-query/queries/QueryTest/hbase-compute-stats-incremental.test
M testdata/workloads/functional-query/queries/QueryTest/insert_bad_expr.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_alter.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_create.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_insert.test
M testdata/workloads/functional-query/queries/QueryTest/load.test
M testdata/workloads/functional-query/queries/QueryTest/local-filesystem.test
M 
testdata/workloads/functional-query/queries/QueryTest/partition-ddl-predicates-all-fs.test
M testdata/workloads/functional-query/queries/QueryTest/truncate-table.test
M testdata/workloads/functional-query/queries/QueryTest/views-ddl.test
26 files changed, 291 insertions(+), 70 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/90/9090/6
--
To view, visit http://gerrit.cloudera.org:8080/9090
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ic542fb8e49e850052416ac663ee329ee3974e3b9
Gerrit-Change-Number: 9090
Gerrit-PatchSet: 6
Gerrit-Owner: Zoltan Borok-Nagy 
Gerrit-Reviewer: Dimitris Tsirogiannis 
Gerrit-Reviewer: Fredy Wijaya 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Reviewer: Zoltan Borok-Nagy 

[Impala-ASF-CR] IMPALA-6314: Add run time scalar subquery check for uncorrelated subqueries

[Zoltan Borok-Nagy (Code Review)]

Zoltan Borok-Nagy has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9005 )

Change subject: IMPALA-6314: Add run time scalar subquery check for 
uncorrelated subqueries
..


Patch Set 21:

(3 comments)

http://gerrit.cloudera.org:8080/#/c/9005/11/fe/src/main/java/org/apache/impala/analysis/ArithmeticExpr.java
File fe/src/main/java/org/apache/impala/analysis/ArithmeticExpr.java:

http://gerrit.cloudera.org:8080/#/c/9005/11/fe/src/main/java/org/apache/impala/analysis/ArithmeticExpr.java@175
PS11, Line 175:   if (!operand.type_.isNumericType() && 
!operand.type_.isNull()) {
> Sorry confusing sentence. I mean:
Yeah, now I vaguely remember from those formal languages courses that the type 
deduction should only go bottom-up. :)
(However, we still set isRuntimeScalar based on the context)

There are some places where I have to use the old semantics of 
isScalarSubquery(), ie. to only test that the type of a subquery is scalar and 
not test whether it returns multiple rows.


http://gerrit.cloudera.org:8080/#/c/9005/19/fe/src/main/java/org/apache/impala/analysis/ExistsPredicate.java
File fe/src/main/java/org/apache/impala/analysis/ExistsPredicate.java:

http://gerrit.cloudera.org:8080/#/c/9005/19/fe/src/main/java/org/apache/impala/analysis/ExistsPredicate.java@66
PS19, Line 66:   @Override
> We really should't be setting the type of an expression from the outside. T
Removed this line.


http://gerrit.cloudera.org:8080/#/c/9005/21/fe/src/main/java/org/apache/impala/analysis/StmtRewriter.java
File fe/src/main/java/org/apache/impala/analysis/StmtRewriter.java:

http://gerrit.cloudera.org:8080/#/c/9005/21/fe/src/main/java/org/apache/impala/analysis/StmtRewriter.java@a614
PS21, Line 614:
This wasn't needed for this JIRA, but looked weird, so I simplified it.



--
To view, visit http://gerrit.cloudera.org:8080/9005
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I0f52b93a60eeacedd242a2f17fa6b99c4fc38e06
Gerrit-Change-Number: 9005
Gerrit-PatchSet: 21
Gerrit-Owner: Zoltan Borok-Nagy 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Attila Jeges 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Dimitris Tsirogiannis 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Reviewer: Vuk Ercegovac 
Gerrit-Reviewer: Zoltan Borok-Nagy 
Gerrit-Comment-Date: Thu, 05 Apr 2018 12:45:48 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6314: Add run time scalar subquery check for uncorrelated subqueries

[Zoltan Borok-Nagy (Code Review)]

Hello Attila Jeges, Dimitris Tsirogiannis, Tim Armstrong, Csaba Ringhofer, Alex 
Behm, Vuk Ercegovac,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/9005

to look at the new patch set (#21).

Change subject: IMPALA-6314: Add run time scalar subquery check for 
uncorrelated subqueries
..

IMPALA-6314: Add run time scalar subquery check for uncorrelated subqueries

If a scalar subquery is used with a binary predicate,
or, used in an arithmetic expression, it must return
only one row/column to be valid. If this cannot be
guaranteed at parse time through a single row aggregate
or limit clause, Impala fails the query like such.

E.g., currently the following query is not allowed:
SELECT bigint_col
FROM alltypesagg
WHERE id = (SELECT id FROM alltypesagg WHERE id = 1)

However, it would be allowed if the query contained
a LIMIT 1 clause, or instead of id it was max(id).

This commit makes the example valid by introducing a
runtime check to test if the subquery returns a single
row. If the subquery returns more than one row, it
aborts the query with an error.

I added a new node type, called CardinalityCheckNode. It
is created during planning on top of the subquery when
needed, then during execution it checks if its child
only returns a single row.

I extended the frontend tests and e2e tests as well.

Change-Id: I0f52b93a60eeacedd242a2f17fa6b99c4fc38e06
---
M be/src/exec/CMakeLists.txt
A be/src/exec/cardinality-check-node.cc
A be/src/exec/cardinality-check-node.h
M be/src/exec/exec-node.cc
M common/thrift/PlanNodes.thrift
M fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java
M fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java
M fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java
M fe/src/main/java/org/apache/impala/analysis/CreateViewStmt.java
M fe/src/main/java/org/apache/impala/analysis/ExistsPredicate.java
M fe/src/main/java/org/apache/impala/analysis/Expr.java
M fe/src/main/java/org/apache/impala/analysis/HdfsCachingOp.java
M fe/src/main/java/org/apache/impala/analysis/InPredicate.java
M fe/src/main/java/org/apache/impala/analysis/QueryStmt.java
M fe/src/main/java/org/apache/impala/analysis/SelectStmt.java
M fe/src/main/java/org/apache/impala/analysis/StatementBase.java
M fe/src/main/java/org/apache/impala/analysis/StmtRewriter.java
M fe/src/main/java/org/apache/impala/analysis/Subquery.java
A fe/src/main/java/org/apache/impala/planner/CardinalityCheckNode.java
M fe/src/main/java/org/apache/impala/planner/DistributedPlanner.java
M fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeSubqueriesTest.java
M fe/src/test/java/org/apache/impala/analysis/ToSqlTest.java
M 
testdata/workloads/functional-planner/queries/PlannerTest/nested-collections.test
M 
testdata/workloads/functional-planner/queries/PlannerTest/subquery-rewrite.test
M 
testdata/workloads/functional-query/queries/QueryTest/nested-types-subplan.test
M testdata/workloads/functional-query/queries/QueryTest/subquery.test
28 files changed, 914 insertions(+), 69 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/05/9005/21
--
To view, visit http://gerrit.cloudera.org:8080/9005
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I0f52b93a60eeacedd242a2f17fa6b99c4fc38e06
Gerrit-Change-Number: 9005
Gerrit-PatchSet: 21
Gerrit-Owner: Zoltan Borok-Nagy 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Attila Jeges 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Dimitris Tsirogiannis 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Reviewer: Vuk Ercegovac 
Gerrit-Reviewer: Zoltan Borok-Nagy 

[Impala-ASF-CR] IMPALA-5717: Support for reading ORC data files

[Quanlong Huang (Code Review)]

Quanlong Huang has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9134 )

Change subject: IMPALA-5717: Support for reading ORC data files
..


Patch Set 13:

(4 comments)

Sure, Tim. Thank you for doing local tests for me! Just refactor the patch.

BTW, how do you test it in S3 and AddressSanitizer? Is there a doc for this?

http://gerrit.cloudera.org:8080/#/c/9134/12/be/src/exec/hdfs-orc-scanner.cc
File be/src/exec/hdfs-orc-scanner.cc:

http://gerrit.cloudera.org:8080/#/c/9134/12/be/src/exec/hdfs-orc-scanner.cc@24
PS12, Line 24: #include "runtime/runtime-filter.inline.h"
> Some includes not needed:
Done


http://gerrit.cloudera.org:8080/#/c/9134/12/be/src/exec/hdfs-orc-scanner.cc@350
PS12, Line 350: default: break;
> Should include filename() in the message so users can trace back the error.
Done


http://gerrit.cloudera.org:8080/#/c/9134/12/be/src/exec/hdfs-orc-scanner.cc@355
PS12, Line 355: }
> Should include filename() in the message so users can trace back the error.
Done


http://gerrit.cloudera.org:8080/#/c/9134/12/be/src/exec/hdfs-orc-scanner.cc@606
PS12, Line 606: return Co
> I think this should be VLOG_ROW because it logs many times per file.
Done



--
To view, visit http://gerrit.cloudera.org:8080/9134
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia7b6ae4ce3b9ee8125b21993702faa87537790a4
Gerrit-Change-Number: 9134
Gerrit-PatchSet: 13
Gerrit-Owner: Quanlong Huang 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Thu, 05 Apr 2018 09:31:42 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-5717: Support for reading ORC data files

[Quanlong Huang (Code Review)]

Hello Tim Armstrong, Joe McDonnell, Dan Hecht,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/9134

to look at the new patch set (#13).

Change subject: IMPALA-5717: Support for reading ORC data files
..

IMPALA-5717: Support for reading ORC data files

This patch integrates the orc library into Impala and implements
HdfsOrcScanner as a middle layer between them. The HdfsOrcScanner
supplies input needed from the orc-reader, tracks memory consumption of
the reader and transfers the reader's output (orc::ColumnVectorBatch)
into impala::RowBatch. The ORC version we used is release-1.4.3.

A startup option --enable_orc_scanner is added for this feature. It's
set to true by default. Setting it to false will fail queries on ORC
tables.

Currently, we only support reading primitive types. Writing into ORC
table has not been supported neither.

Tests
 - Most of the end-to-end tests can run on ORC format.
 - Add tpcds, tpch tests for ORC.
 - Add some ORC specific tests.
 - Haven't enabled test_scanner_fuzz for ORC yet, since the ORC library
   is not robust for corrupt files (ORC-315).

Change-Id: Ia7b6ae4ce3b9ee8125b21993702faa87537790a4
---
M CMakeLists.txt
M be/CMakeLists.txt
M be/src/codegen/gen_ir_descriptions.py
M be/src/exec/CMakeLists.txt
A be/src/exec/hdfs-orc-scanner.cc
A be/src/exec/hdfs-orc-scanner.h
M be/src/exec/hdfs-parquet-scanner-ir.cc
M be/src/exec/hdfs-parquet-scanner.cc
M be/src/exec/hdfs-parquet-scanner.h
M be/src/exec/hdfs-scan-node-base.cc
M be/src/exec/hdfs-scan-node-mt.cc
M be/src/exec/hdfs-scanner-ir.cc
M be/src/exec/hdfs-scanner.cc
M be/src/exec/hdfs-scanner.h
M be/src/util/backend-gflag-util.cc
M bin/bootstrap_toolchain.py
M bin/impala-config.sh
A cmake_modules/FindOrc.cmake
M common/thrift/BackendGflags.thrift
M common/thrift/CatalogObjects.thrift
M fe/src/main/cup/sql-parser.cup
M fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java
M fe/src/main/java/org/apache/impala/catalog/HdfsFileFormat.java
M fe/src/main/java/org/apache/impala/catalog/HdfsStorageDescriptor.java
M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java
M fe/src/main/java/org/apache/impala/service/BackendConfig.java
M fe/src/main/java/org/apache/impala/service/Frontend.java
M fe/src/main/jflex/sql-scanner.flex
M testdata/LineItemMultiBlock/README.dox
A testdata/LineItemMultiBlock/lineitem_orc_multiblock_one_stripe.orc
A testdata/LineItemMultiBlock/lineitem_sixblocks.orc
A testdata/LineItemMultiBlock/lineitem_threeblocks.orc
M testdata/bin/create-load-data.sh
M testdata/bin/generate-schema-statements.py
M testdata/bin/run-hive-server.sh
M testdata/cluster/node_templates/common/etc/hadoop/conf/hdfs-site.xml.tmpl
A testdata/data/chars-formats.orc
M testdata/datasets/functional/functional_schema_template.sql
M testdata/datasets/functional/schema_constraints.csv
M 
testdata/workloads/functional-planner/queries/PlannerTest/complex-types-file-formats.test
M testdata/workloads/functional-query/functional-query_core.csv
M testdata/workloads/functional-query/functional-query_dimensions.csv
M testdata/workloads/functional-query/functional-query_exhaustive.csv
M testdata/workloads/functional-query/functional-query_pairwise.csv
A 
testdata/workloads/functional-query/queries/DataErrorsTest/orc-type-checks.test
M testdata/workloads/tpcds/tpcds_core.csv
M testdata/workloads/tpcds/tpcds_dimensions.csv
M testdata/workloads/tpcds/tpcds_exhaustive.csv
M testdata/workloads/tpcds/tpcds_pairwise.csv
M testdata/workloads/tpch/tpch_core.csv
M testdata/workloads/tpch/tpch_dimensions.csv
M testdata/workloads/tpch/tpch_exhaustive.csv
M testdata/workloads/tpch/tpch_pairwise.csv
M tests/common/impala_test_suite.py
M tests/common/test_dimensions.py
M tests/common/test_vector.py
M tests/comparison/cli_options.py
M tests/query_test/test_chars.py
M tests/query_test/test_decimal_queries.py
M tests/query_test/test_scanners.py
M tests/query_test/test_scanners_fuzz.py
M tests/query_test/test_tpch_queries.py
62 files changed, 1,753 insertions(+), 307 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/34/9134/13
--
To view, visit http://gerrit.cloudera.org:8080/9134
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia7b6ae4ce3b9ee8125b21993702faa87537790a4
Gerrit-Change-Number: 9134
Gerrit-PatchSet: 13
Gerrit-Owner: Quanlong Huang 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Reviewer: Tim Armstrong 

[Impala-ASF-CR] Loosen hive-exec.jar glob pattern in copy-udfs-udas.sh.

[Impala Public Jenkins (Code Review)]

Impala Public Jenkins has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/9925 )

Change subject: Loosen hive-exec.jar glob pattern in copy-udfs-udas.sh.
..

Loosen hive-exec.jar glob pattern in copy-udfs-udas.sh.

This commit slightly loosens the coupling between IMPALA_HIVE_VERSION
and "hive.version" in the Maven sense.

Cherry-picks: not for 2.x

Change-Id: Ifbe6f5208b4ad0ffc9cbfe4e93d712ce698beb23
Reviewed-on: http://gerrit.cloudera.org:8080/9925
Reviewed-by: Tim Armstrong 
Tested-by: Impala Public Jenkins
---
M testdata/bin/copy-udfs-udas.sh
1 file changed, 1 insertion(+), 1 deletion(-)

Approvals:
  Tim Armstrong: Looks good to me, approved
  Impala Public Jenkins: Verified

--
To view, visit http://gerrit.cloudera.org:8080/9925
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ifbe6f5208b4ad0ffc9cbfe4e93d712ce698beb23
Gerrit-Change-Number: 9925
Gerrit-PatchSet: 2
Gerrit-Owner: Philip Zeyliger 
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Tim Armstrong 

[Impala-ASF-CR] Loosen hive-exec.jar glob pattern in copy-udfs-udas.sh.

[Impala Public Jenkins (Code Review)]

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9925 )

Change subject: Loosen hive-exec.jar glob pattern in copy-udfs-udas.sh.
..


Patch Set 1: Verified+1


--
To view, visit http://gerrit.cloudera.org:8080/9925
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifbe6f5208b4ad0ffc9cbfe4e93d712ce698beb23
Gerrit-Change-Number: 9925
Gerrit-PatchSet: 1
Gerrit-Owner: Philip Zeyliger 
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Thu, 05 Apr 2018 06:54:52 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-5717: Support for reading ORC data files

[Tim Armstrong (Code Review)]

Tim Armstrong has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9134 )

Change subject: IMPALA-5717: Support for reading ORC data files
..


Patch Set 12:

(4 comments)

Had some minor comments, but I'm ready to +2 this otherwise. I want to do some 
additional testing myself to make sure that it passes some common build types - 
S3, AddressSanitizer, etc, but I can do that in parallel with the last 
revisions.

http://gerrit.cloudera.org:8080/#/c/9134/12/be/src/exec/hdfs-orc-scanner.cc
File be/src/exec/hdfs-orc-scanner.cc:

http://gerrit.cloudera.org:8080/#/c/9134/12/be/src/exec/hdfs-orc-scanner.cc@24
PS12, Line 24: #include "runtime/collection-value-builder.h"
Some includes not needed:
collection-value-builder.h
scoped-buffer.h
bitmap.h
dict-encoding.h


http://gerrit.cloudera.org:8080/#/c/9134/12/be/src/exec/hdfs-orc-scanner.cc@350
PS12, Line 350:   return Status(Substitute("ORC column $0 can't be 
truncated to table column $1",
Should include filename() in the message so users can trace back the error.


http://gerrit.cloudera.org:8080/#/c/9134/12/be/src/exec/hdfs-orc-scanner.cc@355
PS12, Line 355:   return Status(Substitute("Type mismatch: table column $0 is 
map to ORC column $1",
Should include filename() in the message so users can trace back the error.


http://gerrit.cloudera.org:8080/#/c/9134/12/be/src/exec/hdfs-orc-scanner.cc@606
PS12, Line 606: VLOG_FILE
I think this should be VLOG_ROW because it logs many times per file.



--
To view, visit http://gerrit.cloudera.org:8080/9134
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia7b6ae4ce3b9ee8125b21993702faa87537790a4
Gerrit-Change-Number: 9134
Gerrit-PatchSet: 12
Gerrit-Owner: Quanlong Huang 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Thu, 05 Apr 2018 06:38:23 +
Gerrit-HasComments: Yes