[Impala-ASF-CR] Revert IMPALA-4835 and dependent changes

[Impala Public Jenkins (Code Review)]

Impala Public Jenkins has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/9480 )

Change subject: Revert IMPALA-4835 and dependent changes
..

Revert IMPALA-4835 and dependent changes

Revert "IMPALA-6585: increase test_low_mem_limit_q21 limit"

This reverts commit 25bcb258dfd712f1514cf188206667a5e6be0e26.

Revert "IMPALA-6588: don't add empty list of ranges in text scan"

This reverts commit d57fbec6f67b83227b4c6117476da8f7d75fc4f6.

Revert "IMPALA-4835: Part 3: switch I/O buffers to buffer pool"

This reverts commit 24b4ed0b29a44090350e630d625291c01b753a36.

Revert "IMPALA-4835: Part 2: Allocate scan range buffers upfront"

This reverts commit 5699b59d0c5cbe37e888a367adb42fa12dfb0916.

Revert "IMPALA-4835: Part 1: simplify I/O mgr mem mgmt and cancellation"

This reverts commit 65680dc42107db4ff2273c635cedf83d20f0ea94.

Change-Id: Ie5ca451cd96602886b0a8ecaa846957df0269cbb
Reviewed-on: http://gerrit.cloudera.org:8080/9480
Reviewed-by: Dan Hecht 
Tested-by: Impala Public Jenkins
---
M be/src/exec/CMakeLists.txt
M be/src/exec/base-sequence-scanner.cc
M be/src/exec/base-sequence-scanner.h
M be/src/exec/hdfs-lzo-text-scanner.cc
D be/src/exec/hdfs-parquet-scanner-test.cc
M be/src/exec/hdfs-parquet-scanner.cc
M be/src/exec/hdfs-parquet-scanner.h
M be/src/exec/hdfs-scan-node-base.cc
M be/src/exec/hdfs-scan-node-base.h
M be/src/exec/hdfs-scan-node-mt.cc
M be/src/exec/hdfs-scan-node.cc
M be/src/exec/hdfs-scan-node.h
M be/src/exec/hdfs-text-scanner.cc
M be/src/exec/parquet-column-readers.cc
M be/src/exec/parquet-column-readers.h
M be/src/exec/scanner-context.cc
M be/src/exec/scanner-context.h
M be/src/runtime/bufferpool/buffer-pool.h
M be/src/runtime/bufferpool/reservation-tracker-test.cc
M be/src/runtime/bufferpool/reservation-util.cc
M be/src/runtime/exec-env.cc
M be/src/runtime/io/disk-io-mgr-internal.h
M be/src/runtime/io/disk-io-mgr-stress-test.cc
M be/src/runtime/io/disk-io-mgr-stress.cc
M be/src/runtime/io/disk-io-mgr-stress.h
M be/src/runtime/io/disk-io-mgr-test.cc
M be/src/runtime/io/disk-io-mgr.cc
M be/src/runtime/io/disk-io-mgr.h
M be/src/runtime/io/request-context.cc
M be/src/runtime/io/request-context.h
M be/src/runtime/io/request-ranges.h
M be/src/runtime/io/scan-range.cc
M be/src/runtime/mem-tracker.h
M be/src/runtime/test-env.cc
M be/src/runtime/tmp-file-mgr-test.cc
M be/src/runtime/tmp-file-mgr.cc
M be/src/runtime/tmp-file-mgr.h
M be/src/util/bit-util-test.cc
M be/src/util/bit-util.h
M be/src/util/impalad-metrics.cc
M be/src/util/impalad-metrics.h
M common/thrift/PlanNodes.thrift
M fe/src/main/java/org/apache/impala/analysis/SlotDescriptor.java
M fe/src/main/java/org/apache/impala/analysis/SlotRef.java
M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java
M fe/src/main/java/org/apache/impala/util/BitUtil.java
M fe/src/test/java/org/apache/impala/util/BitUtilTest.java
M 
testdata/workloads/functional-planner/queries/PlannerTest/constant-folding.test
M testdata/workloads/functional-planner/queries/PlannerTest/disable-codegen.test
M 
testdata/workloads/functional-planner/queries/PlannerTest/fk-pk-join-detection.test
M testdata/workloads/functional-planner/queries/PlannerTest/max-row-size.test
M 
testdata/workloads/functional-planner/queries/PlannerTest/min-max-runtime-filters.test
M 
testdata/workloads/functional-planner/queries/PlannerTest/mt-dop-validation.test
M 
testdata/workloads/functional-planner/queries/PlannerTest/parquet-filtering.test
M 
testdata/workloads/functional-planner/queries/PlannerTest/partition-pruning.test
M 
testdata/workloads/functional-planner/queries/PlannerTest/resource-requirements.test
M 
testdata/workloads/functional-planner/queries/PlannerTest/sort-expr-materialization.test
M 
testdata/workloads/functional-planner/queries/PlannerTest/spillable-buffer-sizing.test
M testdata/workloads/functional-planner/queries/PlannerTest/tablesample.test
M testdata/workloads/functional-planner/queries/PlannerTest/union.test
M 
testdata/workloads/functional-query/queries/QueryTest/admission-reject-min-reservation.test
M testdata/workloads/functional-query/queries/QueryTest/analytic-fns.test
M testdata/workloads/functional-query/queries/QueryTest/codegen-mem-limit.test
M 
testdata/workloads/functional-query/queries/QueryTest/disk-spill-encryption.test
M testdata/workloads/functional-query/queries/QueryTest/explain-level0.test
M testdata/workloads/functional-query/queries/QueryTest/explain-level1.test
M testdata/workloads/functional-query/queries/QueryTest/explain-level2.test
M testdata/workloads/functional-query/queries/QueryTest/explain-level3.test
M testdata/workloads/functional-query/queries/QueryTest/nested-types-tpch.test
M testdata/workloads/functional-query/queries/QueryTest/runtime_row_filters.test
M testdata/workloads/functional-query/queries/QueryTest/scanners.test
M testdata/workloads/functional-query/queries/QueryTest/set.test
M 

[Impala-ASF-CR] Revert IMPALA-4835 and dependent changes

[Impala Public Jenkins (Code Review)]

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9480 )

Change subject: Revert IMPALA-4835 and dependent changes
..


Patch Set 1: Verified+1


--
To view, visit http://gerrit.cloudera.org:8080/9480
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie5ca451cd96602886b0a8ecaa846957df0269cbb
Gerrit-Change-Number: 9480
Gerrit-PatchSet: 1
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Sat, 03 Mar 2018 04:22:11 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6551: Change Kudu TPCDS and TPCH columns to DECIMAL

[Grant Henke (Code Review)]

Grant Henke has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9484 )

Change subject: IMPALA-6551: Change Kudu TPCDS and TPCH columns to DECIMAL
..


Patch Set 1:

I am not 100% sure how to best verify the changes. It looks like the 
create-load-data.sh script (used by buildall.sh) also loads tpc-h for Kudu but 
not tpc-ds. It also looks like some kudu create statements are in the standard 
tpch_schema_template.sql file, but the intention going forward seams to be to 
used the kudu specific templates for both tpc-h and tpc-ds (according to TODO 
notes in those files).

I tested tpc-h when using buildall.sh to load data and also when using 
load-tpc-kudu.py to load data.

Here is a sample verification script I ran just derived from experimenting:
```
# Load Test Data
export JAVA_HOME=/usr/lib/jvm/default-java
source bin/impala-config.sh
./buildall.sh -ninja -skiptests -noclean -format -testdata
# Load & Run Kudu TCP-DS
./testdata/bin/load-tpc-kudu.py --clean -s tpch -t tpch_kudu -w tpch 
--kudu_master localhost
rm /tmp/None_query_runtime_info.json
rm -rf /tmp/profiles/
rm -rf /tmp/result_hashes/
./tests/stress/concurrent_select.py --minicluster-num-impalads 3 
--max-queries=1000 --tpch-kudu-db=tpch_kudu --cancel-current-queries 
--timeout-multiplier 1000
# Load & Run Kudu TCP-DS
./testdata/bin/load-tpc-kudu.py --clean -s tpcds -t tpcds_kudu -w tpcds 
--kudu_master localhost
rm /tmp/None_query_runtime_info.json
rm -rf /tmp/profiles/
rm -rf /tmp/result_hashes/
./tests/stress/concurrent_select.py --minicluster-num-impalads 3 
--max-queries=1000 --tpcds-kudu-db=tpcds_kudu --cancel-current-queries 
--timeout-multiplier 1000
```


--
To view, visit http://gerrit.cloudera.org:8080/9484
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2f7e4464dc6705cadd610a82c459390a9c0dfe4f
Gerrit-Change-Number: 9484
Gerrit-PatchSet: 1
Gerrit-Owner: Grant Henke 
Gerrit-Reviewer: Dimitris Tsirogiannis 
Gerrit-Reviewer: Grant Henke 
Gerrit-Reviewer: Thomas Tauber-Marshall 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Sat, 03 Mar 2018 02:38:34 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6551: Change Kudu TPCDS and TPCH columns to DECIMAL

[Grant Henke (Code Review)]

Grant Henke has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/9484


Change subject: IMPALA-6551: Change Kudu TPCDS and TPCH columns to DECIMAL
..

IMPALA-6551: Change Kudu TPCDS and TPCH columns to DECIMAL

Before Kudu supported DECIMAL columns the TPCDS and TPCH
columns were djusted to use DOUBLE in place of DECIMAL. This
patch undoes that change now that Kudu supports DECIMAL.

Testing:
Ran concurrent_select.py on Kudu for tpc-h and tpc-ds.

Change-Id: I2f7e4464dc6705cadd610a82c459390a9c0dfe4f
---
M testdata/datasets/tpcds/tpcds_kudu_template.sql
M testdata/datasets/tpch/tpch_kudu_template.sql
M testdata/datasets/tpch/tpch_schema_template.sql
D testdata/workloads/tpcds/queries/tpcds-kudu-q19.test
D testdata/workloads/tpcds/queries/tpcds-kudu-q27.test
D testdata/workloads/tpcds/queries/tpcds-kudu-q3.test
D testdata/workloads/tpcds/queries/tpcds-kudu-q34.test
D testdata/workloads/tpcds/queries/tpcds-kudu-q42.test
D testdata/workloads/tpcds/queries/tpcds-kudu-q43.test
D testdata/workloads/tpcds/queries/tpcds-kudu-q46.test
D testdata/workloads/tpcds/queries/tpcds-kudu-q47.test
D testdata/workloads/tpcds/queries/tpcds-kudu-q52.test
D testdata/workloads/tpcds/queries/tpcds-kudu-q53.test
D testdata/workloads/tpcds/queries/tpcds-kudu-q55.test
D testdata/workloads/tpcds/queries/tpcds-kudu-q59.test
D testdata/workloads/tpcds/queries/tpcds-kudu-q6.test
D testdata/workloads/tpcds/queries/tpcds-kudu-q61.test
D testdata/workloads/tpcds/queries/tpcds-kudu-q63.test
D testdata/workloads/tpcds/queries/tpcds-kudu-q65.test
D testdata/workloads/tpcds/queries/tpcds-kudu-q68.test
D testdata/workloads/tpcds/queries/tpcds-kudu-q7.test
D testdata/workloads/tpcds/queries/tpcds-kudu-q73.test
D testdata/workloads/tpcds/queries/tpcds-kudu-q79.test
D testdata/workloads/tpcds/queries/tpcds-kudu-q8.test
D testdata/workloads/tpcds/queries/tpcds-kudu-q88.test
D testdata/workloads/tpcds/queries/tpcds-kudu-q89.test
D testdata/workloads/tpcds/queries/tpcds-kudu-q96.test
D testdata/workloads/tpcds/queries/tpcds-kudu-q98.test
D testdata/workloads/tpch/queries/tpch-kudu-q1.test
D testdata/workloads/tpch/queries/tpch-kudu-q10.test
D testdata/workloads/tpch/queries/tpch-kudu-q11.test
D testdata/workloads/tpch/queries/tpch-kudu-q12.test
D testdata/workloads/tpch/queries/tpch-kudu-q13.test
D testdata/workloads/tpch/queries/tpch-kudu-q14.test
D testdata/workloads/tpch/queries/tpch-kudu-q15.test
D testdata/workloads/tpch/queries/tpch-kudu-q16.test
D testdata/workloads/tpch/queries/tpch-kudu-q17.test
D testdata/workloads/tpch/queries/tpch-kudu-q18.test
D testdata/workloads/tpch/queries/tpch-kudu-q19.test
D testdata/workloads/tpch/queries/tpch-kudu-q2.test
D testdata/workloads/tpch/queries/tpch-kudu-q20.test
D testdata/workloads/tpch/queries/tpch-kudu-q21.test
D testdata/workloads/tpch/queries/tpch-kudu-q22.test
D testdata/workloads/tpch/queries/tpch-kudu-q3.test
D testdata/workloads/tpch/queries/tpch-kudu-q4.test
D testdata/workloads/tpch/queries/tpch-kudu-q5.test
D testdata/workloads/tpch/queries/tpch-kudu-q6.test
D testdata/workloads/tpch/queries/tpch-kudu-q7.test
D testdata/workloads/tpch/queries/tpch-kudu-q8.test
D testdata/workloads/tpch/queries/tpch-kudu-q9.test
M tests/stress/concurrent_select.py
51 files changed, 105 insertions(+), 22,147 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/84/9484/1
--
To view, visit http://gerrit.cloudera.org:8080/9484
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I2f7e4464dc6705cadd610a82c459390a9c0dfe4f
Gerrit-Change-Number: 9484
Gerrit-PatchSet: 1
Gerrit-Owner: Grant Henke 

[Impala-ASF-CR] IMPALA-6347: Monitor queue depth size for outgoing RPCs for Reactor threads

[Michael Ho (Code Review)]

Michael Ho has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9384 )

Change subject: IMPALA-6347: Monitor queue depth size for outgoing RPCs for 
Reactor threads
..


Patch Set 11:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/9384/11/www/rpcz.tmpl
File www/rpcz.tmpl:

http://gerrit.cloudera.org:8080/#/c/9384/11/www/rpcz.tmpl@108
PS11, Line 108: RPC
Thrift RPC ?


http://gerrit.cloudera.org:8080/#/c/9384/11/www/rpcz.tmpl@230
PS11, Line 230: $.map(json["per_conn_metrics"], function(row) {
  : return [$.map(row, function(cell) { return cell; })];
Does this rely on the order of the fields in the json document ? Seems a bit 
fragile when we add / remove or reorder fields in the future in rpc-mgr.cc. 
Will using the approach of update_krpc_services() be a bit more robust ?



--
To view, visit http://gerrit.cloudera.org:8080/9384
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I84fea531e98d3e84fcc57bf7533655218bc91f4c
Gerrit-Change-Number: 9384
Gerrit-PatchSet: 11
Gerrit-Owner: Sailesh Mukil 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Comment-Date: Sat, 03 Mar 2018 02:18:54 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-3651: [DOCS] Doc for MURMUR HASH() function

[Impala Public Jenkins (Code Review)]

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9031 )

Change subject: IMPALA-3651: [DOCS] Doc for MURMUR_HASH() function
..


Patch Set 3: Verified+1


--
To view, visit http://gerrit.cloudera.org:8080/9031
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I784a5a080d7d13192aac2fca67f841d2d19fc99b
Gerrit-Change-Number: 9031
Gerrit-PatchSet: 3
Gerrit-Owner: Kim Jin Chul 
Gerrit-Reviewer: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: John Russell 
Gerrit-Reviewer: Kim Jin Chul 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Sat, 03 Mar 2018 02:12:05 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-3651: [DOCS] Doc for MURMUR HASH() function

[Impala Public Jenkins (Code Review)]

Impala Public Jenkins has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/9031 )

Change subject: IMPALA-3651: [DOCS] Doc for MURMUR_HASH() function
..

IMPALA-3651: [DOCS] Doc for MURMUR_HASH() function

Change-Id: I784a5a080d7d13192aac2fca67f841d2d19fc99b
Reviewed-on: http://gerrit.cloudera.org:8080/9031
Reviewed-by: Tim Armstrong 
Tested-by: Impala Public Jenkins
---
M docs/impala_keydefs.ditamap
M docs/topics/impala_math_functions.xml
2 files changed, 76 insertions(+), 0 deletions(-)

Approvals:
  Tim Armstrong: Looks good to me, approved
  Impala Public Jenkins: Verified

--
To view, visit http://gerrit.cloudera.org:8080/9031
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I784a5a080d7d13192aac2fca67f841d2d19fc99b
Gerrit-Change-Number: 9031
Gerrit-PatchSet: 4
Gerrit-Owner: Kim Jin Chul 
Gerrit-Reviewer: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: John Russell 
Gerrit-Reviewer: Kim Jin Chul 
Gerrit-Reviewer: Tim Armstrong 

[Impala-ASF-CR] IMPALA-3651: [DOCS] Doc for MURMUR HASH() function

[Impala Public Jenkins (Code Review)]

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9031 )

Change subject: IMPALA-3651: [DOCS] Doc for MURMUR_HASH() function
..


Patch Set 3:

Build started: https://jenkins.impala.io/job/gerrit-docs-submit/202/


--
To view, visit http://gerrit.cloudera.org:8080/9031
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I784a5a080d7d13192aac2fca67f841d2d19fc99b
Gerrit-Change-Number: 9031
Gerrit-PatchSet: 3
Gerrit-Owner: Kim Jin Chul 
Gerrit-Reviewer: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: John Russell 
Gerrit-Reviewer: Kim Jin Chul 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Sat, 03 Mar 2018 02:09:48 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-3651: [DOCS] Doc for MURMUR HASH() function

[Impala Public Jenkins (Code Review)]

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9031 )

Change subject: IMPALA-3651: [DOCS] Doc for MURMUR_HASH() function
..


Patch Set 3: Verified-1

Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/2040/


--
To view, visit http://gerrit.cloudera.org:8080/9031
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I784a5a080d7d13192aac2fca67f841d2d19fc99b
Gerrit-Change-Number: 9031
Gerrit-PatchSet: 3
Gerrit-Owner: Kim Jin Chul 
Gerrit-Reviewer: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: John Russell 
Gerrit-Reviewer: Kim Jin Chul 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Sat, 03 Mar 2018 02:09:36 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6338: xfail test profile fragment instances

[Impala Public Jenkins (Code Review)]

Impala Public Jenkins has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/9467 )

Change subject: IMPALA-6338: xfail test_profile_fragment_instances
..

IMPALA-6338: xfail test_profile_fragment_instances

test_profile_fragment_instances has been flaky. The reason is
understood and low impact - there is a very hard to hit race condition
where queries that complete successfully but are cancelled internally
by the coordinator may not have a complete runtime profile.

A patch to fix this was already submitted, but after merging some
crashes were observed in tests so it was reverted. While we figure out
what is going on there, temporarily xfail the test.

Change-Id: I06adc9d8f412fa54b07bd1ebc2c062dd3adde1f7
Reviewed-on: http://gerrit.cloudera.org:8080/9467
Reviewed-by: Sailesh Mukil 
Reviewed-by: David Knupp 
Tested-by: Impala Public Jenkins
---
M tests/query_test/test_observability.py
1 file changed, 1 insertion(+), 0 deletions(-)

Approvals:
  Sailesh Mukil: Looks good to me, but someone else must approve
  David Knupp: Looks good to me, approved
  Impala Public Jenkins: Verified

--
To view, visit http://gerrit.cloudera.org:8080/9467
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I06adc9d8f412fa54b07bd1ebc2c062dd3adde1f7
Gerrit-Change-Number: 9467
Gerrit-PatchSet: 2
Gerrit-Owner: Thomas Tauber-Marshall 
Gerrit-Reviewer: David Knupp 
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Sailesh Mukil 

[Impala-ASF-CR] IMPALA-6338: xfail test profile fragment instances

[Impala Public Jenkins (Code Review)]

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9467 )

Change subject: IMPALA-6338: xfail test_profile_fragment_instances
..


Patch Set 1: Verified+1


--
To view, visit http://gerrit.cloudera.org:8080/9467
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I06adc9d8f412fa54b07bd1ebc2c062dd3adde1f7
Gerrit-Change-Number: 9467
Gerrit-PatchSet: 1
Gerrit-Owner: Thomas Tauber-Marshall 
Gerrit-Reviewer: David Knupp 
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Comment-Date: Sat, 03 Mar 2018 01:51:02 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-5315: Cast to timestamp fails for YYYY-M-D format

[Tim Armstrong (Code Review)]

Tim Armstrong has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/7009 )

Change subject: IMPALA-5315: Cast to timestamp fails for -M-D format
..


Patch Set 14:

I've been neglecting reviews - need to get back to this one.


--
To view, visit http://gerrit.cloudera.org:8080/7009
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib9a184a09d7e7783f04d47588537612c2ecec28f
Gerrit-Change-Number: 7009
Gerrit-PatchSet: 14
Gerrit-Owner: Vincent Tran 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Reviewer: Vincent Tran 
Gerrit-Comment-Date: Sat, 03 Mar 2018 01:20:07 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-3651: [DOCS] Doc for MURMUR HASH() function

[Impala Public Jenkins (Code Review)]

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9031 )

Change subject: IMPALA-3651: [DOCS] Doc for MURMUR_HASH() function
..


Patch Set 3:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2040/


--
To view, visit http://gerrit.cloudera.org:8080/9031
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I784a5a080d7d13192aac2fca67f841d2d19fc99b
Gerrit-Change-Number: 9031
Gerrit-PatchSet: 3
Gerrit-Owner: Kim Jin Chul 
Gerrit-Reviewer: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: John Russell 
Gerrit-Reviewer: Kim Jin Chul 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Sat, 03 Mar 2018 01:17:57 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-3651: [DOCS] Doc for MURMUR HASH() function

[Tim Armstrong (Code Review)]

Tim Armstrong has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9031 )

Change subject: IMPALA-3651: [DOCS] Doc for MURMUR_HASH() function
..


Patch Set 2: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/9031
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I784a5a080d7d13192aac2fca67f841d2d19fc99b
Gerrit-Change-Number: 9031
Gerrit-PatchSet: 2
Gerrit-Owner: Kim Jin Chul 
Gerrit-Reviewer: Alex Rodoni 
Gerrit-Reviewer: John Russell 
Gerrit-Reviewer: Kim Jin Chul 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Sat, 03 Mar 2018 01:17:41 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-3651: [DOCS] Doc for MURMUR HASH() function

[Tim Armstrong (Code Review)]

Tim Armstrong has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9031 )

Change subject: IMPALA-3651: [DOCS] Doc for MURMUR_HASH() function
..


Patch Set 3: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/9031
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I784a5a080d7d13192aac2fca67f841d2d19fc99b
Gerrit-Change-Number: 9031
Gerrit-PatchSet: 3
Gerrit-Owner: Kim Jin Chul 
Gerrit-Reviewer: Alex Rodoni 
Gerrit-Reviewer: John Russell 
Gerrit-Reviewer: Kim Jin Chul 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Sat, 03 Mar 2018 01:17:47 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-5237: Support a quoted string in date/time format

[Tim Armstrong (Code Review)]

Tim Armstrong has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/8508 )

Change subject: IMPALA-5237: Support a quoted string in date/time format
..


Patch Set 10:

(1 comment)

Sorry for the slow turnaround time here - things got crazy for a while.

http://gerrit.cloudera.org:8080/#/c/8508/9/be/src/runtime/timestamp-parse-util.cc
File be/src/runtime/timestamp-parse-util.cc:

http://gerrit.cloudera.org:8080/#/c/8508/9/be/src/runtime/timestamp-parse-util.cc@175
PS9, Line 175:   // e.g. '''foo''bar''' => 'foo'bar'
> Let me provide a more intuitive algorithm to find escaped single quotes bet
I think we still need a comment like my above one, since we are still treating 
the last single quote found as the start of a new string.

To be honest I think the new approach less intuitive, because we still split 
the literal token up into multiple tokens if there are escaped quotes in the 
middle of the string. It seems like the approach only changes when dealing with 
quotes at the end of the string.

E.g. my understanding is that "'foo''bar'" is still broken into two tokens, 
"foo'" and "bar", whereas "foobar'''" is handled as a single token "foo'".

I think fundamentally we need to do this token-splitting because of the memory 
management - we can't convert "'foo''bar'" into a single token 'foo'bar' 
without creating a new string and copying the data. So my preference is to use 
the previous approach and explain it clearly.



--
To view, visit http://gerrit.cloudera.org:8080/8508
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie34055ac695748bcfb110bfa6ed5308f469ea178
Gerrit-Change-Number: 8508
Gerrit-PatchSet: 10
Gerrit-Owner: Kim Jin Chul 
Gerrit-Reviewer: Attila Jeges 
Gerrit-Reviewer: Gabor Kaszab 
Gerrit-Reviewer: Kim Jin Chul 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Sat, 03 Mar 2018 01:09:53 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6230, IMPALA-6468: Fix the output type of round() and related fns

[anujphadke (Code Review)]

anujphadke has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9346 )

Change subject: IMPALA-6230, IMPALA-6468: Fix the output type of round() and 
related fns
..


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/9346/1/be/src/exprs/math-functions-ir.cc
File be/src/exprs/math-functions-ir.cc:

http://gerrit.cloudera.org:8080/#/c/9346/1/be/src/exprs/math-functions-ir.cc@121
PS1, Line 121:   v.val * pow(10.0, scale.val) + ((v.val < 0) ? -0.5 : 0.5)) 
/ pow(10.0, scale.val));
Should we also check for overflows here , set FunctionContext with an error and 
return a NULL?
ctx->SetError

Found a couple of possible inconsistencies (without your change). If your 
change fixes it we can add these tests?

[nightly-unsecure-2.gce.cloudera.com:21000] > select dround(cast(2.51 as 
double), 256);
Query: select dround(cast(2.51 as double), 256)
Query submitted at: 2018-03-02 16:44:05 (Coordinator: 
http://nightly-unsecure-2.gce.cloudera.com:25000)
Query progress can be monitored at: 
http://nightly-unsecure-2.gce.cloudera.com:25000/query_plan?query_id=c5421271ce74b797:4da87021
+---+
| dround(cast(2.51 as double), 256) |
+---+
| 2.51  |
+---+
Fetched 1 row(s) in 0.01s
[nightly-unsecure-2.gce.cloudera.com:21000] > select dround(cast(2.51 as 
double), 257);
Query: select dround(cast(2.51 as double), 257)
Query submitted at: 2018-03-02 16:44:09 (Coordinator: 
http://nightly-unsecure-2.gce.cloudera.com:25000)
Query progress can be monitored at: 
http://nightly-unsecure-2.gce.cloudera.com:25000/query_plan?query_id=d4969d969dba34a:639969f
+---+
| dround(cast(2.51 as double), 257) |
+---+
| 2.511 |
+---+
Fetched 1 row(s) in 0.01s

[nightly-unsecure-2.gce.cloudera.com:21000] > select dround(cast(2.51 as 
double), 400);
Query: select dround(cast(2.51 as double), 400)
Query submitted at: 2018-03-02 16:50:01 (Coordinator: 
http://nightly-unsecure-2.gce.cloudera.com:25000)
Query progress can be monitored at: 
http://nightly-unsecure-2.gce.cloudera.com:25000/query_plan?query_id=c741e669098c64a0:d7514ce1
+---+
| dround(cast(2.51 as double), 400) |
+---+
| NaN   |
+---+
Fetched 1 row(s) in 0.01s



--
To view, visit http://gerrit.cloudera.org:8080/9346
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I77541678012edab70b182378b11ca8753be53f97
Gerrit-Change-Number: 9346
Gerrit-PatchSet: 1
Gerrit-Owner: Taras Bobrovytsky 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Reviewer: Zach Amsden 
Gerrit-Reviewer: anujphadke 
Gerrit-Comment-Date: Sat, 03 Mar 2018 00:58:46 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6230, IMPALA-6468: Fix the output type of round() and related fns

[anujphadke (Code Review)]

anujphadke has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9346 )

Change subject: IMPALA-6230, IMPALA-6468: Fix the output type of round() and 
related fns
..


Patch Set 1:

impalad crashes with this patch at startup.

I0302 16:29:52.235250 19443 status.cc:125] Unable to find 
_ZN6impala13MathFunctions9RoundUpToEPN10impala_udf15FunctionContextERKNS1_9DoubleValERKNS1_9BigIntValE
dlerror: /home/anuj/Impala/be/build/latest/service/impalad: undefined symbol: 
_ZN6impala13MathFunctions9RoundUpToEPN10impala_udf15FunctionContextERKNS1_9DoubleValERKNS1_9BigIntValE
@  0x166cd33  impala::Status::Status()
@  0x1a664ec  impala::DynamicLookup()
@  0x182e579  impala::LibCache::GetSoFunctionPtr()
@  0x182ef5a  impala::LibCache::CheckSymbolExists()
@  0x193a533  ResolveSymbolLookup()
@  0x193b00b  
Java_org_apache_impala_service_FeSupport_NativeLookupSymbol
@ 0x7fb3508edc39  (unknown)
F0302 16:29:52.236079 19443 frontend.cc:108] 
java.lang.ExceptionInInitializerError
  at org.apache.impala.service.BackendConfig.create(BackendConfig.java:47)
  at org.apache.impala.service.JniFrontend.(JniFrontend.java:123)
Caused by: java.lang.RuntimeException: Builtin symbol 
'impala::MathFunctions::RoundUpTo'[DOUBLE, BIGINT] not found!
  at 
org.apache.impala.catalog.ScalarFunction.createBuiltin(ScalarFunction.java:82)
  at org.apache.impala.catalog.Db.addScalarBuiltin(Db.java:393)


--
To view, visit http://gerrit.cloudera.org:8080/9346
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I77541678012edab70b182378b11ca8753be53f97
Gerrit-Change-Number: 9346
Gerrit-PatchSet: 1
Gerrit-Owner: Taras Bobrovytsky 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Reviewer: Zach Amsden 
Gerrit-Reviewer: anujphadke 
Gerrit-Comment-Date: Sat, 03 Mar 2018 00:35:37 +
Gerrit-HasComments: No

[Impala-ASF-CR] Revert IMPALA-4835 and dependent changes

[Dan Hecht (Code Review)]

Dan Hecht has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9480 )

Change subject: Revert IMPALA-4835 and dependent changes
..


Patch Set 1: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/9480
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie5ca451cd96602886b0a8ecaa846957df0269cbb
Gerrit-Change-Number: 9480
Gerrit-PatchSet: 1
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Sat, 03 Mar 2018 00:34:44 +
Gerrit-HasComments: No

[Impala-ASF-CR] Revert IMPALA-4835 and dependent changes

[Impala Public Jenkins (Code Review)]

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9480 )

Change subject: Revert IMPALA-4835 and dependent changes
..


Patch Set 1:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2039/


--
To view, visit http://gerrit.cloudera.org:8080/9480
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie5ca451cd96602886b0a8ecaa846957df0269cbb
Gerrit-Change-Number: 9480
Gerrit-PatchSet: 1
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Sat, 03 Mar 2018 00:27:47 +
Gerrit-HasComments: No

[Impala-ASF-CR] Revert IMPALA-4835 and dependent changes

[Tim Armstrong (Code Review)]

Tim Armstrong has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9480 )

Change subject: Revert IMPALA-4835 and dependent changes
..


Patch Set 1:

The reverts were mostly clean. I had to resolve some minor conflicts with 
IMPALA-6530 where the ReaderContext set_*counter interface was changed.


--
To view, visit http://gerrit.cloudera.org:8080/9480
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie5ca451cd96602886b0a8ecaa846957df0269cbb
Gerrit-Change-Number: 9480
Gerrit-PatchSet: 1
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Sat, 03 Mar 2018 00:27:25 +
Gerrit-HasComments: No

[Impala-ASF-CR] Revert IMPALA-4835 and dependent changes

[Tim Armstrong (Code Review)]

Tim Armstrong has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/9480


Change subject: Revert IMPALA-4835 and dependent changes
..

Revert IMPALA-4835 and dependent changes

Revert "IMPALA-6585: increase test_low_mem_limit_q21 limit"

This reverts commit 25bcb258dfd712f1514cf188206667a5e6be0e26.

Revert "IMPALA-6588: don't add empty list of ranges in text scan"

This reverts commit d57fbec6f67b83227b4c6117476da8f7d75fc4f6.

Revert "IMPALA-4835: Part 3: switch I/O buffers to buffer pool"

This reverts commit 24b4ed0b29a44090350e630d625291c01b753a36.

Revert "IMPALA-4835: Part 2: Allocate scan range buffers upfront"

This reverts commit 5699b59d0c5cbe37e888a367adb42fa12dfb0916.

Revert "IMPALA-4835: Part 1: simplify I/O mgr mem mgmt and cancellation"

This reverts commit 65680dc42107db4ff2273c635cedf83d20f0ea94.

Change-Id: Ie5ca451cd96602886b0a8ecaa846957df0269cbb
---
M be/src/exec/CMakeLists.txt
M be/src/exec/base-sequence-scanner.cc
M be/src/exec/base-sequence-scanner.h
M be/src/exec/hdfs-lzo-text-scanner.cc
D be/src/exec/hdfs-parquet-scanner-test.cc
M be/src/exec/hdfs-parquet-scanner.cc
M be/src/exec/hdfs-parquet-scanner.h
M be/src/exec/hdfs-scan-node-base.cc
M be/src/exec/hdfs-scan-node-base.h
M be/src/exec/hdfs-scan-node-mt.cc
M be/src/exec/hdfs-scan-node.cc
M be/src/exec/hdfs-scan-node.h
M be/src/exec/hdfs-text-scanner.cc
M be/src/exec/parquet-column-readers.cc
M be/src/exec/parquet-column-readers.h
M be/src/exec/scanner-context.cc
M be/src/exec/scanner-context.h
M be/src/runtime/bufferpool/buffer-pool.h
M be/src/runtime/bufferpool/reservation-tracker-test.cc
M be/src/runtime/bufferpool/reservation-util.cc
M be/src/runtime/exec-env.cc
M be/src/runtime/io/disk-io-mgr-internal.h
M be/src/runtime/io/disk-io-mgr-stress-test.cc
M be/src/runtime/io/disk-io-mgr-stress.cc
M be/src/runtime/io/disk-io-mgr-stress.h
M be/src/runtime/io/disk-io-mgr-test.cc
M be/src/runtime/io/disk-io-mgr.cc
M be/src/runtime/io/disk-io-mgr.h
M be/src/runtime/io/request-context.cc
M be/src/runtime/io/request-context.h
M be/src/runtime/io/request-ranges.h
M be/src/runtime/io/scan-range.cc
M be/src/runtime/mem-tracker.h
M be/src/runtime/test-env.cc
M be/src/runtime/tmp-file-mgr-test.cc
M be/src/runtime/tmp-file-mgr.cc
M be/src/runtime/tmp-file-mgr.h
M be/src/util/bit-util-test.cc
M be/src/util/bit-util.h
M be/src/util/impalad-metrics.cc
M be/src/util/impalad-metrics.h
M common/thrift/PlanNodes.thrift
M fe/src/main/java/org/apache/impala/analysis/SlotDescriptor.java
M fe/src/main/java/org/apache/impala/analysis/SlotRef.java
M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java
M fe/src/main/java/org/apache/impala/util/BitUtil.java
M fe/src/test/java/org/apache/impala/util/BitUtilTest.java
M 
testdata/workloads/functional-planner/queries/PlannerTest/constant-folding.test
M testdata/workloads/functional-planner/queries/PlannerTest/disable-codegen.test
M 
testdata/workloads/functional-planner/queries/PlannerTest/fk-pk-join-detection.test
M testdata/workloads/functional-planner/queries/PlannerTest/max-row-size.test
M 
testdata/workloads/functional-planner/queries/PlannerTest/min-max-runtime-filters.test
M 
testdata/workloads/functional-planner/queries/PlannerTest/mt-dop-validation.test
M 
testdata/workloads/functional-planner/queries/PlannerTest/parquet-filtering.test
M 
testdata/workloads/functional-planner/queries/PlannerTest/partition-pruning.test
M 
testdata/workloads/functional-planner/queries/PlannerTest/resource-requirements.test
M 
testdata/workloads/functional-planner/queries/PlannerTest/sort-expr-materialization.test
M 
testdata/workloads/functional-planner/queries/PlannerTest/spillable-buffer-sizing.test
M testdata/workloads/functional-planner/queries/PlannerTest/tablesample.test
M testdata/workloads/functional-planner/queries/PlannerTest/union.test
M 
testdata/workloads/functional-query/queries/QueryTest/admission-reject-min-reservation.test
M testdata/workloads/functional-query/queries/QueryTest/analytic-fns.test
M testdata/workloads/functional-query/queries/QueryTest/codegen-mem-limit.test
M 
testdata/workloads/functional-query/queries/QueryTest/disk-spill-encryption.test
M testdata/workloads/functional-query/queries/QueryTest/explain-level0.test
M testdata/workloads/functional-query/queries/QueryTest/explain-level1.test
M testdata/workloads/functional-query/queries/QueryTest/explain-level2.test
M testdata/workloads/functional-query/queries/QueryTest/explain-level3.test
M testdata/workloads/functional-query/queries/QueryTest/nested-types-tpch.test
M testdata/workloads/functional-query/queries/QueryTest/runtime_row_filters.test
M testdata/workloads/functional-query/queries/QueryTest/scanners.test
M testdata/workloads/functional-query/queries/QueryTest/set.test
M testdata/workloads/functional-query/queries/QueryTest/spilling-aggs.test
M 
testdata/workloads/functional-query/queries/QueryTest/spilling-naaj-no-deny-reservation.test

[Impala-ASF-CR] IMPALA-2567: Enable KRPC by default

[Michael Ho (Code Review)]

Michael Ho has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9461 )

Change subject: IMPALA-2567: Enable KRPC by default
..


Patch Set 4:

Hold off from merging until the builds are in better shape.


--
To view, visit http://gerrit.cloudera.org:8080/9461
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iae736c1c1351758969b4d84e34fc5b2d048660a0
Gerrit-Change-Number: 9461
Gerrit-PatchSet: 4
Gerrit-Owner: Michael Ho 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Comment-Date: Sat, 03 Mar 2018 00:04:35 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6547: [DOCS] Correct default encoding for INT

[Impala Public Jenkins (Code Review)]

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9402 )

Change subject: IMPALA-6547: [DOCS] Correct default encoding for INT
..


Patch Set 1: Verified+1


--
To view, visit http://gerrit.cloudera.org:8080/9402
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia71c3411f335c414193bf7cf346ac2e006b8f447
Gerrit-Change-Number: 9402
Gerrit-PatchSet: 1
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: John Russell 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Comment-Date: Fri, 02 Mar 2018 23:40:03 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6547: [DOCS] Correct default encoding for INT

[Impala Public Jenkins (Code Review)]

Impala Public Jenkins has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/9402 )

Change subject: IMPALA-6547: [DOCS] Correct default encoding for INT
..

IMPALA-6547: [DOCS] Correct default encoding for INT

Updated that default encoding for numeric types are bitshuffle.

Change-Id: Ia71c3411f335c414193bf7cf346ac2e006b8f447
Reviewed-on: http://gerrit.cloudera.org:8080/9402
Reviewed-by: Todd Lipcon 
Reviewed-by: Tim Armstrong 
Tested-by: Impala Public Jenkins
---
M docs/topics/impala_kudu.xml
1 file changed, 4 insertions(+), 5 deletions(-)

Approvals:
  Todd Lipcon: Looks good to me, but someone else must approve
  Tim Armstrong: Looks good to me, approved
  Impala Public Jenkins: Verified

--
To view, visit http://gerrit.cloudera.org:8080/9402
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ia71c3411f335c414193bf7cf346ac2e006b8f447
Gerrit-Change-Number: 9402
Gerrit-PatchSet: 2
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: John Russell 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Reviewer: Todd Lipcon 

[Impala-ASF-CR] IMPALA-6394: Restart HDFS when blocks are under replicated

[Tianyi Wang (Code Review)]

Tianyi Wang has uploaded a new patch set (#2). ( 
http://gerrit.cloudera.org:8080/9469 )

Change subject: IMPALA-6394: Restart HDFS when blocks are under replicated
..

IMPALA-6394: Restart HDFS when blocks are under replicated

HDFS sometimes fails to fully replicate all the blocks in 30 seconds
and no progress is made. This patch tries to restart HDFS several times
before aborting the data loading.

Change-Id: Iefd4c2fc6c287f054e385de52bdc42b0bdbd7915
---
M testdata/bin/create-load-data.sh
1 file changed, 11 insertions(+), 6 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/69/9469/2
--
To view, visit http://gerrit.cloudera.org:8080/9469
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iefd4c2fc6c287f054e385de52bdc42b0bdbd7915
Gerrit-Change-Number: 9469
Gerrit-PatchSet: 2
Gerrit-Owner: Tianyi Wang 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Tianyi Wang 

[Impala-ASF-CR] IMPALA-6547: [DOCS] Correct default encoding for INT

[Impala Public Jenkins (Code Review)]

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9402 )

Change subject: IMPALA-6547: [DOCS] Correct default encoding for INT
..


Patch Set 1:

Build started: https://jenkins.impala.io/job/gerrit-docs-submit/201/


--
To view, visit http://gerrit.cloudera.org:8080/9402
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia71c3411f335c414193bf7cf346ac2e006b8f447
Gerrit-Change-Number: 9402
Gerrit-PatchSet: 1
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: John Russell 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Comment-Date: Fri, 02 Mar 2018 23:26:22 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6547: [DOCS] Correct default encoding for INT

[Tim Armstrong (Code Review)]

Tim Armstrong has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9402 )

Change subject: IMPALA-6547: [DOCS] Correct default encoding for INT
..


Patch Set 1: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/9402
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia71c3411f335c414193bf7cf346ac2e006b8f447
Gerrit-Change-Number: 9402
Gerrit-PatchSet: 1
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: John Russell 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Comment-Date: Fri, 02 Mar 2018 23:26:10 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-5690: Part 2: Upgrade thrift to 0.9.3-p3

[Tianyi Wang (Code Review)]

Tianyi Wang has uploaded a new patch set (#7). ( 
http://gerrit.cloudera.org:8080/9300 )

Change subject: IMPALA-5690: Part 2: Upgrade thrift to 0.9.3-p3
..

IMPALA-5690: Part 2: Upgrade thrift to 0.9.3-p3

Dependency changes:
- BE and python use thrift 0.9.3-p3 from native-toolchain.
- FE uses thrift 0.9.3 from apache maven repo.
- Fb303 and http components dependencies are no longer needed in FE and
  are removed.
- The minimum openssl version requirement is increased to 1.0.1.

Configuration change:
- Thrift codegen option movable_type is enabled. New code no longer
  needs to use std::swap to avoid copying.

Change-Id: I639227721502eaa10398d9490ff6ac63aa71b3a6
---
M CMakeLists.txt
M be/src/common/init.cc
M be/src/rpc/TAcceptQueueServer.cpp
M be/src/rpc/TAcceptQueueServer.h
M be/src/rpc/thrift-server-test.cc
M be/src/rpc/thrift-server.cc
M be/src/rpc/thrift-server.h
M be/src/rpc/thrift-thread.h
M be/src/rpc/thrift-util.cc
M bin/impala-config.sh
M buildall.sh
M common/thrift/CMakeLists.txt
M fe/pom.xml
M infra/python/deps/compiled-requirements.txt
14 files changed, 82 insertions(+), 174 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/00/9300/7
--
To view, visit http://gerrit.cloudera.org:8080/9300
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I639227721502eaa10398d9490ff6ac63aa71b3a6
Gerrit-Change-Number: 9300
Gerrit-PatchSet: 7
Gerrit-Owner: Tianyi Wang 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Reviewer: Thomas Tauber-Marshall 
Gerrit-Reviewer: Tianyi Wang 
Gerrit-Reviewer: anujphadke 

[Impala-ASF-CR] IMPALA-6547: [DOCS] Correct default encoding for INT

[Todd Lipcon (Code Review)]

Todd Lipcon has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9402 )

Change subject: IMPALA-6547: [DOCS] Correct default encoding for INT
..


Patch Set 1: Code-Review+1


--
To view, visit http://gerrit.cloudera.org:8080/9402
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia71c3411f335c414193bf7cf346ac2e006b8f447
Gerrit-Change-Number: 9402
Gerrit-PatchSet: 1
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: John Russell 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Comment-Date: Fri, 02 Mar 2018 23:21:43 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-2567: Enable KRPC by default

[Impala Public Jenkins (Code Review)]

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9461 )

Change subject: IMPALA-2567: Enable KRPC by default
..


Patch Set 4:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2038/


--
To view, visit http://gerrit.cloudera.org:8080/9461
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iae736c1c1351758969b4d84e34fc5b2d048660a0
Gerrit-Change-Number: 9461
Gerrit-PatchSet: 4
Gerrit-Owner: Michael Ho 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Comment-Date: Fri, 02 Mar 2018 23:18:24 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6394: Restart HDFS when blocks are under replicated

[Lars Volker (Code Review)]

Lars Volker has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9469 )

Change subject: IMPALA-6394: Restart HDFS when blocks are under replicated
..


Patch Set 1:

(3 comments)

http://gerrit.cloudera.org:8080/#/c/9469/1/testdata/bin/create-load-data.sh
File testdata/bin/create-load-data.sh:

http://gerrit.cloudera.org:8080/#/c/9469/1/testdata/bin/create-load-data.sh@454
PS1, Line 454: true
I think it would be easier to read if we kept the loop condition here, e.g. 
while [[ $FAIL_COUNT -lt $MAX_RETRIES ]]; then ... and then after the loop 
check if [[ $FAIL_COUNT -eq $MAX_RETRIES ]]; then exit 1;


http://gerrit.cloudera.org:8080/#/c/9469/1/testdata/bin/create-load-data.sh@459
PS1, Line 459: r
Is this the return on success?


http://gerrit.cloudera.org:8080/#/c/9469/1/testdata/bin/create-load-data.sh@462
PS1, Line 462: 6
Can you move this into a variable MAX_RETRIES and re-use that inside the error 
message?



--
To view, visit http://gerrit.cloudera.org:8080/9469
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iefd4c2fc6c287f054e385de52bdc42b0bdbd7915
Gerrit-Change-Number: 9469
Gerrit-PatchSet: 1
Gerrit-Owner: Tianyi Wang 
Gerrit-Reviewer: Lars Volker 
Gerrit-Comment-Date: Fri, 02 Mar 2018 23:04:20 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-5690: Part 2: Upgrade thrift to 0.9.3-p3

[anujphadke (Code Review)]

anujphadke has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9300 )

Change subject: IMPALA-5690: Part 2: Upgrade thrift to 0.9.3-p3
..


Patch Set 6:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/9300/6/be/src/common/init.cc
File be/src/common/init.cc:

http://gerrit.cloudera.org:8080/#/c/9300/6/be/src/common/init.cc@240
PS6, Line 240:   signal(SIGPIPE, SIG_IGN);
Could ignoring SIGPIPE have any side effects?

E.g -
IMPALA-5187 depends on Breakpad #728:
When writing a minidump on Linux, Breakpad called clone() in
linux/handler/exception_handler.cc with the CLONE_FILES flag. If the
parent process died while the child waited for the continuation signal,
the write side of the pipe 'fdes' stayed open in the child. The child
would not receive a SIGPIPE and would wait forever.



--
To view, visit http://gerrit.cloudera.org:8080/9300
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I639227721502eaa10398d9490ff6ac63aa71b3a6
Gerrit-Change-Number: 9300
Gerrit-PatchSet: 6
Gerrit-Owner: Tianyi Wang 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Reviewer: Thomas Tauber-Marshall 
Gerrit-Reviewer: Tianyi Wang 
Gerrit-Reviewer: anujphadke 
Gerrit-Comment-Date: Fri, 02 Mar 2018 22:29:13 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6394: Restart HDFS when blocks are under replicated

[Tianyi Wang (Code Review)]

Tianyi Wang has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/9469


Change subject: IMPALA-6394: Restart HDFS when blocks are under replicated
..

IMPALA-6394: Restart HDFS when blocks are under replicated

HDFS sometimes fails to fully replicate all the blocks in 30 seconds
and no progress is made. This patch tries to restart HDFS several times
before aborting the data loading.

Change-Id: Iefd4c2fc6c287f054e385de52bdc42b0bdbd7915
---
M testdata/bin/create-load-data.sh
1 file changed, 9 insertions(+), 6 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/69/9469/1
--
To view, visit http://gerrit.cloudera.org:8080/9469
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Iefd4c2fc6c287f054e385de52bdc42b0bdbd7915
Gerrit-Change-Number: 9469
Gerrit-PatchSet: 1
Gerrit-Owner: Tianyi Wang 

[Impala-ASF-CR] IMPALA-2567: Enable KRPC by default

[Michael Ho (Code Review)]

Michael Ho has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9461 )

Change subject: IMPALA-2567: Enable KRPC by default
..


Patch Set 4:

(3 comments)

http://gerrit.cloudera.org:8080/#/c/9461/3/bin/start-impala-cluster.py
File bin/start-impala-cluster.py:

http://gerrit.cloudera.org:8080/#/c/9461/3/bin/start-impala-cluster.py@57
PS3, Line 57: , help
> thrift is always enabled. How about "Enable Thrift DataStream service ..."
Done


http://gerrit.cloudera.org:8080/#/c/9461/3/tests/common/test_skip.py
File tests/common/test_skip.py:

http://gerrit.cloudera.org:8080/#/c/9461/3/tests/common/test_skip.py@35
PS3, Line 35: assert not pytest.config.option.test_no_krpc
:
:   @SkipIf.not_thrift
:   def test_skip_if_not_thrift(self):
: assert pytest.config.option.test_no_krpc
> these look backwards. don't you have to move the 'not'?
Oops. Fixed.


http://gerrit.cloudera.org:8080/#/c/9461/3/tests/conftest.py
File tests/conftest.py:

http://gerrit.cloudera.org:8080/#/c/9461/3/tests/conftest.py@120
PS3, Line 120: all tests with
> Thrift DataStream service enabled.
Done



--
To view, visit http://gerrit.cloudera.org:8080/9461
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iae736c1c1351758969b4d84e34fc5b2d048660a0
Gerrit-Change-Number: 9461
Gerrit-PatchSet: 4
Gerrit-Owner: Michael Ho 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Comment-Date: Fri, 02 Mar 2018 22:11:19 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-2567: Enable KRPC by default

[Michael Ho (Code Review)]

Hello Sailesh Mukil, Dan Hecht,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/9461

to look at the new patch set (#4).

Change subject: IMPALA-2567: Enable KRPC by default
..

IMPALA-2567: Enable KRPC by default

This change enables the switch to use KRPC by default.
This change also fixes a bug in KrpcDataStreamMgr to
check if maintenance thread was started before calling
Join() on it. This shows up in BE tests as the maintenance
thread isn't started in them.

Also bumped the memory limit for a query in test_row_filters().
It seems to fail consistently in exhastive build after recent
scanner memory changes.

Testing done: exhaustive build.

Change-Id: Iae736c1c1351758969b4d84e34fc5b2d048660a0
---
M be/src/runtime/exec-env.cc
M be/src/runtime/krpc-data-stream-mgr.cc
M bin/run-all-tests.sh
M bin/start-impala-cluster.py
M testdata/workloads/functional-query/queries/QueryTest/runtime_row_filters.test
M tests/common/custom_cluster_test_suite.py
M tests/common/skip.py
M tests/common/test_skip.py
M tests/conftest.py
9 files changed, 26 insertions(+), 27 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/61/9461/4
--
To view, visit http://gerrit.cloudera.org:8080/9461
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iae736c1c1351758969b4d84e34fc5b2d048660a0
Gerrit-Change-Number: 9461
Gerrit-PatchSet: 4
Gerrit-Owner: Michael Ho 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Sailesh Mukil 

[Impala-ASF-CR] IMPALA-6338: xfail test profile fragment instances

[Impala Public Jenkins (Code Review)]

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9467 )

Change subject: IMPALA-6338: xfail test_profile_fragment_instances
..


Patch Set 1:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2037/


--
To view, visit http://gerrit.cloudera.org:8080/9467
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I06adc9d8f412fa54b07bd1ebc2c062dd3adde1f7
Gerrit-Change-Number: 9467
Gerrit-PatchSet: 1
Gerrit-Owner: Thomas Tauber-Marshall 
Gerrit-Reviewer: David Knupp 
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Comment-Date: Fri, 02 Mar 2018 21:58:52 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6338: xfail test profile fragment instances

[David Knupp (Code Review)]

David Knupp has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9467 )

Change subject: IMPALA-6338: xfail test_profile_fragment_instances
..


Patch Set 1: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/9467
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I06adc9d8f412fa54b07bd1ebc2c062dd3adde1f7
Gerrit-Change-Number: 9467
Gerrit-PatchSet: 1
Gerrit-Owner: Thomas Tauber-Marshall 
Gerrit-Reviewer: David Knupp 
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Comment-Date: Fri, 02 Mar 2018 21:58:21 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6338: xfail test profile fragment instances

[Sailesh Mukil (Code Review)]

Sailesh Mukil has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9467 )

Change subject: IMPALA-6338: xfail test_profile_fragment_instances
..


Patch Set 1: Code-Review+1


--
To view, visit http://gerrit.cloudera.org:8080/9467
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I06adc9d8f412fa54b07bd1ebc2c062dd3adde1f7
Gerrit-Change-Number: 9467
Gerrit-PatchSet: 1
Gerrit-Owner: Thomas Tauber-Marshall 
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Comment-Date: Fri, 02 Mar 2018 21:47:21 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6338: xfail test profile fragment instances

[Thomas Tauber-Marshall (Code Review)]

Thomas Tauber-Marshall has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/9467


Change subject: IMPALA-6338: xfail test_profile_fragment_instances
..

IMPALA-6338: xfail test_profile_fragment_instances

test_profile_fragment_instances has been flaky. The reason is
understood and low impact - there is a very hard to hit race condition
where queries that complete successfully but are cancelled internally
by the coordinator may not have a complete runtime profile.

A patch to fix this was already submitted, but after merging some
crashes were observed in tests so it was reverted. While we figure out
what is going on there, temporarily xfail the test.

Change-Id: I06adc9d8f412fa54b07bd1ebc2c062dd3adde1f7
---
M tests/query_test/test_observability.py
1 file changed, 1 insertion(+), 0 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/67/9467/1
--
To view, visit http://gerrit.cloudera.org:8080/9467
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I06adc9d8f412fa54b07bd1ebc2c062dd3adde1f7
Gerrit-Change-Number: 9467
Gerrit-PatchSet: 1
Gerrit-Owner: Thomas Tauber-Marshall 

[Impala-ASF-CR] IMPALA-5690: Part 1: Rename ostream operators for thrift types

[Tianyi Wang (Code Review)]

Tianyi Wang has uploaded a new patch set (#11). ( 
http://gerrit.cloudera.org:8080/9168 )

Change subject: IMPALA-5690: Part 1: Rename ostream operators for thrift types
..

IMPALA-5690: Part 1: Rename ostream operators for thrift types

Thrift 0.9.3 implements "ostream& operator<<(ostream&, T)" for thrift
data types while impala did the same to enums and special types
including TNetworkAddress and TUniqueId. To prepare for the upgrade of
thrift 0.9.3, this patch renames these impala defined functions. In the
absence of operator<<, assertion macros like DCHECK_EQ can no longer be
used on non-enum thrift defined types.

Change-Id: I9c303997411237e988ef960157f781776f6fcb60
---
M be/src/benchmarks/scheduler-benchmark.cc
M be/src/codegen/llvm-codegen.cc
M be/src/exec/exchange-node.cc
M be/src/exec/exec-node.cc
M be/src/exec/hdfs-scan-node-base.cc
M be/src/exec/hdfs-scan-node.cc
M be/src/exec/parquet-column-readers.cc
M be/src/exec/parquet-metadata-utils.cc
M be/src/rpc/rpc-trace.cc
M be/src/rpc/thrift-client.cc
M be/src/rpc/thrift-util.cc
M be/src/rpc/thrift-util.h
M be/src/runtime/client-cache.cc
M be/src/runtime/coordinator-backend-state.cc
M be/src/runtime/coordinator.cc
M be/src/runtime/data-stream-mgr.cc
M be/src/runtime/data-stream-recvr.cc
M be/src/runtime/data-stream-sender.cc
M be/src/runtime/fragment-instance-state.cc
M be/src/runtime/krpc-data-stream-mgr.cc
M be/src/runtime/krpc-data-stream-recvr.cc
M be/src/runtime/krpc-data-stream-sender.cc
M be/src/runtime/mem-tracker.cc
M be/src/runtime/query-exec-mgr.cc
M be/src/runtime/query-state.cc
M be/src/runtime/runtime-filter-bank.cc
M be/src/runtime/runtime-state.cc
M be/src/scheduling/admission-controller.cc
M be/src/scheduling/scheduler.cc
M be/src/service/child-query.cc
M be/src/service/client-request-state.cc
M be/src/service/impala-beeswax-server.cc
M be/src/service/impala-hs2-server.cc
M be/src/service/impala-http-handler.cc
M be/src/service/impala-internal-service.cc
M be/src/service/impala-server.cc
M be/src/service/query-options-test.cc
M be/src/service/query-options.cc
M be/src/service/query-result-set.cc
M be/src/statestore/statestore.cc
M be/src/util/collection-metrics.h
M be/src/util/debug-util.cc
M be/src/util/debug-util.h
M be/src/util/histogram-metric.h
M be/src/util/metrics.h
M be/src/util/network-util.cc
M be/src/util/network-util.h
M be/src/util/webserver.cc
M testdata/workloads/functional-query/queries/QueryTest/set.test
M tests/shell/test_shell_commandline.py
50 files changed, 260 insertions(+), 275 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/68/9168/11
--
To view, visit http://gerrit.cloudera.org:8080/9168
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I9c303997411237e988ef960157f781776f6fcb60
Gerrit-Change-Number: 9168
Gerrit-PatchSet: 11
Gerrit-Owner: Tianyi Wang 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Reviewer: Thomas Tauber-Marshall 
Gerrit-Reviewer: Tianyi Wang 

[Impala-ASF-CR] IMPALA-4430: Update build scripts to die hard when IMPALA HOME has spaces

[David Knupp (Code Review)]

David Knupp has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9385 )

Change subject: IMPALA-4430: Update build scripts to die hard when IMPALA_HOME 
has spaces
..


Patch Set 5:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/9385/5/buildall.sh
File buildall.sh:

http://gerrit.cloudera.org:8080/#/c/9385/5/buildall.sh@28
PS5, Line 28: $
I don't think you want variable expansion here.



--
To view, visit http://gerrit.cloudera.org:8080/9385
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I08b3d2b3f3e14c568d1672ee86ff2c52e8017b81
Gerrit-Change-Number: 9385
Gerrit-PatchSet: 5
Gerrit-Owner: Nithya Janarthanan 
Gerrit-Reviewer: David Knupp 
Gerrit-Reviewer: Michael Brown 
Gerrit-Reviewer: Nithya Janarthanan 
Gerrit-Comment-Date: Fri, 02 Mar 2018 21:28:34 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6341, IMPALA-5917: Reduce mem-limit for start-impala-cluster.

[Tim Armstrong (Code Review)]

Tim Armstrong has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9395 )

Change subject: IMPALA-6341, IMPALA-5917: Reduce mem-limit for 
start-impala-cluster.
..


Patch Set 1: Code-Review+2

I'm ok with this once you fix my comments.


--
To view, visit http://gerrit.cloudera.org:8080/9395
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iabca7a95560bd27c2de2b0a147ee9a3c45199db7
Gerrit-Change-Number: 9395
Gerrit-PatchSet: 1
Gerrit-Owner: Philip Zeyliger 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Fri, 02 Mar 2018 21:14:24 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6542: Fix inconsistent write path of Parquet min/max statistics

[Zoltan Borok-Nagy (Code Review)]

Zoltan Borok-Nagy has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9381 )

Change subject: IMPALA-6542: Fix inconsistent write path of Parquet min/max 
statistics
..


Patch Set 5:

(2 comments)

Thanks for the comments!

http://gerrit.cloudera.org:8080/#/c/9381/1/be/src/exec/parquet-column-stats.inline.h
File be/src/exec/parquet-column-stats.inline.h:

http://gerrit.cloudera.org:8080/#/c/9381/1/be/src/exec/parquet-column-stats.inline.h@37
PS1, Line 37:
> If you prefer, you can use
Yes, it is also a viable approach, however I'd stick with the current solution 
because of the followings:
* more concise
* fewer enable_ifs
* decltype(auto) chooses the right return type we need, and I think it is more 
readable as well


http://gerrit.cloudera.org:8080/#/c/9381/4/be/src/util/stat-util.h
File be/src/util/stat-util.h:

http://gerrit.cloudera.org:8080/#/c/9381/4/be/src/util/stat-util.h@29
PS4, Line 29:   template 
> I think this would be better placed in some parquet-specific files since it
Yeah I agree, btw ColumnStatsBase was my other candidate as well.

Fixed.



--
To view, visit http://gerrit.cloudera.org:8080/9381
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3957806948f7c661af4be5495f2ec92d1e9fc9d6
Gerrit-Change-Number: 9381
Gerrit-PatchSet: 5
Gerrit-Owner: Zoltan Borok-Nagy 
Gerrit-Reviewer: Gabor Kaszab 
Gerrit-Reviewer: Jim Apple 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Reviewer: Zoltan Borok-Nagy 
Gerrit-Comment-Date: Fri, 02 Mar 2018 20:08:46 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6542: Fix inconsistent write path of Parquet min/max statistics

[Zoltan Borok-Nagy (Code Review)]

Hello Lars Volker, Gabor Kaszab, Jim Apple, Tim Armstrong,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/9381

to look at the new patch set (#5).

Change subject: IMPALA-6542: Fix inconsistent write path of Parquet min/max 
statistics
..

IMPALA-6542: Fix inconsistent write path of Parquet min/max statistics

Quick fix of Parquet write path until the Parquet community
agrees on the ordering of floating point numbers.

This commit ignores NaNs during min/max statistics
calculation. Only write NaN as 'min_value' and 'max_value'
if all the values are NaNs in the column chunk.

Added e2e tests as well.

Change-Id: I3957806948f7c661af4be5495f2ec92d1e9fc9d6
---
M be/src/exec/parquet-column-stats.h
M be/src/exec/parquet-column-stats.inline.h
M testdata/workloads/functional-query/queries/QueryTest/parquet-stats.test
3 files changed, 41 insertions(+), 17 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/81/9381/5
--
To view, visit http://gerrit.cloudera.org:8080/9381
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I3957806948f7c661af4be5495f2ec92d1e9fc9d6
Gerrit-Change-Number: 9381
Gerrit-PatchSet: 5
Gerrit-Owner: Zoltan Borok-Nagy 
Gerrit-Reviewer: Gabor Kaszab 
Gerrit-Reviewer: Jim Apple 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Reviewer: Zoltan Borok-Nagy 

[Impala-ASF-CR] IMPALA-2567: Enable KRPC by default

[Dan Hecht (Code Review)]

Dan Hecht has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9461 )

Change subject: IMPALA-2567: Enable KRPC by default
..


Patch Set 3:

(3 comments)

http://gerrit.cloudera.org:8080/#/c/9461/3/bin/start-impala-cluster.py
File bin/start-impala-cluster.py:

http://gerrit.cloudera.org:8080/#/c/9461/3/bin/start-impala-cluster.py@57
PS3, Line 57: Thrift
thrift is always enabled. How about "Enable Thrift DataStream service ..."
Also, flag effect would probably be clearer if named --disable_krpc


http://gerrit.cloudera.org:8080/#/c/9461/3/tests/common/test_skip.py
File tests/common/test_skip.py:

http://gerrit.cloudera.org:8080/#/c/9461/3/tests/common/test_skip.py@35
PS3, Line 35: assert pytest.config.option.test_thrift
:
:   @SkipIf.not_thrift
:   def test_skip_if_not_thrift(self):
: assert not pytest.config.option.test_thrift
these look backwards. don't you have to move the 'not'?


http://gerrit.cloudera.org:8080/#/c/9461/3/tests/conftest.py
File tests/conftest.py:

http://gerrit.cloudera.org:8080/#/c/9461/3/tests/conftest.py@120
PS3, Line 120: Thrift enabled
Thrift DataStream service enabled.
Or "KRPC disabled".
Whichever is consistent with what you decide in the other place.



--
To view, visit http://gerrit.cloudera.org:8080/9461
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iae736c1c1351758969b4d84e34fc5b2d048660a0
Gerrit-Change-Number: 9461
Gerrit-PatchSet: 3
Gerrit-Owner: Michael Ho 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Comment-Date: Fri, 02 Mar 2018 19:08:12 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-2567: Enable KRPC by default

[Michael Ho (Code Review)]

Michael Ho has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9461 )

Change subject: IMPALA-2567: Enable KRPC by default
..


Patch Set 2:

(3 comments)

The existing jenkins jobs to test with KRPC enabled will be replaced with tests 
with Thrift enabled.

http://gerrit.cloudera.org:8080/#/c/9461/2/be/src/runtime/exec-env.cc
File be/src/runtime/exec-env.cc:

http://gerrit.cloudera.org:8080/#/c/9461/2/be/src/runtime/exec-env.cc@83
PS2, Line 83: Used to indicate whether to use KRPC for the "
: "DataStream subsystem, or the Thrift RPC layer instead.
> This could be clearer:
Done


http://gerrit.cloudera.org:8080/#/c/9461/2/bin/run-all-tests.sh
File bin/run-all-tests.sh:

http://gerrit.cloudera.org:8080/#/c/9461/2/bin/run-all-tests.sh@70
PS2, Line 70: if [[ "${TEST_KRPC}" != "false" ]]; then
:   TEST_START_CLUSTER_ARGS="${TEST_START_CLUSTER_ARGS} --use_krpc"
: fi
> and this
Done


http://gerrit.cloudera.org:8080/#/c/9461/2/bin/run-all-tests.sh@124
PS2, Line 124: if [[ "${TEST_KRPC}" != "false" ]]; then
 :   COMMON_PYTEST_ARGS+=" --test_krpc"
 : fi
> I think this needs to be adjusted (so we can test with KRPC disabled)
Done



--
To view, visit http://gerrit.cloudera.org:8080/9461
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iae736c1c1351758969b4d84e34fc5b2d048660a0
Gerrit-Change-Number: 9461
Gerrit-PatchSet: 2
Gerrit-Owner: Michael Ho 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Comment-Date: Fri, 02 Mar 2018 18:59:30 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-2567: Enable KRPC by default

[Michael Ho (Code Review)]

Hello Sailesh Mukil, Dan Hecht,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/9461

to look at the new patch set (#3).

Change subject: IMPALA-2567: Enable KRPC by default
..

IMPALA-2567: Enable KRPC by default

This change enables the switch to use KRPC by default.
This change also fixes a bug in KrpcDataStreamMgr to
check if maintenance thread was started before calling
Join() on it. This shows up in BE tests as the maintenance
thread isn't started in them.

Also bumped the memory limit for a query in test_row_filters().
It seems to fail consistently in exhastive build after recent
scanner memory changes.

Testing done: exhaustive build.

Change-Id: Iae736c1c1351758969b4d84e34fc5b2d048660a0
---
M be/src/runtime/exec-env.cc
M be/src/runtime/krpc-data-stream-mgr.cc
M bin/run-all-tests.sh
M bin/start-impala-cluster.py
M testdata/workloads/functional-query/queries/QueryTest/runtime_row_filters.test
M tests/common/custom_cluster_test_suite.py
M tests/common/skip.py
M tests/common/test_skip.py
M tests/conftest.py
9 files changed, 26 insertions(+), 27 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/61/9461/3
--
To view, visit http://gerrit.cloudera.org:8080/9461
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iae736c1c1351758969b4d84e34fc5b2d048660a0
Gerrit-Change-Number: 9461
Gerrit-PatchSet: 3
Gerrit-Owner: Michael Ho 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Sailesh Mukil 

[Impala-ASF-CR] IMPALA-6585: increase test low mem limit q21 limit

[Impala Public Jenkins (Code Review)]

Impala Public Jenkins has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/9449 )

Change subject: IMPALA-6585: increase test_low_mem_limit_q21 limit
..

IMPALA-6585: increase test_low_mem_limit_q21 limit

Testing:
I could reproduce the failure locally after a few iterations before the
change. After the change, I cannot reproduce it.

Change-Id: I8c721a154e7f8fbb19d043e03fd001990be3f5fd
Reviewed-on: http://gerrit.cloudera.org:8080/9449
Reviewed-by: Tim Armstrong 
Tested-by: Impala Public Jenkins
---
M tests/query_test/test_mem_usage_scaling.py
1 file changed, 1 insertion(+), 1 deletion(-)

Approvals:
  Tim Armstrong: Looks good to me, approved
  Impala Public Jenkins: Verified

--
To view, visit http://gerrit.cloudera.org:8080/9449
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I8c721a154e7f8fbb19d043e03fd001990be3f5fd
Gerrit-Change-Number: 9449
Gerrit-PatchSet: 4
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Tim Armstrong 

[Impala-ASF-CR] IMPALA-6585: increase test low mem limit q21 limit

[Impala Public Jenkins (Code Review)]

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9449 )

Change subject: IMPALA-6585: increase test_low_mem_limit_q21 limit
..


Patch Set 3: Verified+1


--
To view, visit http://gerrit.cloudera.org:8080/9449
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8c721a154e7f8fbb19d043e03fd001990be3f5fd
Gerrit-Change-Number: 9449
Gerrit-PatchSet: 3
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Fri, 02 Mar 2018 10:34:33 +
Gerrit-HasComments: No