date:20190703

Alex Rodoni has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/13802


Change subject: IMPALA-8729: [DOCS] Describe on-demand metadata feature
..

IMPALA-8729: [DOCS] Describe on-demand metadata feature

- Overview of on-demand metadata.
- Config flags to enable/disable on-demand metadata.

Change-Id: I64261625c1d9b122c7cca59f9b004dda05810351
---
M docs/topics/impala_metadata.xml
1 file changed, 102 insertions(+), 0 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/02/13802/1
--
To view, visit http://gerrit.cloudera.org:8080/13802
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I64261625c1d9b122c7cca59f9b004dda05810351
Gerrit-Change-Number: 13802
Gerrit-PatchSet: 1
Gerrit-Owner: Alex Rodoni

[Impala-ASF-CR] Revert "build: use thin static archives"

2019-07-03 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13798 )

Change subject: Revert "build: use thin static archives"
..


Patch Set 1: Verified+1


--
To view, visit http://gerrit.cloudera.org:8080/13798
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I777ca8b4e862c8bfd8c06135834161fe91fefe67
Gerrit-Change-Number: 13798
Gerrit-PatchSet: 1
Gerrit-Owner: Joe McDonnell 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Comment-Date: Wed, 03 Jul 2019 23:32:07 +
Gerrit-HasComments: No

[Impala-ASF-CR] [WIP] Bump CDP BUILD NUMBER to 1235229

2019-07-03 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13799 )

Change subject: [WIP] Bump CDP_BUILD_NUMBER to 1235229
..


Patch Set 2:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/3822/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/13799
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I6d0fa856fa6c7cb1f4c9ee0e4250cfa1e8a14595
Gerrit-Change-Number: 13799
Gerrit-PatchSet: 2
Gerrit-Owner: Vihang Karajgaonkar 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Wed, 03 Jul 2019 22:25:59 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-5149: Provide query profile in JSON format

2019-07-03 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13801 )

Change subject: IMPALA-5149: Provide query profile in JSON format
..


Patch Set 1:

Build Failed

https://jenkins.impala.io/job/gerrit-code-review-checks/3821/ : Initial code 
review checks failed. See linked job for details on the failure.


--
To view, visit http://gerrit.cloudera.org:8080/13801
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8181ac818bf22207ca1deabd9220c397ae723ec1
Gerrit-Change-Number: 13801
Gerrit-PatchSet: 1
Gerrit-Owner: Jiawei Wang 
Gerrit-Reviewer: Bharath Vissapragada 
Gerrit-Reviewer: David Rorke 
Gerrit-Reviewer: Greg Rahn 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Wed, 03 Jul 2019 22:09:39 +
Gerrit-HasComments: No

[Impala-ASF-CR] [WIP] Bump CDP BUILD NUMBER to 1235229

2019-07-03 Thread Vihang Karajgaonkar (Code Review)

Vihang Karajgaonkar has uploaded a new patch set (#2). ( 
http://gerrit.cloudera.org:8080/13799 )

Change subject: [WIP] Bump CDP_BUILD_NUMBER to 1235229
..

[WIP] Bump CDP_BUILD_NUMBER to 1235229

The newer CDP build includes fix for HIVE-21932 which is required to fix
test failures on CDP builds.

Also, fixed the MetastoreEventsProcessorTest on CDP caused due to the
serialization difference of the event messages between CDH and CDP
builds.

[WIP] Testing:
1. Build against CDP and CDH builds
2. Run the previously failing tests on CDP which were caused by
HIVE-21932
3. Run full tests with CDP and CDH builds

Change-Id: I6d0fa856fa6c7cb1f4c9ee0e4250cfa1e8a14595
---
M bin/impala-config.sh
M fe/src/compat-hive-2/java/org/apache/impala/compat/MetastoreShim.java
M fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java
M 
fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java
4 files changed, 34 insertions(+), 15 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/99/13799/2
--
To view, visit http://gerrit.cloudera.org:8080/13799
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I6d0fa856fa6c7cb1f4c9ee0e4250cfa1e8a14595
Gerrit-Change-Number: 13799
Gerrit-PatchSet: 2
Gerrit-Owner: Vihang Karajgaonkar 
Gerrit-Reviewer: Impala Public Jenkins

[Impala-ASF-CR] IMPALA-5149: Provide query profile in JSON format

2019-07-03 Thread Jiawei Wang (Code Review)

Jiawei Wang has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/13801


Change subject: IMPALA-5149: Provide query profile in JSON format
..

IMPALA-5149: Provide query profile in JSON format

Today there is a text and Thrift version of the query profile, but
it would be useful to have a JSON version for portability and machine
consumption. The ultimate goal is to have a Download JSON format
profile link along with the other two formats.

Design Document:
https://docs.google.com/document/d/
15P_Lmjf1rlZUD4PZLXdUwTE8Lv_ME3NQ9Yf4MITzO2M/edit

Change-Id: I8181ac818bf22207ca1deabd9220c397ae723ec1
---
M be/src/service/impala-beeswax-server.cc
M be/src/service/impala-hs2-server.cc
M be/src/service/impala-http-handler.cc
M be/src/service/impala-http-handler.h
M be/src/service/impala-server.cc
M be/src/service/impala-server.h
M be/src/util/runtime-profile-counters.h
M be/src/util/runtime-profile.cc
M be/src/util/runtime-profile.h
M common/thrift/RuntimeProfile.thrift
M www/query_profile.tmpl
11 files changed, 363 insertions(+), 11 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/01/13801/1
--
To view, visit http://gerrit.cloudera.org:8080/13801
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I8181ac818bf22207ca1deabd9220c397ae723ec1
Gerrit-Change-Number: 13801
Gerrit-PatchSet: 1
Gerrit-Owner: Jiawei Wang 
Gerrit-Reviewer: Bharath Vissapragada 
Gerrit-Reviewer: David Rorke 
Gerrit-Reviewer: Greg Rahn 
Gerrit-Reviewer: Impala Public Jenkins

[Impala-ASF-CR] IMPALA-5149: Provide query profile in JSON format

2019-07-03 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13801 )

Change subject: IMPALA-5149: Provide query profile in JSON format
..


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/13801/1/be/src/util/runtime-profile.cc
File be/src/util/runtime-profile.cc:

http://gerrit.cloudera.org:8080/#/c/13801/1/be/src/util/runtime-profile.cc@718
PS1, Line 718:  const string& 
counter_name, const CounterMap& counter_map,
line too long (99 > 90)



--
To view, visit http://gerrit.cloudera.org:8080/13801
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8181ac818bf22207ca1deabd9220c397ae723ec1
Gerrit-Change-Number: 13801
Gerrit-PatchSet: 1
Gerrit-Owner: Jiawei Wang 
Gerrit-Reviewer: Bharath Vissapragada 
Gerrit-Reviewer: David Rorke 
Gerrit-Reviewer: Greg Rahn 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Wed, 03 Jul 2019 21:29:52 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] [DOCS] A typo fix in impala metadata

Alex Rodoni has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/13800 )

Change subject: [DOCS] A typo fix in impala_metadata
..

[DOCS] A typo fix in impala_metadata

Change-Id: I12c6a9777622e0142a17e58629bb852d8a9aa934
Reviewed-on: http://gerrit.cloudera.org:8080/13800
Reviewed-by: Alex Rodoni 
Tested-by: Impala Public Jenkins 
---
M docs/topics/impala_metadata.xml
1 file changed, 2 insertions(+), 2 deletions(-)

Approvals:
  Alex Rodoni: Looks good to me, approved
  Impala Public Jenkins: Verified

--
To view, visit http://gerrit.cloudera.org:8080/13800
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I12c6a9777622e0142a17e58629bb852d8a9aa934
Gerrit-Change-Number: 13800
Gerrit-PatchSet: 2
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins

[Impala-ASF-CR] [DOCS] A typo fix in impala metadata

2019-07-03 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13800 )

Change subject: [DOCS] A typo fix in impala_metadata
..


Patch Set 1: Verified+1

Build Successful

https://jenkins.impala.io/job/gerrit-docs-auto-test/377/ : Doc tests passed.


--
To view, visit http://gerrit.cloudera.org:8080/13800
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I12c6a9777622e0142a17e58629bb852d8a9aa934
Gerrit-Change-Number: 13800
Gerrit-PatchSet: 1
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Wed, 03 Jul 2019 20:49:32 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8681: Only show ValidWriteIdLists for Acid tables

2019-07-03 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13736 )

Change subject: IMPALA-8681: Only show ValidWriteIdLists for Acid tables
..


Patch Set 4: Verified+1


--
To view, visit http://gerrit.cloudera.org:8080/13736
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifcc31c7ddcfc471b0e5308f7e4aaadfa8189a905
Gerrit-Change-Number: 13736
Gerrit-PatchSet: 4
Gerrit-Owner: Yongzhi Chen 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Reviewer: Yongzhi Chen 
Gerrit-Reviewer: Zoltan Borok-Nagy 
Gerrit-Comment-Date: Wed, 03 Jul 2019 20:46:44 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8681: Only show ValidWriteIdLists for Acid tables

2019-07-03 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/13736 )

Change subject: IMPALA-8681: Only show ValidWriteIdLists for Acid tables
..

IMPALA-8681: Only show ValidWriteIdLists for Acid tables

Lists ValidWriteIds for transactional tables in profile.
If a query does not trigger any transactional table loading,
the query profile will not have the "Loaded ValidWriteIdLists"
timeline.

Tests:
Manual tests.
Fixed StmtMetadataLoaderTest.
Added acid_profile test

Sample output:
Query Compilation: 3s525ms
   - Metadata load started: 37.369ms (37.369ms)
   - Metadata load finished. loaded-tables=1/1 ...
   - Loaded ValidWriteIdLists for transactional tables:
   functional.insert_only_transactional_table:0:9223372036854775807::
 : 3s312ms (551.463us)
   - Analysis finished: 3s370ms (58.110ms)
   ...

Change-Id: Ifcc31c7ddcfc471b0e5308f7e4aaadfa8189a905
Reviewed-on: http://gerrit.cloudera.org:8080/13736
Reviewed-by: Csaba Ringhofer 
Tested-by: Impala Public Jenkins 
---
M fe/src/main/java/org/apache/impala/analysis/StmtMetadataLoader.java
M fe/src/test/java/org/apache/impala/analysis/StmtMetadataLoaderTest.java
A testdata/workloads/functional-query/queries/QueryTest/acid-profile.test
M tests/common/skip.py
M tests/query_test/test_acid.py
5 files changed, 46 insertions(+), 10 deletions(-)

Approvals:
  Csaba Ringhofer: Looks good to me, approved
  Impala Public Jenkins: Verified

--
To view, visit http://gerrit.cloudera.org:8080/13736
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ifcc31c7ddcfc471b0e5308f7e4aaadfa8189a905
Gerrit-Change-Number: 13736
Gerrit-PatchSet: 5
Gerrit-Owner: Yongzhi Chen 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Reviewer: Yongzhi Chen 
Gerrit-Reviewer: Zoltan Borok-Nagy

[Impala-ASF-CR] IMPALA-8663 : FileMetadataLoader should skip hidden and tmp directories

2019-07-03 Thread Vihang Karajgaonkar (Code Review)

Vihang Karajgaonkar has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13665 )

Change subject: IMPALA-8663 : FileMetadataLoader should skip hidden and tmp 
directories
..


Patch Set 8:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/13665/8/fe/src/main/java/org/apache/impala/common/FileSystemUtil.java
File fe/src/main/java/org/apache/impala/common/FileSystemUtil.java:

http://gerrit.cloudera.org:8080/#/c/13665/8/fe/src/main/java/org/apache/impala/common/FileSystemUtil.java@545
PS8, Line 545: all underlying files (except which are
 :* in the ignored directories)
> hrm, were the semantics already broken then? See the unit test AcidUtilsTes
yeah, it looks like its broken currently for this case. I will file a JIRA. The 
AcidUtils test works because it works off the List not the ones 
which are actually returned by the FileMetadataLoader.

Specifically, if the table is transactional we cannot rely on using listFiles 
API since it will skip empty directories (Or we should implement a version of 
it for own purposes). Not sure how to handle the S3 recursive listFiles case 
though.

Created https://issues.apache.org/jira/browse/IMPALA-8739 as suggested



--
To view, visit http://gerrit.cloudera.org:8080/13665
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2c4a22908304fe9e377d77d6c18d401c3f3294aa
Gerrit-Change-Number: 13665
Gerrit-PatchSet: 8
Gerrit-Owner: Vihang Karajgaonkar 
Gerrit-Reviewer: Bharath Vissapragada 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Reviewer: Vihang Karajgaonkar 
Gerrit-Comment-Date: Wed, 03 Jul 2019 20:45:32 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] [WIP] Bump CDP BUILD NUMBER to 1235229

2019-07-03 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13799 )

Change subject: [WIP] Bump CDP_BUILD_NUMBER to 1235229
..


Patch Set 1:

Build Failed

https://jenkins.impala.io/job/gerrit-code-review-checks/3820/ : Initial code 
review checks failed. See linked job for details on the failure.


--
To view, visit http://gerrit.cloudera.org:8080/13799
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I6d0fa856fa6c7cb1f4c9ee0e4250cfa1e8a14595
Gerrit-Change-Number: 13799
Gerrit-PatchSet: 1
Gerrit-Owner: Vihang Karajgaonkar 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Wed, 03 Jul 2019 20:36:30 +
Gerrit-HasComments: No

[Impala-ASF-CR] [DOCS] A typo fix in impala metadata

Alex Rodoni has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/13800


Change subject: [DOCS] A typo fix in impala_metadata
..

[DOCS] A typo fix in impala_metadata

Change-Id: I12c6a9777622e0142a17e58629bb852d8a9aa934
---
M docs/topics/impala_metadata.xml
1 file changed, 2 insertions(+), 2 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/00/13800/1
--
To view, visit http://gerrit.cloudera.org:8080/13800
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I12c6a9777622e0142a17e58629bb852d8a9aa934
Gerrit-Change-Number: 13800
Gerrit-PatchSet: 1
Gerrit-Owner: Alex Rodoni

[Impala-ASF-CR] [DOCS] A typo fix in impala metadata

Alex Rodoni has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13800 )

Change subject: [DOCS] A typo fix in impala_metadata
..


Patch Set 1: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/13800
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I12c6a9777622e0142a17e58629bb852d8a9aa934
Gerrit-Change-Number: 13800
Gerrit-PatchSet: 1
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Wed, 03 Jul 2019 20:26:31 +
Gerrit-HasComments: No

[Impala-ASF-CR] [DOCS] A typo fix in impala metadata

2019-07-03 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13800 )

Change subject: [DOCS] A typo fix in impala_metadata
..


Patch Set 1:

Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/377/

Testing docs change - this change appears to modify docs/ and no code. This is 
experimental - please report any issues to tarmstr...@cloudera.com or on this 
JIRA: IMPALA-7317


--
To view, visit http://gerrit.cloudera.org:8080/13800
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I12c6a9777622e0142a17e58629bb852d8a9aa934
Gerrit-Change-Number: 13800
Gerrit-PatchSet: 1
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Wed, 03 Jul 2019 20:26:16 +
Gerrit-HasComments: No

[Impala-ASF-CR] [WIP] Bump CDP BUILD NUMBER to 1235229

2019-07-03 Thread Vihang Karajgaonkar (Code Review)

Vihang Karajgaonkar has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/13799


Change subject: [WIP] Bump CDP_BUILD_NUMBER to 1235229
..

[WIP] Bump CDP_BUILD_NUMBER to 1235229

The newer CDP build includes fix for HIVE-21932 which is required to fix
test failures on CDP builds.

Also, fixed the MetastoreEventsProcessorTest on CDP caused due to the
serialization difference of the event messages between CDH and CDP
builds.

[WIP] Testing:
1. Build against CDP and CDH builds
2. Run the previously failing tests on CDP which were caused by
HIVE-21932
3. Run full tests with CDP and CDH builds

Change-Id: I6d0fa856fa6c7cb1f4c9ee0e4250cfa1e8a14595
---
M bin/impala-config.sh
M fe/src/compat-hive-2/java/org/apache/impala/compat/MetastoreShim.java
M fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java
M 
fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java
4 files changed, 33 insertions(+), 15 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/99/13799/1
--
To view, visit http://gerrit.cloudera.org:8080/13799
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I6d0fa856fa6c7cb1f4c9ee0e4250cfa1e8a14595
Gerrit-Change-Number: 13799
Gerrit-PatchSet: 1
Gerrit-Owner: Vihang Karajgaonkar

[Impala-ASF-CR] IMPALA-8736: test hive parquet codec interop fails on S3

2019-07-03 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13781 )

Change subject: IMPALA-8736: test_hive_parquet_codec_interop fails on S3
..


Patch Set 2:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/3819/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/13781
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie8eee8ba44e6acf4a069467f749756f9040ef641
Gerrit-Change-Number: 13781
Gerrit-PatchSet: 2
Gerrit-Owner: Abhishek Rawat 
Gerrit-Reviewer: Anurag Mantripragada 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Comment-Date: Wed, 03 Jul 2019 19:42:51 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8427: [DOCS] Document the new startup flag IMPALA-7800 introduced

Alex Rodoni has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/13762 )

Change subject: IMPALA-8427: [DOCS] Document the new startup flag IMPALA-7800 
introduced
..

IMPALA-8427: [DOCS] Document the new startup flag IMPALA-7800 introduced

- Added a new doc impala_client.xml as the overview of Impala impala
  client access.

Change-Id: I1a4c1975721c32a78a003d91babc5d2bb90f3949
Reviewed-on: http://gerrit.cloudera.org:8080/13762
Tested-by: Impala Public Jenkins 
Reviewed-by: Michael Ho 
---
M docs/impala.ditamap
A docs/topics/impala_client.xml
M docs/topics/impala_config_options.xml
3 files changed, 153 insertions(+), 55 deletions(-)

Approvals:
  Impala Public Jenkins: Verified
  Michael Ho: Looks good to me, approved

-- 
To view, visit http://gerrit.cloudera.org:8080/13762
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I1a4c1975721c32a78a003d91babc5d2bb90f3949
Gerrit-Change-Number: 13762
Gerrit-PatchSet: 3
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Michael Ho

[Impala-ASF-CR] IMPALA-8736: test hive parquet codec interop fails on S3

2019-07-03 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13781 )

Change subject: IMPALA-8736: test_hive_parquet_codec_interop fails on S3
..


Patch Set 3: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/13781
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie8eee8ba44e6acf4a069467f749756f9040ef641
Gerrit-Change-Number: 13781
Gerrit-PatchSet: 3
Gerrit-Owner: Abhishek Rawat 
Gerrit-Reviewer: Anurag Mantripragada 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Comment-Date: Wed, 03 Jul 2019 19:08:25 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8736: test hive parquet codec interop fails on S3

2019-07-03 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13781 )

Change subject: IMPALA-8736: test_hive_parquet_codec_interop fails on S3
..


Patch Set 3:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4581/ 
DRY_RUN=false


--
To view, visit http://gerrit.cloudera.org:8080/13781
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie8eee8ba44e6acf4a069467f749756f9040ef641
Gerrit-Change-Number: 13781
Gerrit-PatchSet: 3
Gerrit-Owner: Abhishek Rawat 
Gerrit-Reviewer: Anurag Mantripragada 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Comment-Date: Wed, 03 Jul 2019 19:08:26 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8736: test hive parquet codec interop fails on S3

Joe McDonnell has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13781 )

Change subject: IMPALA-8736: test_hive_parquet_codec_interop fails on S3
..


Patch Set 2: Code-Review+2

Thanks for fixing this


--
To view, visit http://gerrit.cloudera.org:8080/13781
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie8eee8ba44e6acf4a069467f749756f9040ef641
Gerrit-Change-Number: 13781
Gerrit-PatchSet: 2
Gerrit-Owner: Abhishek Rawat 
Gerrit-Reviewer: Anurag Mantripragada 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Comment-Date: Wed, 03 Jul 2019 19:05:08 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8736: test hive parquet codec interop fails on S3

2019-07-03 Thread Abhishek Rawat (Code Review)

Abhishek Rawat has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/13781


Change subject: IMPALA-8736: test_hive_parquet_codec_interop fails on S3
..

IMPALA-8736: test_hive_parquet_codec_interop fails on S3

The test case fails (hanging untill the timeout is hit),  when trying
to do hive insert on S3. Support for hive insert is flaky in S3 and so
the testcase should be disabled for S3.

Change-Id: Ie8eee8ba44e6acf4a069467f749756f9040ef641
---
M tests/custom_cluster/test_hive_parquet_codec_interop.py
1 file changed, 2 insertions(+), 0 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/81/13781/2
--
To view, visit http://gerrit.cloudera.org:8080/13781
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ie8eee8ba44e6acf4a069467f749756f9040ef641
Gerrit-Change-Number: 13781
Gerrit-PatchSet: 2
Gerrit-Owner: Abhishek Rawat 
Gerrit-Reviewer: Anurag Mantripragada 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell

[Impala-ASF-CR] IMPALA-8519: [DOCS] Doc the limitation in insert events from SparkSQL

Alex Rodoni has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/13777 )

Change subject: IMPALA-8519: [DOCS] Doc the limitation in insert events from 
SparkSQL
..

IMPALA-8519: [DOCS] Doc the limitation in insert events from SparkSQL

- Also made a few formatting changes.
- Removed the Preview Release note for Invalidation of Metadata cache.

Change-Id: I36cfc7e592ed2588a8c1f8375033d60492b27a4f
Reviewed-on: http://gerrit.cloudera.org:8080/13777
Reviewed-by: Vihang Karajgaonkar 
Tested-by: Impala Public Jenkins 
---
M docs/topics/impala_metadata.xml
1 file changed, 68 insertions(+), 34 deletions(-)

Approvals:
  Vihang Karajgaonkar: Looks good to me, approved
  Impala Public Jenkins: Verified

--
To view, visit http://gerrit.cloudera.org:8080/13777
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I36cfc7e592ed2588a8c1f8375033d60492b27a4f
Gerrit-Change-Number: 13777
Gerrit-PatchSet: 7
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Alex Rodoni 
Gerrit-Reviewer: Bharath Vissapragada 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Vihang Karajgaonkar

[Impala-ASF-CR] IMPALA-8519: [DOCS] Doc the limitation in insert events from SparkSQL

2019-07-03 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13777 )

Change subject: IMPALA-8519: [DOCS] Doc the limitation in insert events from 
SparkSQL
..


Patch Set 6: Verified+1

Build Successful 

https://jenkins.impala.io/job/gerrit-docs-auto-test/376/ : Doc tests passed.


--
To view, visit http://gerrit.cloudera.org:8080/13777
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I36cfc7e592ed2588a8c1f8375033d60492b27a4f
Gerrit-Change-Number: 13777
Gerrit-PatchSet: 6
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Alex Rodoni 
Gerrit-Reviewer: Bharath Vissapragada 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Vihang Karajgaonkar 
Gerrit-Comment-Date: Wed, 03 Jul 2019 18:48:37 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8519: [DOCS] Doc the limitation in insert events from SparkSQL

2019-07-03 Thread Vihang Karajgaonkar (Code Review)

Vihang Karajgaonkar has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13777 )

Change subject: IMPALA-8519: [DOCS] Doc the limitation in insert events from 
SparkSQL
..


Patch Set 6: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/13777
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I36cfc7e592ed2588a8c1f8375033d60492b27a4f
Gerrit-Change-Number: 13777
Gerrit-PatchSet: 6
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Alex Rodoni 
Gerrit-Reviewer: Bharath Vissapragada 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Vihang Karajgaonkar 
Gerrit-Comment-Date: Wed, 03 Jul 2019 18:39:29 +
Gerrit-HasComments: No

[Impala-ASF-CR] Revert "build: use thin static archives"

Joe McDonnell has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/13798 )

Change subject: Revert "build: use thin static archives"
..

Revert "build: use thin static archives"

This reverts commit 78c55230288f8874bcd16454eb3c55277211719a.

Change-Id: I777ca8b4e862c8bfd8c06135834161fe91fefe67
Reviewed-on: http://gerrit.cloudera.org:8080/13798
Reviewed-by: Joe McDonnell 
Tested-by: Joe McDonnell 
---
M CMakeLists.txt
1 file changed, 0 insertions(+), 9 deletions(-)

Approvals:
  Joe McDonnell: Looks good to me, approved; Verified

--
To view, visit http://gerrit.cloudera.org:8080/13798
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I777ca8b4e862c8bfd8c06135834161fe91fefe67
Gerrit-Change-Number: 13798
Gerrit-PatchSet: 2
Gerrit-Owner: Joe McDonnell 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell

[Impala-ASF-CR] Revert "build: use thin static archives"

Joe McDonnell has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13798 )

Change subject: Revert "build: use thin static archives"
..


Patch Set 1: Verified+1

This is reverting a small CMake change that broke Centos 6. Impala has been 
building without this change for a long time (and the verification jobs are 
past the build stage), so I'm just going to verify and push this in.


--
To view, visit http://gerrit.cloudera.org:8080/13798
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I777ca8b4e862c8bfd8c06135834161fe91fefe67
Gerrit-Change-Number: 13798
Gerrit-PatchSet: 1
Gerrit-Owner: Joe McDonnell 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Comment-Date: Wed, 03 Jul 2019 18:32:31 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8519: [DOCS] Doc the limitation in insert events from SparkSQL

2019-07-03 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13777 )

Change subject: IMPALA-8519: [DOCS] Doc the limitation in insert events from 
SparkSQL
..


Patch Set 6:

Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/376/

Testing docs change - this change appears to modify docs/ and no code. This is 
experimental - please report any issues to tarmstr...@cloudera.com or on this 
JIRA: IMPALA-7317


--
To view, visit http://gerrit.cloudera.org:8080/13777
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I36cfc7e592ed2588a8c1f8375033d60492b27a4f
Gerrit-Change-Number: 13777
Gerrit-PatchSet: 6
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Alex Rodoni 
Gerrit-Reviewer: Bharath Vissapragada 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Vihang Karajgaonkar 
Gerrit-Comment-Date: Wed, 03 Jul 2019 18:25:45 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8519: [DOCS] Doc the limitation in insert events from SparkSQL

Alex Rodoni has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13777 )

Change subject: IMPALA-8519: [DOCS] Doc the limitation in insert events from 
SparkSQL
..


Patch Set 5:

(4 comments)

http://gerrit.cloudera.org:8080/#/c/13777/5/docs/topics/impala_metadata.xml
File docs/topics/impala_metadata.xml:

http://gerrit.cloudera.org:8080/#/c/13777/5/docs/topics/impala_metadata.xml@137
PS5, Line 137: table
> In this case, it actually refreshes the partition not the table
Done


http://gerrit.cloudera.org:8080/#/c/13777/5/docs/topics/impala_metadata.xml@216
PS5, Line 216: operation
> Using APIs instead of operation is better in my opinion.
Done


http://gerrit.cloudera.org:8080/#/c/13777/5/docs/topics/impala_metadata.xml@218
PS5, Line 218: /tmp/spark_unpart
> I think it would be easier to understand if the example provides a realisti
Done


http://gerrit.cloudera.org:8080/#/c/13777/5/docs/topics/impala_metadata.xml@258
PS5, Line 258: hive.metastore.dml.events
> We should modify this text to say that this configuration should also be se
Done



--
To view, visit http://gerrit.cloudera.org:8080/13777
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I36cfc7e592ed2588a8c1f8375033d60492b27a4f
Gerrit-Change-Number: 13777
Gerrit-PatchSet: 5
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Alex Rodoni 
Gerrit-Reviewer: Bharath Vissapragada 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Vihang Karajgaonkar 
Gerrit-Comment-Date: Wed, 03 Jul 2019 18:25:43 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-8519: [DOCS] Doc the limitation in insert events from SparkSQL

Hello Bharath Vissapragada, Vihang Karajgaonkar, Impala Public Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/13777

to look at the new patch set (#6).

Change subject: IMPALA-8519: [DOCS] Doc the limitation in insert events from 
SparkSQL
..

IMPALA-8519: [DOCS] Doc the limitation in insert events from SparkSQL

- Also made a few formatting changes.
- Removed the Preview Release note for Invalidation of Metadata cache.

Change-Id: I36cfc7e592ed2588a8c1f8375033d60492b27a4f
---
M docs/topics/impala_metadata.xml
1 file changed, 68 insertions(+), 34 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/77/13777/6
--
To view, visit http://gerrit.cloudera.org:8080/13777
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I36cfc7e592ed2588a8c1f8375033d60492b27a4f
Gerrit-Change-Number: 13777
Gerrit-PatchSet: 6
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Alex Rodoni 
Gerrit-Reviewer: Bharath Vissapragada 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Vihang Karajgaonkar

[Impala-ASF-CR] Revert "build: use thin static archives"

2019-07-03 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13798 )

Change subject: Revert "build: use thin static archives"
..


Patch Set 1:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/3818/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/13798
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I777ca8b4e862c8bfd8c06135834161fe91fefe67
Gerrit-Change-Number: 13798
Gerrit-PatchSet: 1
Gerrit-Owner: Joe McDonnell 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Comment-Date: Wed, 03 Jul 2019 18:23:12 +
Gerrit-HasComments: No

[Impala-ASF-CR] Revert "build: use thin static archives"

2019-07-03 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13798 )

Change subject: Revert "build: use thin static archives"
..


Patch Set 1:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4580/ 
DRY_RUN=false


--
To view, visit http://gerrit.cloudera.org:8080/13798
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I777ca8b4e862c8bfd8c06135834161fe91fefe67
Gerrit-Change-Number: 13798
Gerrit-PatchSet: 1
Gerrit-Owner: Joe McDonnell 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Comment-Date: Wed, 03 Jul 2019 18:04:20 +
Gerrit-HasComments: No

[Impala-ASF-CR] Revert "build: use thin static archives"

Joe McDonnell has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13798 )

Change subject: Revert "build: use thin static archives"
..


Patch Set 1: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/13798
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I777ca8b4e862c8bfd8c06135834161fe91fefe67
Gerrit-Change-Number: 13798
Gerrit-PatchSet: 1
Gerrit-Owner: Joe McDonnell 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Comment-Date: Wed, 03 Jul 2019 18:03:54 +
Gerrit-HasComments: No

[Impala-ASF-CR] Revert "build: use thin static archives"

2019-07-03 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13798 )

Change subject: Revert "build: use thin static archives"
..


Patch Set 1:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4579/ 
DRY_RUN=true


--
To view, visit http://gerrit.cloudera.org:8080/13798
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I777ca8b4e862c8bfd8c06135834161fe91fefe67
Gerrit-Change-Number: 13798
Gerrit-PatchSet: 1
Gerrit-Owner: Joe McDonnell 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Wed, 03 Jul 2019 17:43:56 +
Gerrit-HasComments: No

[Impala-ASF-CR] Revert "build: use thin static archives"

Joe McDonnell has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/13798


Change subject: Revert "build: use thin static archives"
..

Revert "build: use thin static archives"

This reverts commit 78c55230288f8874bcd16454eb3c55277211719a.

Change-Id: I777ca8b4e862c8bfd8c06135834161fe91fefe67
---
M CMakeLists.txt
1 file changed, 0 insertions(+), 9 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/98/13798/1
--
To view, visit http://gerrit.cloudera.org:8080/13798
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I777ca8b4e862c8bfd8c06135834161fe91fefe67
Gerrit-Change-Number: 13798
Gerrit-PatchSet: 1
Gerrit-Owner: Joe McDonnell

[Impala-ASF-CR] IMPALA-8709: Add Damerau-Levenshtein edit distance built-in function

2019-07-03 Thread Greg Rahn (Code Review)

Greg Rahn has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13794 )

Change subject: IMPALA-8709: Add Damerau-Levenshtein edit distance built-in 
function
..


Patch Set 1:

> (1 comment)

I'll clarify in the commit msg and docs -- this is the restricted DL (or OSA) 
as you pointed out, not the full DL.


--
To view, visit http://gerrit.cloudera.org:8080/13794
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib759817ec15e7075bf49d51e494e45c8af4db94d
Gerrit-Change-Number: 13794
Gerrit-PatchSet: 1
Gerrit-Owner: Greg Rahn 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Greg Rahn 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Zoltan Borok-Nagy 
Gerrit-Comment-Date: Wed, 03 Jul 2019 17:41:13 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8341: [DOCS] Added a note about the requirement for existing dirs

Alex Rodoni has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/13793 )

Change subject: IMPALA-8341: [DOCS] Added a note about the requirement for 
existing dirs
..

IMPALA-8341: [DOCS] Added a note about the requirement for existing dirs

Change-Id: I5feff3ab7c09ee681098ec3e630977cb92f1
Reviewed-on: http://gerrit.cloudera.org:8080/13793
Reviewed-by: Alex Rodoni 
Tested-by: Impala Public Jenkins 
---
M docs/topics/impala_data_cache.xml
1 file changed, 3 insertions(+), 2 deletions(-)

Approvals:
  Alex Rodoni: Looks good to me, approved
  Impala Public Jenkins: Verified

--
To view, visit http://gerrit.cloudera.org:8080/13793
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I5feff3ab7c09ee681098ec3e630977cb92f1
Gerrit-Change-Number: 13793
Gerrit-PatchSet: 2
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins

[Impala-ASF-CR] IMPALA-8519: [DOCS] Doc the limitation in insert events from SparkSQL

2019-07-03 Thread Vihang Karajgaonkar (Code Review)

Vihang Karajgaonkar has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13777 )

Change subject: IMPALA-8519: [DOCS] Doc the limitation in insert events from 
SparkSQL
..


Patch Set 5:

(4 comments)

http://gerrit.cloudera.org:8080/#/c/13777/5/docs/topics/impala_metadata.xml
File docs/topics/impala_metadata.xml:

http://gerrit.cloudera.org:8080/#/c/13777/5/docs/topics/impala_metadata.xml@137
PS5, Line 137: table
In this case, it actually refreshes the partition not the table


http://gerrit.cloudera.org:8080/#/c/13777/5/docs/topics/impala_metadata.xml@216
PS5, Line 216: operation
Using APIs instead of operation is better in my opinion.


http://gerrit.cloudera.org:8080/#/c/13777/5/docs/topics/impala_metadata.xml@218
PS5, Line 218: /tmp/spark_unpart
I think it would be easier to understand if the example provides a realistic 
table location like /user/hive/warehouse/spark_etl.db/customers/date=01012019


http://gerrit.cloudera.org:8080/#/c/13777/5/docs/topics/impala_metadata.xml@258
PS5, Line 258: hive.metastore.dml.events
We should modify this text to say that this configuration should also be set to 
true in the hive-site.xml used by the spark applications (typically 
/etc/hive/conf/hive-site.xml) so that INSERT events are generated when the 
spark application inserts data into existing tables and partitions.



--
To view, visit http://gerrit.cloudera.org:8080/13777
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I36cfc7e592ed2588a8c1f8375033d60492b27a4f
Gerrit-Change-Number: 13777
Gerrit-PatchSet: 5
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Alex Rodoni 
Gerrit-Reviewer: Bharath Vissapragada 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Vihang Karajgaonkar 
Gerrit-Comment-Date: Wed, 03 Jul 2019 17:36:39 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] [DOCS] A format fix in impala txtfile

Alex Rodoni has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/13797 )

Change subject: [DOCS] A format fix in impala_txtfile
..

[DOCS] A format fix in impala_txtfile

Change-Id: I63c2d4dfb4985bc2560fb559bee3f4987d415405
Reviewed-on: http://gerrit.cloudera.org:8080/13797
Reviewed-by: Alex Rodoni 
Tested-by: Impala Public Jenkins 
---
M docs/topics/impala_txtfile.xml
1 file changed, 2 insertions(+), 3 deletions(-)

Approvals:
  Alex Rodoni: Looks good to me, approved
  Impala Public Jenkins: Verified

--
To view, visit http://gerrit.cloudera.org:8080/13797
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I63c2d4dfb4985bc2560fb559bee3f4987d415405
Gerrit-Change-Number: 13797
Gerrit-PatchSet: 2
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins

[Impala-ASF-CR] [DOCS] A format fix in impala txtfile

2019-07-03 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13797 )

Change subject: [DOCS] A format fix in impala_txtfile
..


Patch Set 1: Verified+1

Build Successful

https://jenkins.impala.io/job/gerrit-docs-auto-test/375/ : Doc tests passed.


--
To view, visit http://gerrit.cloudera.org:8080/13797
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I63c2d4dfb4985bc2560fb559bee3f4987d415405
Gerrit-Change-Number: 13797
Gerrit-PatchSet: 1
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Wed, 03 Jul 2019 17:20:05 +
Gerrit-HasComments: No

[Impala-ASF-CR] [DOCS] A format fix in impala txtfile

Alex Rodoni has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/13797


Change subject: [DOCS] A format fix in impala_txtfile
..

[DOCS] A format fix in impala_txtfile

Change-Id: I63c2d4dfb4985bc2560fb559bee3f4987d415405
---
M docs/topics/impala_txtfile.xml
1 file changed, 2 insertions(+), 3 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/97/13797/1
--
To view, visit http://gerrit.cloudera.org:8080/13797
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I63c2d4dfb4985bc2560fb559bee3f4987d415405
Gerrit-Change-Number: 13797
Gerrit-PatchSet: 1
Gerrit-Owner: Alex Rodoni

[Impala-ASF-CR] [DOCS] A format fix in impala txtfile

Alex Rodoni has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13797 )

Change subject: [DOCS] A format fix in impala_txtfile
..


Patch Set 1: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/13797
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I63c2d4dfb4985bc2560fb559bee3f4987d415405
Gerrit-Change-Number: 13797
Gerrit-PatchSet: 1
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Wed, 03 Jul 2019 17:08:30 +
Gerrit-HasComments: No

[Impala-ASF-CR] [DOCS] A format fix in impala txtfile

2019-07-03 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13797 )

Change subject: [DOCS] A format fix in impala_txtfile
..


Patch Set 1:

Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/375/

Testing docs change - this change appears to modify docs/ and no code. This is 
experimental - please report any issues to tarmstr...@cloudera.com or on this 
JIRA: IMPALA-7317


--
To view, visit http://gerrit.cloudera.org:8080/13797
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I63c2d4dfb4985bc2560fb559bee3f4987d415405
Gerrit-Change-Number: 13797
Gerrit-PatchSet: 1
Gerrit-Owner: Alex Rodoni 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Wed, 03 Jul 2019 17:08:10 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8681: Only show ValidWriteIdLists for Acid tables

2019-07-03 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13736 )

Change subject: IMPALA-8681: Only show ValidWriteIdLists for Acid tables
..


Patch Set 4:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/3817/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/13736
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifcc31c7ddcfc471b0e5308f7e4aaadfa8189a905
Gerrit-Change-Number: 13736
Gerrit-PatchSet: 4
Gerrit-Owner: Yongzhi Chen 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Reviewer: Yongzhi Chen 
Gerrit-Reviewer: Zoltan Borok-Nagy 
Gerrit-Comment-Date: Wed, 03 Jul 2019 15:13:26 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8636: Implement INSERT for insert-only ACID tables

2019-07-03 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13559 )

Change subject: IMPALA-8636: Implement INSERT for insert-only ACID tables
..


Patch Set 11:

Build Failed

https://jenkins.impala.io/job/gerrit-code-review-checks/3816/ : Initial code 
review checks failed. See linked job for details on the failure.


--
To view, visit http://gerrit.cloudera.org:8080/13559
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id6c36fa6902676f06b4e38730f737becfc7c06ad
Gerrit-Change-Number: 13559
Gerrit-PatchSet: 11
Gerrit-Owner: Zoltan Borok-Nagy 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Reviewer: Zoltan Borok-Nagy 
Gerrit-Comment-Date: Wed, 03 Jul 2019 15:07:49 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8681: Only show ValidWriteIdLists for Acid tables

2019-07-03 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13736 )

Change subject: IMPALA-8681: Only show ValidWriteIdLists for Acid tables
..


Patch Set 4:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4578/ 
DRY_RUN=false


--
To view, visit http://gerrit.cloudera.org:8080/13736
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifcc31c7ddcfc471b0e5308f7e4aaadfa8189a905
Gerrit-Change-Number: 13736
Gerrit-PatchSet: 4
Gerrit-Owner: Yongzhi Chen 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Reviewer: Yongzhi Chen 
Gerrit-Reviewer: Zoltan Borok-Nagy 
Gerrit-Comment-Date: Wed, 03 Jul 2019 15:01:58 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8681: Only show ValidWriteIdLists for Acid tables

Csaba Ringhofer has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13736 )

Change subject: IMPALA-8681: Only show ValidWriteIdLists for Acid tables
..


Patch Set 4: Code-Review+2

Thanks for the changes!


--
To view, visit http://gerrit.cloudera.org:8080/13736
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifcc31c7ddcfc471b0e5308f7e4aaadfa8189a905
Gerrit-Change-Number: 13736
Gerrit-PatchSet: 4
Gerrit-Owner: Yongzhi Chen 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Reviewer: Yongzhi Chen 
Gerrit-Reviewer: Zoltan Borok-Nagy 
Gerrit-Comment-Date: Wed, 03 Jul 2019 14:47:47 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8709: Add Damerau-Levenshtein edit distance built-in function

Csaba Ringhofer has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13794 )

Change subject: IMPALA-8709: Add Damerau-Levenshtein edit distance built-in 
function
..


Patch Set 1:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/13794/1/be/src/exprs/expr-test.cc
File be/src/exprs/expr-test.cc:

http://gerrit.cloudera.org:8080/#/c/13794/1/be/src/exprs/expr-test.cc@4003
PS1, Line 4003:   TestErrorString("damerau_levenshtein('z', repeat('x', 256))",
  :   "damerau-levenshtein argument exceeds maximum length of 
255 characters\n");
  :   TestErrorString("damerau_levenshtein(repeat('x', 256), 'z')",
  :   "damerau-levenshtein argument exceeds maximum length of 
255 characters\n");
It could be useful to test the 255*255 case to ensure that there are no 
allocation issues.


http://gerrit.cloudera.org:8080/#/c/13794/1/be/src/exprs/string-functions-ir.cc
File be/src/exprs/string-functions-ir.cc:

http://gerrit.cloudera.org:8080/#/c/13794/1/be/src/exprs/string-functions-ir.cc@1177
PS1, Line 1177:   int d[s1len + 1][s2len + 1];
This worries me a bit:
1. The s1len > 255 || s2len > 255 does not protect from the allocation itself, 
so it is possible that a call with huge strings will lead to stack overflow. A 
solution that seems safe enough to me is creating an inner function that is 
called after the check and declare / use the array there,

2. Even if we exclude the s1len > 255 || s2len > 255 case, 64K on the stack 
seems too much to me, StringFunctions::Levenshtein uses dynamic allocation for 
much shorter arrays.



--
To view, visit http://gerrit.cloudera.org:8080/13794
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib759817ec15e7075bf49d51e494e45c8af4db94d
Gerrit-Change-Number: 13794
Gerrit-PatchSet: 1
Gerrit-Owner: Greg Rahn 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Zoltan Borok-Nagy 
Gerrit-Comment-Date: Wed, 03 Jul 2019 14:42:09 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-8681: Only show ValidWriteIdLists for Acid tables

Yongzhi Chen has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13736 )

Change subject: IMPALA-8681: Only show ValidWriteIdLists for Acid tables
..


Patch Set 4:

(1 comment)

The comment for skipping is added.

http://gerrit.cloudera.org:8080/#/c/13736/2/tests/query_test/test_acid.py
File tests/query_test/test_acid.py:

http://gerrit.cloudera.org:8080/#/c/13736/2/tests/query_test/test_acid.py@55
PS2, Line 55:   # that Catalog loads tables by itself, the q
> Can you add this information in a comment?
Done



--
To view, visit http://gerrit.cloudera.org:8080/13736
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifcc31c7ddcfc471b0e5308f7e4aaadfa8189a905
Gerrit-Change-Number: 13736
Gerrit-PatchSet: 4
Gerrit-Owner: Yongzhi Chen 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Reviewer: Yongzhi Chen 
Gerrit-Reviewer: Zoltan Borok-Nagy 
Gerrit-Comment-Date: Wed, 03 Jul 2019 14:34:31 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-8681: Only show ValidWriteIdLists for Acid tables

Hello Zoltan Borok-Nagy, Todd Lipcon, Csaba Ringhofer, Impala Public Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/13736

to look at the new patch set (#4).

Change subject: IMPALA-8681: Only show ValidWriteIdLists for Acid tables
..

IMPALA-8681: Only show ValidWriteIdLists for Acid tables

Lists ValidWriteIds for transactional tables in profile.
If a query does not trigger any transactional table loading,
the query profile will not have the "Loaded ValidWriteIdLists"
timeline.

Tests:
Manual tests.
Fixed StmtMetadataLoaderTest.
Added acid_profile test

Sample output:
Query Compilation: 3s525ms
   - Metadata load started: 37.369ms (37.369ms)
   - Metadata load finished. loaded-tables=1/1 ...
   - Loaded ValidWriteIdLists for transactional tables:
   functional.insert_only_transactional_table:0:9223372036854775807::
 : 3s312ms (551.463us)
   - Analysis finished: 3s370ms (58.110ms)
   ...

Change-Id: Ifcc31c7ddcfc471b0e5308f7e4aaadfa8189a905
---
M fe/src/main/java/org/apache/impala/analysis/StmtMetadataLoader.java
M fe/src/test/java/org/apache/impala/analysis/StmtMetadataLoaderTest.java
A testdata/workloads/functional-query/queries/QueryTest/acid-profile.test
M tests/common/skip.py
M tests/query_test/test_acid.py
5 files changed, 46 insertions(+), 10 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/36/13736/4
--
To view, visit http://gerrit.cloudera.org:8080/13736
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ifcc31c7ddcfc471b0e5308f7e4aaadfa8189a905
Gerrit-Change-Number: 13736
Gerrit-PatchSet: 4
Gerrit-Owner: Yongzhi Chen 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Reviewer: Yongzhi Chen 
Gerrit-Reviewer: Zoltan Borok-Nagy

[Impala-ASF-CR] IMPALA-8636: Implement INSERT for insert-only ACID tables

Zoltan Borok-Nagy has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13559 )

Change subject: IMPALA-8636: Implement INSERT for insert-only ACID tables
..


Patch Set 10:

(13 comments)

http://gerrit.cloudera.org:8080/#/c/13559/10/be/src/exec/hdfs-table-sink.h
File be/src/exec/hdfs-table-sink.h:

http://gerrit.cloudera.org:8080/#/c/13559/10/be/src/exec/hdfs-table-sink.h@264
PS10, Line 264: long
> int64_t?
Done


http://gerrit.cloudera.org:8080/#/c/13559/10/be/src/exec/hdfs-table-sink.cc
File be/src/exec/hdfs-table-sink.cc:

http://gerrit.cloudera.org:8080/#/c/13559/10/be/src/exec/hdfs-table-sink.cc@500
PS10, Line 500:   // However, for transactional tables we should create a new 
empty base directory.
> Why? I assume there is some good reason but it's not immediately obvious to
Sorry, I left out some information. We only need to do that in case of INSERT 
OVERWRITEs with empty result sets. For insert-only ACID tables we don't delete 
old files, we just create a new base directory.

Extended the comment and modified the condition.


http://gerrit.cloudera.org:8080/#/c/13559/10/be/src/exec/hdfs-table-sink.cc@700
PS10, Line 700:   if (IsTransactional()) return true;
> This one seems more obvious to me, but makes me think that the class commen
Extended the class comment.


http://gerrit.cloudera.org:8080/#/c/13559/10/be/src/runtime/coordinator.cc
File be/src/runtime/coordinator.cc:

http://gerrit.cloudera.org:8080/#/c/13559/10/be/src/runtime/coordinator.cc@571
PS10, Line 571: Status Coordinator::FinalizeHdfsInsert() {
> I think we should probably do the transaction abort in this function, since
Thanks, done.


http://gerrit.cloudera.org:8080/#/c/13559/10/be/src/runtime/coordinator.cc@571
PS10, Line 571: FinalizeHdfsInsert
> We should maybe rename this to FinalizeHdfsDml(), now or later.
Done


http://gerrit.cloudera.org:8080/#/c/13559/10/be/src/service/client-request-state.h
File be/src/service/client-request-state.h:

http://gerrit.cloudera.org:8080/#/c/13559/10/be/src/service/client-request-state.h@307
PS10, Line 307:   /// True if there is an open transaction.
> Hive ACID transaction, just to be clear.
Since then removed this bool because it caused more issues than solved because 
the handover of the transaction ownership between the FE and BE should happen 
immediately.

Now I'm using a member of FinalizeParams to detect whether we are in a 
transaction or not although it might not be the cleanest solution.


http://gerrit.cloudera.org:8080/#/c/13559/10/be/src/service/client-request-state.h@308
PS10, Line 308:   bool in_transaction_ = false;
> I think this would best fit in DmlExecState.
We'll have transactions for queries as well.


http://gerrit.cloudera.org:8080/#/c/13559/10/be/src/service/client-request-state.cc
File be/src/service/client-request-state.cc:

http://gerrit.cloudera.org:8080/#/c/13559/10/be/src/service/client-request-state.cc@720
PS10, Line 720: DCHECK(exec_request().__isset.transaction_id);
> I think we would prefer to abort the transaction earlier in the query lifec
Thanks for the suggestion, moved it to FinalizeHdfsDml()


http://gerrit.cloudera.org:8080/#/c/13559/10/be/src/service/client-request-state.cc@820
PS10, Line 820:   //TODO: HMS is not updated at this point, only in 
UpdateCatalog(). But
> Zoli explained this to me out-of-band. I think I agree that having the cata
Added commit to catalog. Was also thinking about adding abort as well, but the 
error handling in CatalogOpExecutor.UpdateCatalog() was inconsistent (sometimes 
an exception is thrown, sometimes we set an error in the result object), and in 
some cases we still need to abort the transaction from the BE.


http://gerrit.cloudera.org:8080/#/c/13559/10/be/src/service/frontend.h
File be/src/service/frontend.h:

http://gerrit.cloudera.org:8080/#/c/13559/10/be/src/service/frontend.h@170
PS10, Line 170:   /// Commits transaction with the given transaction id.
> Are there any invariants we should document for these methods? I guess we'r
Removed it since now we commit through the Catalog.


http://gerrit.cloudera.org:8080/#/c/13559/10/be/src/service/frontend.h@171
PS10, Line 171: long
> I think we want to use int64_t, to match the thrift definition and our styl
Done. I think I got used to using longs in Java code.


http://gerrit.cloudera.org:8080/#/c/13559/10/fe/src/main/java/org/apache/impala/analysis/InsertStmt.java
File fe/src/main/java/org/apache/impala/analysis/InsertStmt.java:

http://gerrit.cloudera.org:8080/#/c/13559/10/fe/src/main/java/org/apache/impala/analysis/InsertStmt.java@179
PS10, Line 179:   private Long writeId_;
> This is just a nit, but it would be slightly more intuitive to me to use -1
Done


http://gerrit.cloudera.org:8080/#/c/13559/10/testdata/workloads/functional-query/queries/QueryTest/acid-insert.test
File testdata/workloads/functional-query/queries/QueryTest/acid-insert.test:

[Impala-ASF-CR] IMPALA-8636: Implement INSERT for insert-only ACID tables

Hello Csaba Ringhofer, Todd Lipcon, Tim Armstrong, Impala Public Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/13559

to look at the new patch set (#11).

Change subject: IMPALA-8636: Implement INSERT for insert-only ACID tables
..

IMPALA-8636: Implement INSERT for insert-only ACID tables

This commit adds INSERT support for insert-only ACID tables.

The Frontend opens a transaction for queries that refer to
transactional tables. For INSERT statements that write insert-only
ACID tables it also allocates a write ID. The Frontend aborts the
transaction if an error occurs during analysis/planning.

The Backend gets the transaction id in TExecRequestState and the
write id is set for the HDFS table sinks. The sinks write the files
at their final destination which is an ACID base/delta directory.
There is no need for finalization of transactional INSERTS.

ClientRequestState commits the transaction in WaitInternal() if
everything went well. If the transaction is still open in Done(), it
means there was an error, therefore the transaction needs to be aborted.

The Backend commits/aborts the transaction by calling the Frontend via
JNI.

Testing:
* added new tables during dataload
* added acid-insert.test file with INSERT statements against the new
  tables
* added integration test with Hive to test_hms_integration.py. The test
  inserts data with Impala and reads with Hive. (These integration
  tests only run with exhaustive exploration strategy)

TODO in following commits:
* add locks and heartbeats
* implement TRUNCATE (maybe in another commit)
* CTAS creates files in the 'root' directory of the table/partition. It
  is handled correctly during SELECT, but would be better to create a
  base directory from the beginning.

Change-Id: Id6c36fa6902676f06b4e38730f737becfc7c06ad
---
M be/src/exec/hdfs-table-sink.cc
M be/src/exec/hdfs-table-sink.h
M be/src/runtime/coordinator.cc
M be/src/runtime/coordinator.h
M be/src/service/client-request-state.cc
M be/src/service/client-request-state.h
M be/src/service/frontend.cc
M be/src/service/frontend.h
M be/src/util/jni-util.h
M common/thrift/CatalogService.thrift
M common/thrift/DataSinks.thrift
M common/thrift/Frontend.thrift
M common/thrift/ImpalaInternalService.thrift
M fe/src/compat-hive-2/java/org/apache/impala/compat/MetastoreShim.java
M fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java
M fe/src/main/java/org/apache/impala/analysis/InsertStmt.java
M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java
A fe/src/main/java/org/apache/impala/common/TransactionException.java
M fe/src/main/java/org/apache/impala/planner/HdfsTableSink.java
M fe/src/main/java/org/apache/impala/planner/TableSink.java
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
M fe/src/main/java/org/apache/impala/service/Frontend.java
M fe/src/main/java/org/apache/impala/service/JniFrontend.java
M testdata/datasets/functional/functional_schema_template.sql
M testdata/datasets/functional/schema_constraints.csv
A testdata/workloads/functional-query/queries/QueryTest/acid-insert.test
M tests/metadata/test_hms_integration.py
M tests/query_test/test_insert.py
28 files changed, 765 insertions(+), 126 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/59/13559/11
--
To view, visit http://gerrit.cloudera.org:8080/13559
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Id6c36fa6902676f06b4e38730f737becfc7c06ad
Gerrit-Change-Number: 13559
Gerrit-PatchSet: 11
Gerrit-Owner: Zoltan Borok-Nagy 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Reviewer: Zoltan Borok-Nagy

[Impala-ASF-CR] IMPALA-8681: Only show ValidWriteIdLists for Acid tables

Csaba Ringhofer has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13736 )

Change subject: IMPALA-8681: Only show ValidWriteIdLists for Acid tables
..


Patch Set 2:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/13736/2/tests/query_test/test_acid.py
File tests/query_test/test_acid.py:

http://gerrit.cloudera.org:8080/#/c/13736/2/tests/query_test/test_acid.py@55
PS2, Line 55:   @SkipIfCatalogV2.hms_event_polling_enabled()
> When local catalogV2 combines with hms_enent_polling enabled, it seems cata
Can you add this information in a comment?



--
To view, visit http://gerrit.cloudera.org:8080/13736
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifcc31c7ddcfc471b0e5308f7e4aaadfa8189a905
Gerrit-Change-Number: 13736
Gerrit-PatchSet: 2
Gerrit-Owner: Yongzhi Chen 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Reviewer: Yongzhi Chen 
Gerrit-Reviewer: Zoltan Borok-Nagy 
Gerrit-Comment-Date: Wed, 03 Jul 2019 13:09:48 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-8703: ISO:SQL:2016 datetime patterns - Milestone 1

2019-07-03 Thread Attila Jeges (Code Review)

Attila Jeges has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13722 )

Change subject: IMPALA-8703: ISO:SQL:2016 datetime patterns - Milestone 1
..


Patch Set 2:

(3 comments)

http://gerrit.cloudera.org:8080/#/c/13722/2/be/src/runtime/datetime-iso-sql-format-parser.h
File be/src/runtime/datetime-iso-sql-format-parser.h:

http://gerrit.cloudera.org:8080/#/c/13722/2/be/src/runtime/datetime-iso-sql-format-parser.h@39
PS2, Line 39: ParseDateTime
I could be mistaken but I couldn't find any BE-tests for this function.


http://gerrit.cloudera.org:8080/#/c/13722/2/be/src/runtime/datetime-iso-sql-format-tokenizer.h
File be/src/runtime/datetime-iso-sql-format-tokenizer.h:

http://gerrit.cloudera.org:8080/#/c/13722/2/be/src/runtime/datetime-iso-sql-format-tokenizer.h@51
PS2, Line 51: Tokenize
I could be mistaken but I couldn't find any BE-tests for this and the other 
functions in this class.


http://gerrit.cloudera.org:8080/#/c/13722/2/be/src/runtime/datetime-simple-date-format-parser.h
File be/src/runtime/datetime-simple-date-format-parser.h:

http://gerrit.cloudera.org:8080/#/c/13722/2/be/src/runtime/datetime-simple-date-format-parser.h@68
PS2, Line 68: /// Constants to hold default format lengths.
Maybe the stuff in this file could be moved to separate classes, like 
SimpleDateFormatParser and SimpleDateFormatTokenizer ? I think that naming 
would make the distinction between code dealing with simple-date format and 
code dealing with iso-sql format cleaner.



--
To view, visit http://gerrit.cloudera.org:8080/13722
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I19d8d097a45ae6f103b6cd1b2d81aad38dfd9e23
Gerrit-Change-Number: 13722
Gerrit-PatchSet: 2
Gerrit-Owner: Gabor Kaszab 
Gerrit-Reviewer: Attila Jeges 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Wed, 03 Jul 2019 13:04:39 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-8593: Prohibit write operations for bucketed tables

Csaba Ringhofer has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13558 )

Change subject: IMPALA-8593: Prohibit write operations for bucketed tables
..


Patch Set 4: Code-Review+2

I give +2, but I would prefer to run 
https://jenkins.impala.io/job/ubuntu-16.04-from-scratch-cdp-hive/ with the 
change before merging it. The cdp build is still red, but there are only a few 
( 3 ) errors left on master.


--
To view, visit http://gerrit.cloudera.org:8080/13558
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia08d01168660830b6e0d08b55a95eac129889cec
Gerrit-Change-Number: 13558
Gerrit-PatchSet: 4
Gerrit-Owner: Yongzhi Chen 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Sahil Takiar 
Gerrit-Reviewer: Sudhanshu Arora 
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Reviewer: Vihang Karajgaonkar 
Gerrit-Reviewer: Yongzhi Chen 
Gerrit-Reviewer: Zoltan Borok-Nagy 
Gerrit-Comment-Date: Wed, 03 Jul 2019 12:56:57 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8703: ISO:SQL:2016 datetime patterns - Milestone 1

2019-07-03 Thread Attila Jeges (Code Review)

Attila Jeges has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13722 )

Change subject: IMPALA-8703: ISO:SQL:2016 datetime patterns - Milestone 1
..


Patch Set 2:

(7 comments)

http://gerrit.cloudera.org:8080/#/c/13722/2/fe/src/main/java/org/apache/impala/analysis/CastExpr.java
File fe/src/main/java/org/apache/impala/analysis/CastExpr.java:

http://gerrit.cloudera.org:8080/#/c/13722/2/fe/src/main/java/org/apache/impala/analysis/CastExpr.java@51
PS2, Line 51: private final String castFormat_;
Add a comment to describe the new member.


http://gerrit.cloudera.org:8080/#/c/13722/2/fe/src/main/java/org/apache/impala/analysis/CastExpr.java@208
PS2, Line 208: castFormat_
What if 'castFormat_' includes an apostrophe?


http://gerrit.cloudera.org:8080/#/c/13722/2/fe/src/main/java/org/apache/impala/analysis/CastExpr.java@226
PS2, Line 226: boolean isIntermediateStep =
 : type_.getPrimitiveType() == PrimitiveType.CHAR &&
 : children_.get(0).getType().getPrimitiveType() == 
PrimitiveType.STRING;
The naming of this variable is a bit confusing. 'isIntermediateStep' is set to 
true even if we have a simple CAST(string_col AS char(20)) expression.

I would just put the right-hand side directly into the if condition.


http://gerrit.cloudera.org:8080/#/c/13722/2/fe/src/main/java/org/apache/impala/analysis/CastExpr.java@280
PS2, Line 280: if (null != castFormat_ && !isIntermediateStepNeeded) {
 :   if (!(type_.isDateOrTimeType() && 
getChild(0).getType().isStringType()) &&
 :   !(type_.isStringType() && 
getChild(0).getType().isDateOrTimeType())) {
 : // FORMAT clause works only for casting between date 
types and string types
 : throw new AnalysisException("FORMAT clause is not 
applicable from " +
 : getChild(0).getType() + " to " + type_);
 :   }
 :   if (castFormat_.isEmpty()) {
 : throw new AnalysisException("FORMAT clause can't be 
empty");
 :   }
 : }
This block can be used as the else branch of he previous 'if' statement. And 
then you don't need to introduce the 'isIntermediateStepNeeded' variable.


http://gerrit.cloudera.org:8080/#/c/13722/2/testdata/workloads/functional-query/queries/QueryTest/date.test
File testdata/workloads/functional-query/queries/QueryTest/date.test:

http://gerrit.cloudera.org:8080/#/c/13722/2/testdata/workloads/functional-query/queries/QueryTest/date.test@629
PS2, Line 629: #
 : # QUERY
 : #select cast("2014/-; ,11'/:05" as date format "-MM-DD");
 : # RESULTS
 : #2014-11-05
 : # TYPES
 : #DATE
This should work right? Any reason it is commented out?


http://gerrit.cloudera.org:8080/#/c/13722/2/testdata/workloads/functional-query/queries/QueryTest/date.test@841
PS2, Line 841:
Extra space


http://gerrit.cloudera.org:8080/#/c/13722/2/tests/query_test/test_cast_with_format.py
File tests/query_test/test_cast_with_format.py:

http://gerrit.cloudera.org:8080/#/c/13722/2/tests/query_test/test_cast_with_format.py@22
PS2, Line 22: TestCastWithFormat
Could the tests in this script be moved to cast_format.test? Or to another 
'.test' file?



--
To view, visit http://gerrit.cloudera.org:8080/13722
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I19d8d097a45ae6f103b6cd1b2d81aad38dfd9e23
Gerrit-Change-Number: 13722
Gerrit-PatchSet: 2
Gerrit-Owner: Gabor Kaszab 
Gerrit-Reviewer: Attila Jeges 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Wed, 03 Jul 2019 12:50:07 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-8709: Add Damerau-Levenshtein edit distance built-in function

Zoltan Borok-Nagy has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13794 )

Change subject: IMPALA-8709: Add Damerau-Levenshtein edit distance built-in 
function
..


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/13794/1/be/src/exprs/expr-test.cc
File be/src/exprs/expr-test.cc:

http://gerrit.cloudera.org:8080/#/c/13794/1/be/src/exprs/expr-test.cc@3999
PS1, Line 3999:   TestValue("damerau_levenshtein('CA', 'ABC')", TYPE_INT, 3);
Based on 
https://en.wikipedia.org/wiki/Damerau%E2%80%93Levenshtein_distance#Algorithm 
the used algorithm computes "optimal string alignment distance" or "restricted 
edit distance" which is not the "true" Damerau–Levenshtein distance because 
that should return 2 in this case (CA -> AC -> ABC).

I'm not sure how to resolve it, I think you know it best, but I think at least 
some comments needed to clarify the semantics.



--
To view, visit http://gerrit.cloudera.org:8080/13794
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib759817ec15e7075bf49d51e494e45c8af4db94d
Gerrit-Change-Number: 13794
Gerrit-PatchSet: 1
Gerrit-Owner: Greg Rahn 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Zoltan Borok-Nagy 
Gerrit-Comment-Date: Wed, 03 Jul 2019 12:37:17 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-8593: Prohibit write operations for bucketed tables

Yongzhi Chen has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13558 )

Change subject: IMPALA-8593: Prohibit write operations for bucketed tables
..


Patch Set 4:

Hive will treat all acid table as managed and all managed are needed to be 
transactional. Impala created tables will be external tables. The changing and 
discussion is still going on. This patch is just first step, we can make more
fixes later.

Following are related docs for HMS translation layer:
Hive:
https://docs.google.com/document/d/1ZmtMlKvnSz-EFGuCuYfPVA-2kfgHOWenL8sW47vkhnw/edit#
Impala:
https://docs.google.com/document/d/1lgcD_ZqIFVydLdRPQhkYBdFu2C0mH1A8VnXpcSs9psA/edit#heading=h.l9b51051hhma

Three engines:
https://docs.google.com/document/d/1fPIn1WW-tRRSmM-aGEVl0DI-TByIswo5HrLdgSPcw9s/edit?ts=5d1baa75


--
To view, visit http://gerrit.cloudera.org:8080/13558
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia08d01168660830b6e0d08b55a95eac129889cec
Gerrit-Change-Number: 13558
Gerrit-PatchSet: 4
Gerrit-Owner: Yongzhi Chen 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Sahil Takiar 
Gerrit-Reviewer: Sudhanshu Arora 
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Reviewer: Vihang Karajgaonkar 
Gerrit-Reviewer: Yongzhi Chen 
Gerrit-Reviewer: Zoltan Borok-Nagy 
Gerrit-Comment-Date: Wed, 03 Jul 2019 12:14:58 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8709: Add Damerau-Levenshtein edit distance built-in function

Zoltan Borok-Nagy has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13794 )

Change subject: IMPALA-8709: Add Damerau-Levenshtein edit distance built-in 
function
..


Patch Set 1: Code-Review+1

(2 comments)

Found some nits, but LGTM.

http://gerrit.cloudera.org:8080/#/c/13794/1/be/src/exprs/expr-test.cc
File be/src/exprs/expr-test.cc:

http://gerrit.cloudera.org:8080/#/c/13794/1/be/src/exprs/expr-test.cc@3984
PS1, Line 3984:   TestValue("damerau_levenshtein('', '')", TYPE_INT, 0);
  :   TestValue("damerau_levenshtein('abc', 'abc')", TYPE_INT, 0);
  :   TestValue("damerau_levenshtein('a', 'b')", TYPE_INT, 1);
  :   TestValue("damerau_levenshtein('a', '')", TYPE_INT, 1);
  :   TestValue("damerau_levenshtein('aabc', 'abc')", TYPE_INT, 1);
  :   TestValue("damerau_levenshtein('abcc', 'abc')", TYPE_INT, 1);
  :   TestValue("damerau_levenshtein('', 'a')", TYPE_INT, 1);
  :   TestValue("damerau_levenshtein('abc', 'abcc')", TYPE_INT, 1);
  :   TestValue("damerau_levenshtein('abc', 'aabc')", TYPE_INT, 1);
  :   TestValue("damerau_levenshtein('teh', 'the')", TYPE_INT, 1);
  :   TestValue("damerau_levenshtein('tets', 'test')", TYPE_INT, 1);
  :   TestValue("damerau_levenshtein('fuor', 'four')", TYPE_INT, 1);
  :   TestValue("damerau_levenshtein('kitten', 'sitting')", 
TYPE_INT, 3);
  :   TestValue("damerau_levenshtein('Saturday', 'Sunday')", 
TYPE_INT, 3);
  :   TestValue("damerau_levenshtein('rosettacode', 
'raisethysword')", TYPE_INT, 8);
  :   TestValue("damerau_levenshtein('CA', 'ABC')", TYPE_INT, 3);
  :   TestIsNull("damerau_levenshtein('foo', NULL)", TYPE_INT);
  :   TestIsNull("damerau_levenshtein(NULL, 'foo')", TYPE_INT);
  :   TestIsNull("damerau_levenshtein(NULL, NULL)", TYPE_INT);
  :   TestErrorString("damerau_levenshtein('z', repeat('x', 256))",
  :   "damerau-levenshtein argument exceeds maximum length of 
255 characters\n");
  :   TestErrorString("damerau_levenshtein(repeat('x', 256), 'z')",
  :   "damerau-levenshtein argument exceeds maximum length of 
255 characters\n");
  :
  :   TestValue("dle_dst('', '')", TYPE_INT, 0);
  :   TestValue("dle_dst('abc', 'abc')", TYPE_INT, 0);
  :   TestValue("dle_dst('a', 'b')", TYPE_INT, 1);
  :   TestValue("dle_dst('a', '')", TYPE_INT, 1);
  :   TestValue("dle_dst('aabc', 'abc')", TYPE_INT, 1);
  :   TestValue("dle_dst('abcc', 'abc')", TYPE_INT, 1);
  :   TestValue("dle_dst('', 'a')", TYPE_INT, 1);
  :   TestValue("dle_dst('abc', 'abcc')", TYPE_INT, 1);
  :   TestValue("dle_dst('abc', 'aabc')", TYPE_INT, 1);
  :   TestValue("dle_dst('teh', 'the')", TYPE_INT, 1);
  :   TestValue("dle_dst('tets', 'test')", TYPE_INT, 1);
  :   TestValue("dle_dst('fuor', 'four')", TYPE_INT, 1);
  :   TestValue("dle_dst('kitten', 'sitting')", TYPE_INT, 3);
  :   TestValue("dle_dst('Saturday', 'Sunday')", TYPE_INT, 3);
  :   TestValue("dle_dst('rosettacode', 'raisethysword')", 
TYPE_INT, 8);
  :   TestValue("dle_dst('CA', 'ABC')", TYPE_INT, 3);
  :   TestIsNull("dle_dst('foo', NULL)", TYPE_INT);
  :   TestIsNull("dle_dst(NULL, 'foo')", TYPE_INT);
  :   TestIsNull("dle_dst(NULL, NULL)", TYPE_INT);
  :   TestErrorString("dle_dst('z', repeat('x', 256))",
  :   "damerau-levenshtein argument exceeds maximum length of 
255 characters\n");
  :   TestErrorString("dle_dst(repeat('x', 256), 'z')",
  :   "damerau-levenshtein argument exceeds maximum length of 
255 characters\n");
nit: instead of duplicating maybe you could just write something like

 for (string fn_name : {"damerau_levenshtein", "dle_dst"}) {
   TestValue(fn_name + "('', '')" ...);
   ...
 }


http://gerrit.cloudera.org:8080/#/c/13794/1/be/src/exprs/string-functions-ir.cc
File be/src/exprs/string-functions-ir.cc:

http://gerrit.cloudera.org:8080/#/c/13794/1/be/src/exprs/string-functions-ir.cc@1218
PS1, Line 1218: d[i][j] = std::min(d[i][j],
  :d[i - 2][j - 2] + l_cost // 
transposition
  : );
nit: fits single line



--
To view, visit http://gerrit.cloudera.org:8080/13794
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib759817ec15e7075bf49d51e494e45c8af4db94d
Gerrit-Change-Number: 13794
Gerrit-PatchSet: 1
Gerrit-Owner: Greg Rahn 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Zoltan Borok-Nagy

[Impala-ASF-CR] IMPALA-8718: Fixed AnalysisException in inline view with outer join complex type column

2019-07-03 Thread Impala Public Jenkins (Code Review)

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13765 )

Change subject: IMPALA-8718: Fixed AnalysisException in inline view with outer 
join complex type column
..


Patch Set 4:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/3815/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/13765
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ida65503ca4b1342b8fe0049753bc664da227dca9
Gerrit-Change-Number: 13765
Gerrit-PatchSet: 4
Gerrit-Owner: Yongzhi Chen 
Gerrit-Reviewer: Bharath Vissapragada 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Sahil Takiar 
Gerrit-Reviewer: Yongzhi Chen 
Gerrit-Comment-Date: Wed, 03 Jul 2019 09:47:16 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8718: Fixed AnalysisException in inline view with outer join complex type column

Yongzhi Chen has posted comments on this change. (
http://gerrit.cloudera.org:8080/13765 )

Change subject: IMPALA-8718: Fixed AnalysisException in inline view with outer
join complex type column
..

Patch Set 3:

(5 comments)

Answer Sahil's question: IsNull Check may be necessary for other places, but
use IsNullPredicate in requiresNullWrapping is not required. We can workaround
it.

Attach patch 4 with fixed tests. Run all core tests.

http://gerrit.cloudera.org:8080/#/c/13765/3//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/13765/3//COMMIT_MSG@16
PS3, Line 16: cannot both be null and non-null. But during its
> This line is a bit confusing. Can you clarify what you mean here?
Although we do not have the predict to tell if a value is null, we know that a
valid should be either null or nonnull and cannot be both.

http://gerrit.cloudera.org:8080/#/c/13765/3//COMMIT_MSG@17
PS3, Line 17: checking whether the wrapper needed, because of code
> what do you mean by "because of code limitation"?
We do not have IsNullPredicate for complex type, it is mentioned in the latter
part of the message. I will make a change to add it here too.

http://gerrit.cloudera.org:8080/#/c/13765/3/testdata/workloads/functional-query/queries/QueryTest/complex_joins.test
File testdata/workloads/functional-query/queries/QueryTest/complex_joins.test:

http://gerrit.cloudera.org:8080/#/c/13765/3/testdata/workloads/functional-query/queries/QueryTest/complex_joins.test@5
PS3, Line 5: create table array_tbl (col1 string, col2 array ) stored
as parquet;
> databases such as "functional_parquet" already have tables such as "complex
This is the test case that the customer provided. And our functional table does
not have data.

http://gerrit.cloudera.org:8080/#/c/13765/3/testdata/workloads/functional-query/queries/QueryTest/complex_joins.test@35
PS3, Line 35: select t.col1, rank() over(order by t.col1) as rnk
> is the analytical function necessary to re-produce the error? otherwise I t
Yes, it is the customer's case, I must make sure the tests passed with/without
data. Why create two: The testing framework parallels the tests which make
test flaky if switching between impala and hive.

http://gerrit.cloudera.org:8080/#/c/13765/3/tests/query_test/test_complex.py
File tests/query_test/test_complex.py:

http://gerrit.cloudera.org:8080/#/c/13765/3/tests/query_test/test_complex.py@23
PS3, Line 23: class TestComplex(ImpalaTestSuite):
> You can just add this test to test_nested_types.py instead of creating a ne
I will move

--
To view, visit http://gerrit.cloudera.org:8080/13765
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ida65503ca4b1342b8fe0049753bc664da227dca9
Gerrit-Change-Number: 13765
Gerrit-PatchSet: 3
Gerrit-Owner: Yongzhi Chen
Gerrit-Reviewer: Bharath Vissapragada
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Sahil Takiar
Gerrit-Reviewer: Yongzhi Chen
Gerrit-Comment-Date: Wed, 03 Jul 2019 09:11:42 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-8718: Fixed AnalysisException in inline view with outer join complex type column