[jira] [Commented] (HIVE-20233) Review Operator.java
[
https://issues.apache.org/jira/browse/HIVE-20233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749650#comment-16749650
]
Hive QA commented on HIVE-20233:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12955888/HIVE-20233.8.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15701 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15747/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15747/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15747/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12955888 - PreCommit-HIVE-Build
> Review Operator.java
>
>
> Key: HIVE-20233
> URL: https://issues.apache.org/jira/browse/HIVE-20233
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Affects Versions: 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HIVE-20233.1.patch, HIVE-20233.2.patch,
> HIVE-20233.3.patch, HIVE-20233.4.patch, HIVE-20233.4.patch,
> HIVE-20233.5.patch, HIVE-20233.6.patch, HIVE-20233.7.patch, HIVE-20233.8.patch
>
>
> Various improvements to {{Operator.java}}
>
> * Code formatting
> * Remove superfluous logging guards
> * Add logging message parameters
> * Minor performance improvements
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21115) Add support for object versions in metastore
[
https://issues.apache.org/jira/browse/HIVE-21115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749653#comment-16749653
]
Hive QA commented on HIVE-21115:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12955902/HIVE-21115.1.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15748/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15748/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15748/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2019-01-23 08:23:17.264
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-15748/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2019-01-23 08:23:17.267
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at dfd63d9 HIVE-20776 : Run HMS filterHooks on server-side in
addition to client-side (Na Li reviewed by Karthik, Sergio, Morio, Adam and
Vihang Karajgaonkar)
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at dfd63d9 HIVE-20776 : Run HMS filterHooks on server-side in
addition to client-side (Na Li reviewed by Karthik, Sergio, Morio, Adam and
Vihang Karajgaonkar)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2019-01-23 08:23:17.917
+ rm -rf ../yetus_PreCommit-HIVE-Build-15748
+ mkdir ../yetus_PreCommit-HIVE-Build-15748
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-15748
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-15748/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error: patch failed:
standalone-metastore/metastore-server/src/main/sql/derby/upgrade-3.2.0-to-4.0.0.derby.sql:8
Falling back to three-way merge...
Applied patch to
'standalone-metastore/metastore-server/src/main/sql/derby/upgrade-3.2.0-to-4.0.0.derby.sql'
with conflicts.
Going to apply patch with: git apply -p0
/data/hiveptest/working/scratch/build.patch:81: trailing whitespace.
tmpMap.put(_Fields.VERSION, new
org.apache.thrift.meta_data.FieldMetaData("version",
org.apache.thrift.TFieldRequirementType.OPTIONAL,
/data/hiveptest/working/scratch/build.patch:241: trailing whitespace.
} else {
/data/hiveptest/working/scratch/build.patch:355: trailing whitespace.
tmpMap.put(_Fields.VERSION, new
org.apache.thrift.meta_data.FieldMetaData("version",
org.apache.thrift.TFieldRequirementType.OPTIONAL,
/data/hiveptest/working/scratch/build.patch:515: trailing whitespace.
} else {
error: patch failed:
standalone-metastore/metastore-server/src/main/sql/derby/upgrade-3.2.0-to-4.0.0.derby.sql:8
Falling back to three-way merge...
Applied patch to
'standalone-metastore/metastore-server/src/main/sql/derby/upgrade-3.2.0-to-4.0.0.derby.sql'
with conflicts.
U
standalone-metastore/metastore-server/src/main/sql/derby/upgrade-3.2.0-to-4.0.0.derby.sql
warning: 4 lines add whitespace errors.
+ result=1
+ '[' 1 -ne 0 ']'
+ rm -rf yetus_PreCommit-HIVE-Build-15748
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12955902 - PreCommit-HIVE-Build
> Add support for object versions in metastore
>
>
> Key: HIVE-21115
> URL: https://issues.apache.org/jira/browse/HIVE-21115
> Project: Hive
> Issue Type: Improvement
>Reporter: Vihang Karajgaonkar
>Assignee:
[jira] [Updated] (HIVE-21143) Add rewrite rules to open/close Between operators
[ https://issues.apache.org/jira/browse/HIVE-21143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-21143: Attachment: HIVE-21143.03.patch > Add rewrite rules to open/close Between operators > - > > Key: HIVE-21143 > URL: https://issues.apache.org/jira/browse/HIVE-21143 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-21143.01.patch, HIVE-21143.02.patch, > HIVE-21143.03.patch > > > During query compilation it's better to have BETWEEN statements in open form, > as Calcite current not considering them during simplification. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20758) Constraints: Show create table does not show constraints
[
https://issues.apache.org/jira/browse/HIVE-20758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750076#comment-16750076
]
Hive QA commented on HIVE-20758:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12955959/HIVE-20758.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 13 failed/errored test(s), 15707 tests
executed
*Failed tests:*
{noformat}
TestReplicationScenariosIncrementalLoadAcidTables - did not produce a
TEST-*.xml file (likely timed out) (batchId=251)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[cbo_rp_limit]
(batchId=173)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[schemeAuthority]
(batchId=191)
org.apache.hive.jdbc.TestJdbcGenericUDTFGetSplits.testGenericUDTFOrderBySplitCount1
(batchId=261)
org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testCancelRenewTokenFlow
(batchId=276)
org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testConnection
(batchId=276)
org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testIsValid (batchId=276)
org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testIsValidNeg
(batchId=276)
org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testNegativeProxyAuth
(batchId=276)
org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testNegativeTokenAuth
(batchId=276)
org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testProxyAuth
(batchId=276)
org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testRenewDelegationToken
(batchId=276)
org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testTokenAuth
(batchId=276)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15758/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15758/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15758/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 13 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12955959 - PreCommit-HIVE-Build
> Constraints: Show create table does not show constraints
>
>
> Key: HIVE-20758
> URL: https://issues.apache.org/jira/browse/HIVE-20758
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0
>Reporter: Gopal V
>Assignee: Barnabas Maidics
>Priority: Major
> Attachments: HIVE-20758.1.patch, Screen Shot 2019-01-23 at
> 11.52.04.png
>
>
> Even though the desc formatted shows the constraints, the show create table
> does not
> {code}
> | # Primary Key | NULL
> | NULL
>|
> | Table: |
> tpcds_bin_partitioned_orc_1.inventory | NULL
> |
> | Constraint Name: | pk_in
> | NULL
>|
> | Column Names: | inv_date_sk
> | inv_item_sk
>|
> || NULL
> | NULL
>|
> | # Foreign Keys | NULL
> | NULL
>|
> | Table: |
> tpcds_bin_partitioned_orc_1.inventory | NULL
> |
> | Constraint Name: | inv_d
> | NULL
>|
> | Parent Column Name:tpcds_bin_partitioned_orc_1.date_dim.d_date_sk |
> Column Name:inv_date_sk| Key Sequence:1
> |
> || NULL
> | NULL
>|
> | Constraint Name: | inv_i
> | NULL
>|
> | Parent Column
[jira] [Updated] (HIVE-17084) Turn on hive.stats.fetch.column.stats configuration flag
[ https://issues.apache.org/jira/browse/HIVE-17084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-17084: Resolution: Fixed Fix Version/s: (was: 3.2.0) 4.0.0 Status: Resolved (was: Patch Available) pushed to master. Thank you Ashutosh for reviewing the changes! > Turn on hive.stats.fetch.column.stats configuration flag > > > Key: HIVE-17084 > URL: https://issues.apache.org/jira/browse/HIVE-17084 > Project: Hive > Issue Type: Task > Components: Statistics >Reporter: Vineet Garg >Assignee: Zoltan Haindrich >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-17084.08.patch, HIVE-17084.09.patch, > HIVE-17084.1.patch, HIVE-17084.10.patch, HIVE-17084.11.patch, > HIVE-17084.12.patch, HIVE-17084.13.patch, HIVE-17084.14.patch, > HIVE-17084.15.patch, HIVE-17084.16.patch, HIVE-17084.17.patch, > HIVE-17084.18.patch, HIVE-17084.19.patch, HIVE-17084.20.patch, > HIVE-17084.21.patch, HIVE-17084.22.patch, HIVE-17084.23.patch, > HIVE-17084.24.patch, HIVE-17084.25.patch, HIVE-17084.26.patch, > HIVE-17084.27.patch, HIVE-17084.28.patch, HIVE-17084.29.patch, > HIVE-17084.30.patch, HIVE-17084.31.patch, HIVE-17084.31.patch, > HIVE-17084.31.patch, HIVE-170884.4.patch, HIVE-170884.5.patch, > HIVE-170884.7.patch > > > This flag is off by default and could result in bad plans due to missing > column statistics. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21143) Add rewrite rules to open/close Between operators
[ https://issues.apache.org/jira/browse/HIVE-21143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-21143: Attachment: HIVE-21143.03.patch > Add rewrite rules to open/close Between operators > - > > Key: HIVE-21143 > URL: https://issues.apache.org/jira/browse/HIVE-21143 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-21143.01.patch, HIVE-21143.02.patch, > HIVE-21143.03.patch, HIVE-21143.03.patch, HIVE-21143.04.patch > > > During query compilation it's better to have BETWEEN statements in open form, > as Calcite current not considering them during simplification. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21143) Add rewrite rules to open/close Between operators
[ https://issues.apache.org/jira/browse/HIVE-21143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-21143: Attachment: HIVE-21143.04.patch > Add rewrite rules to open/close Between operators > - > > Key: HIVE-21143 > URL: https://issues.apache.org/jira/browse/HIVE-21143 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-21143.01.patch, HIVE-21143.02.patch, > HIVE-21143.03.patch, HIVE-21143.03.patch, HIVE-21143.04.patch > > > During query compilation it's better to have BETWEEN statements in open form, > as Calcite current not considering them during simplification. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-17084) Turn on hive.stats.fetch.column.stats configuration flag
[ https://issues.apache.org/jira/browse/HIVE-17084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750161#comment-16750161 ] Ashutosh Chauhan commented on HIVE-17084: - +1 > Turn on hive.stats.fetch.column.stats configuration flag > > > Key: HIVE-17084 > URL: https://issues.apache.org/jira/browse/HIVE-17084 > Project: Hive > Issue Type: Task > Components: Statistics >Reporter: Vineet Garg >Assignee: Zoltan Haindrich >Priority: Major > Fix For: 3.2.0 > > Attachments: HIVE-17084.08.patch, HIVE-17084.09.patch, > HIVE-17084.1.patch, HIVE-17084.10.patch, HIVE-17084.11.patch, > HIVE-17084.12.patch, HIVE-17084.13.patch, HIVE-17084.14.patch, > HIVE-17084.15.patch, HIVE-17084.16.patch, HIVE-17084.17.patch, > HIVE-17084.18.patch, HIVE-17084.19.patch, HIVE-17084.20.patch, > HIVE-17084.21.patch, HIVE-17084.22.patch, HIVE-17084.23.patch, > HIVE-17084.24.patch, HIVE-17084.25.patch, HIVE-17084.26.patch, > HIVE-17084.27.patch, HIVE-17084.28.patch, HIVE-17084.29.patch, > HIVE-17084.30.patch, HIVE-17084.31.patch, HIVE-17084.31.patch, > HIVE-17084.31.patch, HIVE-170884.4.patch, HIVE-170884.5.patch, > HIVE-170884.7.patch > > > This flag is off by default and could result in bad plans due to missing > column statistics. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21149) Refactor LlapServiceDriver
[
https://issues.apache.org/jira/browse/HIVE-21149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750173#comment-16750173
]
Hive QA commented on HIVE-21149:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12955965/HIVE-21149.03.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 15719 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[schemeAuthority]
(batchId=191)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15759/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15759/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15759/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12955965 - PreCommit-HIVE-Build
> Refactor LlapServiceDriver
> --
>
> Key: HIVE-21149
> URL: https://issues.apache.org/jira/browse/HIVE-21149
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Affects Versions: 3.1.2
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 3.1.2
>
> Attachments: HIVE-21149.01.patch, HIVE-21149.02.patch,
> HIVE-21149.03.patch
>
>
> LlapServiceDriver is one monolith class doing several things, needs to be
> refactor in order to make it clearer how it works.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20255) Review LevelOrderWalker.java
[
https://issues.apache.org/jira/browse/HIVE-20255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HIVE-20255:
---
Attachment: HIVE-20255.10.patch
> Review LevelOrderWalker.java
>
>
> Key: HIVE-20255
> URL: https://issues.apache.org/jira/browse/HIVE-20255
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Affects Versions: 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HIVE-20255.1.patch, HIVE-20255.10.patch,
> HIVE-20255.2.patch, HIVE-20255.3.patch, HIVE-20255.4.patch,
> HIVE-20255.5.patch, HIVE-20255.6.patch, HIVE-20255.7.patch,
> HIVE-20255.8.patch, HIVE-20255.9.patch
>
>
> https://github.com/apache/hive/blob/6d890faf22fd1ede3658a5eed097476eab3c67e9/ql/src/java/org/apache/hadoop/hive/ql/lib/LevelOrderWalker.java
> * Make code more concise
> * Fix some check style issues
> {code}
> if (toWalk.get(index).getChildren() != null) {
> for(Node child : toWalk.get(index).getChildren()) {
> {code}
> Actually, the underlying implementation of {{getChildren()}} has to do some
> real work, so do not throw away the work after checking for null. Simply
> call once and store the results.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20255) Review LevelOrderWalker.java
[
https://issues.apache.org/jira/browse/HIVE-20255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HIVE-20255:
---
Status: Open (was: Patch Available)
> Review LevelOrderWalker.java
>
>
> Key: HIVE-20255
> URL: https://issues.apache.org/jira/browse/HIVE-20255
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Affects Versions: 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HIVE-20255.1.patch, HIVE-20255.2.patch,
> HIVE-20255.3.patch, HIVE-20255.4.patch, HIVE-20255.5.patch,
> HIVE-20255.6.patch, HIVE-20255.7.patch, HIVE-20255.8.patch, HIVE-20255.9.patch
>
>
> https://github.com/apache/hive/blob/6d890faf22fd1ede3658a5eed097476eab3c67e9/ql/src/java/org/apache/hadoop/hive/ql/lib/LevelOrderWalker.java
> * Make code more concise
> * Fix some check style issues
> {code}
> if (toWalk.get(index).getChildren() != null) {
> for(Node child : toWalk.get(index).getChildren()) {
> {code}
> Actually, the underlying implementation of {{getChildren()}} has to do some
> real work, so do not throw away the work after checking for null. Simply
> call once and store the results.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21149) Refactor LlapServiceDriver
[
https://issues.apache.org/jira/browse/HIVE-21149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750145#comment-16750145
]
Hive QA commented on HIVE-21149:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
22s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
25s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
36s{color} | {color:blue} llap-server in master has 81 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 6m
6s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
23s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m
40s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
10s{color} | {color:red} llap-server: The patch generated 3 new + 3 unchanged -
44 fixed = 6 total (was 47) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
45s{color} | {color:green} llap-server generated 0 new + 79 unchanged - 2 fixed
= 79 total (was 81) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 6m
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
11s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 40m 46s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15759/dev-support/hive-personality.sh
|
| git revision | master / 31bdf16 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15759/yetus/diff-checkstyle-llap-server.txt
|
| modules | C: llap-server . U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15759/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Refactor LlapServiceDriver
> --
>
> Key: HIVE-21149
> URL: https://issues.apache.org/jira/browse/HIVE-21149
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Affects Versions: 3.1.2
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 3.1.2
>
> Attachments: HIVE-21149.01.patch, HIVE-21149.02.patch,
> HIVE-21149.03.patch
>
>
> LlapServiceDriver is one monolith class doing several things, needs to be
> refactor in order to make it clearer how it works.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21001) Upgrade to calcite-1.18
[ https://issues.apache.org/jira/browse/HIVE-21001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-21001: Attachment: HIVE-21001.16.patch > Upgrade to calcite-1.18 > --- > > Key: HIVE-21001 > URL: https://issues.apache.org/jira/browse/HIVE-21001 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-21001.01.patch, HIVE-21001.01.patch, > HIVE-21001.02.patch, HIVE-21001.03.patch, HIVE-21001.04.patch, > HIVE-21001.05.patch, HIVE-21001.06.patch, HIVE-21001.06.patch, > HIVE-21001.07.patch, HIVE-21001.08.patch, HIVE-21001.08.patch, > HIVE-21001.08.patch, HIVE-21001.09.patch, HIVE-21001.09.patch, > HIVE-21001.09.patch, HIVE-21001.10.patch, HIVE-21001.11.patch, > HIVE-21001.12.patch, HIVE-21001.13.patch, HIVE-21001.15.patch, > HIVE-21001.16.patch > > > CLEAR LIBRARY CACHE -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20255) Review LevelOrderWalker.java
[
https://issues.apache.org/jira/browse/HIVE-20255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HIVE-20255:
---
Status: Patch Available (was: Open)
> Review LevelOrderWalker.java
>
>
> Key: HIVE-20255
> URL: https://issues.apache.org/jira/browse/HIVE-20255
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Affects Versions: 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HIVE-20255.1.patch, HIVE-20255.10.patch,
> HIVE-20255.2.patch, HIVE-20255.3.patch, HIVE-20255.4.patch,
> HIVE-20255.5.patch, HIVE-20255.6.patch, HIVE-20255.7.patch,
> HIVE-20255.8.patch, HIVE-20255.9.patch
>
>
> https://github.com/apache/hive/blob/6d890faf22fd1ede3658a5eed097476eab3c67e9/ql/src/java/org/apache/hadoop/hive/ql/lib/LevelOrderWalker.java
> * Make code more concise
> * Fix some check style issues
> {code}
> if (toWalk.get(index).getChildren() != null) {
> for(Node child : toWalk.get(index).getChildren()) {
> {code}
> Actually, the underlying implementation of {{getChildren()}} has to do some
> real work, so do not throw away the work after checking for null. Simply
> call once and store the results.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20758) Constraints: Show create table does not show constraints
[
https://issues.apache.org/jira/browse/HIVE-20758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750184#comment-16750184
]
Jesus Camacho Rodriguez commented on HIVE-20758:
Thanks [~b.maidics], this is nice. Note that information about
enable/validate/rely is stored in the metastore too so you could complete the
statement with those keywords.
https://github.com/apache/hive/search?l=SQL=3=%22ENABLE_VALIDATE_RELY%22
https://github.com/apache/hive/blob/d1460174d5322bcfb642b489dd250c59770f0551/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java#L1956
> Constraints: Show create table does not show constraints
>
>
> Key: HIVE-20758
> URL: https://issues.apache.org/jira/browse/HIVE-20758
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0
>Reporter: Gopal V
>Assignee: Barnabas Maidics
>Priority: Major
> Attachments: HIVE-20758.1.patch, Screen Shot 2019-01-23 at
> 11.52.04.png
>
>
> Even though the desc formatted shows the constraints, the show create table
> does not
> {code}
> | # Primary Key | NULL
> | NULL
>|
> | Table: |
> tpcds_bin_partitioned_orc_1.inventory | NULL
> |
> | Constraint Name: | pk_in
> | NULL
>|
> | Column Names: | inv_date_sk
> | inv_item_sk
>|
> || NULL
> | NULL
>|
> | # Foreign Keys | NULL
> | NULL
>|
> | Table: |
> tpcds_bin_partitioned_orc_1.inventory | NULL
> |
> | Constraint Name: | inv_d
> | NULL
>|
> | Parent Column Name:tpcds_bin_partitioned_orc_1.date_dim.d_date_sk |
> Column Name:inv_date_sk| Key Sequence:1
> |
> || NULL
> | NULL
>|
> | Constraint Name: | inv_i
> | NULL
>|
> | Parent Column Name:tpcds_bin_partitioned_orc_1.item.i_item_sk | Column
> Name:inv_item_sk| Key Sequence:1
>|
> || NULL
> | NULL
>|
> | Constraint Name: | inv_w
> | NULL
>|
> | Parent Column Name:tpcds_bin_partitioned_orc_1.warehouse.w_warehouse_sk
> | Column Name:inv_warehouse_sk | Key Sequence:1
> |
> || NULL
> | NULL
>|
> {code}
> But
> {code}
> ++
> | createtab_stmt |
> ++
> | CREATE TABLE `inventory`( |
> | `inv_item_sk` bigint,|
> | `inv_warehouse_sk` bigint, |
> | `inv_quantity_on_hand` int, |
> | `inv_date_sk` bigint)|
> | ROW FORMAT SERDE |
> | 'org.apache.hadoop.hive.ql.io.orc.OrcSerde' |
> | STORED AS INPUTFORMAT |
> | 'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat' |
> | OUTPUTFORMAT |
> | 'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat' |
> | LOCATION |
> |
>
[jira] [Updated] (HIVE-21149) Refactor LlapServiceDriver
[ https://issues.apache.org/jira/browse/HIVE-21149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-21149: -- Status: Patch Available (was: Open) > Refactor LlapServiceDriver > -- > > Key: HIVE-21149 > URL: https://issues.apache.org/jira/browse/HIVE-21149 > Project: Hive > Issue Type: Improvement > Components: Hive >Affects Versions: 3.1.2 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 3.1.2 > > Attachments: HIVE-21149.01.patch, HIVE-21149.02.patch, > HIVE-21149.03.patch, HIVE-21149.04.patch > > > LlapServiceDriver is one monolith class doing several things, needs to be > refactor in order to make it clearer how it works. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21149) Refactor LlapServiceDriver
[ https://issues.apache.org/jira/browse/HIVE-21149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-21149: -- Attachment: HIVE-21149.04.patch > Refactor LlapServiceDriver > -- > > Key: HIVE-21149 > URL: https://issues.apache.org/jira/browse/HIVE-21149 > Project: Hive > Issue Type: Improvement > Components: Hive >Affects Versions: 3.1.2 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 3.1.2 > > Attachments: HIVE-21149.01.patch, HIVE-21149.02.patch, > HIVE-21149.03.patch, HIVE-21149.04.patch > > > LlapServiceDriver is one monolith class doing several things, needs to be > refactor in order to make it clearer how it works. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21149) Refactor LlapServiceDriver
[ https://issues.apache.org/jira/browse/HIVE-21149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-21149: -- Status: Open (was: Patch Available) > Refactor LlapServiceDriver > -- > > Key: HIVE-21149 > URL: https://issues.apache.org/jira/browse/HIVE-21149 > Project: Hive > Issue Type: Improvement > Components: Hive >Affects Versions: 3.1.2 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 3.1.2 > > Attachments: HIVE-21149.01.patch, HIVE-21149.02.patch, > HIVE-21149.03.patch, HIVE-21149.04.patch > > > LlapServiceDriver is one monolith class doing several things, needs to be > refactor in order to make it clearer how it works. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21143) Add rewrite rules to open/close Between operators
[
https://issues.apache.org/jira/browse/HIVE-21143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750354#comment-16750354
]
Hive QA commented on HIVE-21143:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12956009/HIVE-21143.04.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15762/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15762/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15762/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2019-01-23 19:18:01.763
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-15762/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2019-01-23 19:18:01.767
+ cd apache-github-source-source
+ git fetch origin
>From https://github.com/apache/hive
268a6e5..a7e704c master -> origin/master
+ git reset --hard HEAD
HEAD is now at 268a6e5 HIVE-17084: Turn on hive.stats.fetch.column.stats
configuration flag (Zoltan Haindrich reviewed by Ashutosh Chauhan)
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is behind 'origin/master' by 1 commit, and can be fast-forwarded.
(use "git pull" to update your local branch)
+ git reset --hard origin/master
HEAD is now at a7e704c HIVE-21148: Remove Use StandardCharsets Where Possible
(BELUGA BEHR via Slim Bouguerra)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2019-01-23 19:18:02.678
+ rm -rf ../yetus_PreCommit-HIVE-Build-15762
+ mkdir ../yetus_PreCommit-HIVE-Build-15762
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-15762
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-15762/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error: patch failed:
ql/src/test/results/clientpositive/correlationoptimizer8.q.out:34
Falling back to three-way merge...
Applied patch to
'ql/src/test/results/clientpositive/correlationoptimizer8.q.out' with conflicts.
error: patch failed:
ql/src/test/results/clientpositive/druid_timeseries.q.out:31
Falling back to three-way merge...
Applied patch to 'ql/src/test/results/clientpositive/druid_timeseries.q.out'
cleanly.
error: patch failed:
ql/src/test/results/clientpositive/filter_numeric.q.out:1664
Falling back to three-way merge...
Applied patch to 'ql/src/test/results/clientpositive/filter_numeric.q.out' with
conflicts.
error: patch failed:
ql/src/test/results/clientpositive/infer_join_preds.q.out:667
Falling back to three-way merge...
Applied patch to 'ql/src/test/results/clientpositive/infer_join_preds.q.out'
with conflicts.
error: patch failed: ql/src/test/results/clientpositive/join34.q.out:61
Falling back to three-way merge...
Applied patch to 'ql/src/test/results/clientpositive/join34.q.out' with
conflicts.
error: patch failed: ql/src/test/results/clientpositive/join35.q.out:60
Falling back to three-way merge...
Applied patch to 'ql/src/test/results/clientpositive/join35.q.out' with
conflicts.
error: patch failed: ql/src/test/results/clientpositive/join45.q.out:506
Falling back to three-way merge...
Applied patch to 'ql/src/test/results/clientpositive/join45.q.out' with
conflicts.
error: patch failed: ql/src/test/results/clientpositive/join47.q.out:502
Falling back to three-way merge...
Applied patch to 'ql/src/test/results/clientpositive/join47.q.out' with
conflicts.
error: patch failed: ql/src/test/results/clientpositive/mapjoin47.q.out:530
Falling back to three-way merge...
Applied patch to
[jira] [Updated] (HIVE-21143) Add rewrite rules to open/close Between operators
[ https://issues.apache.org/jira/browse/HIVE-21143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-21143: Attachment: HIVE-21143.05.patch > Add rewrite rules to open/close Between operators > - > > Key: HIVE-21143 > URL: https://issues.apache.org/jira/browse/HIVE-21143 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-21143.01.patch, HIVE-21143.02.patch, > HIVE-21143.03.patch, HIVE-21143.03.patch, HIVE-21143.04.patch, > HIVE-21143.05.patch > > > During query compilation it's better to have BETWEEN statements in open form, > as Calcite current not considering them during simplification. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-21142) Druidhandler may miss results when time constrainted by and/ors
[
https://issues.apache.org/jira/browse/HIVE-21142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Haindrich reassigned HIVE-21142:
---
Assignee: Zoltan Haindrich
> Druidhandler may miss results when time constrainted by and/ors
> ---
>
> Key: HIVE-21142
> URL: https://issues.apache.org/jira/browse/HIVE-21142
> Project: Hive
> Issue Type: Bug
> Components: Druid integration
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: add_test.patch
>
>
> For the following query:
> {code}
> FROM druid_table_alltypesorc
> WHERE ('1968-01-01 00:00:00' <= `__time` AND `__time` <= '1970-01-01
> 00:00:00')
> OR ('1968-02-01 00:00:00' <= `__time` AND `__time` <= '1970-04-01
> 00:00:00') ORDER BY `__time` ASC LIMIT 10;
> {code}
> the druid query is:
> {code}
> druid.query.json
> {"queryType":"scan","dataSource":"default.druid_table_alltypesorc","intervals":["1900-01-01T00:00:00.000Z/1968-02-01T08:00:00.001Z"],"virtualColumns":[{"type":"expression","name":"vc","expression":"\"__time\"","outputType":"LONG"}],"columns":["vc"],"resultFormat":"compactedList"}
> {code}
> which has an invalid interval:
> {{"intervals":["1900-01-01T00:00:00.000Z/1968-02-01T08:00:00.001Z"}} which
> prevents valid results from 1969 to appear.
> note: using between the interval is handled correctly
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20255) Review LevelOrderWalker.java
[
https://issues.apache.org/jira/browse/HIVE-20255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750350#comment-16750350
]
Hive QA commented on HIVE-20255:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12956005/HIVE-20255.10.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 15707 tests
executed
*Failed tests:*
{noformat}
TestReplicationScenariosIncrementalLoadAcidTables - did not produce a
TEST-*.xml file (likely timed out) (batchId=251)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15761/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15761/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15761/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12956005 - PreCommit-HIVE-Build
> Review LevelOrderWalker.java
>
>
> Key: HIVE-20255
> URL: https://issues.apache.org/jira/browse/HIVE-20255
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Affects Versions: 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HIVE-20255.1.patch, HIVE-20255.10.patch,
> HIVE-20255.2.patch, HIVE-20255.3.patch, HIVE-20255.4.patch,
> HIVE-20255.5.patch, HIVE-20255.6.patch, HIVE-20255.7.patch,
> HIVE-20255.8.patch, HIVE-20255.9.patch
>
>
> https://github.com/apache/hive/blob/6d890faf22fd1ede3658a5eed097476eab3c67e9/ql/src/java/org/apache/hadoop/hive/ql/lib/LevelOrderWalker.java
> * Make code more concise
> * Fix some check style issues
> {code}
> if (toWalk.get(index).getChildren() != null) {
> for(Node child : toWalk.get(index).getChildren()) {
> {code}
> Actually, the underlying implementation of {{getChildren()}} has to do some
> real work, so do not throw away the work after checking for null. Simply
> call once and store the results.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HIVE-20699) Query based compactor for full CRUD Acid tables
[
https://issues.apache.org/jira/browse/HIVE-20699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749370#comment-16749370
]
Eugene Koifman edited comment on HIVE-20699 at 1/23/19 7:41 PM:
left some comments on RB (Diff Revision 7)
was (Author: ekoifman):
left some comments on RB
> Query based compactor for full CRUD Acid tables
> ---
>
> Key: HIVE-20699
> URL: https://issues.apache.org/jira/browse/HIVE-20699
> Project: Hive
> Issue Type: New Feature
> Components: Transactions
>Affects Versions: 3.1.0
>Reporter: Eugene Koifman
>Assignee: Vaibhav Gumashta
>Priority: Major
> Attachments: HIVE-20699.1.patch, HIVE-20699.1.patch,
> HIVE-20699.2.patch, HIVE-20699.3.patch, HIVE-20699.4.patch,
> HIVE-20699.5.patch, HIVE-20699.6.patch, HIVE-20699.7.patch
>
>
> Currently the Acid compactor is implemented as generated MR job
> ({{CompactorMR.java}}).
> It could also be expressed as a Hive query that reads from a given partition
> and writes data back to the same partition. This will merge the deltas and
> 'apply' the delete events. The simplest would be to just use Insert
> Overwrite but that will change all ROW__IDs which we don't want.
> Need to implement this in a way that preserves ROW__IDs and creates a new
> {{base_x}} directory to handle Major compaction.
> Minor compaction will be investigated separately.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20556) Expose an API to retrieve the TBL_ID from TBLS in the metastore tables
[ https://issues.apache.org/jira/browse/HIVE-20556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750398#comment-16750398 ] Vihang Karajgaonkar commented on HIVE-20556: Hi [~ekoifman] [~jmarhuen] Sorry for commenting on this late. I was taking a look at this patch and I noticed that the fieldID of the thrift objects are being changed in this patch. Isn't that breaking on-the-wire-compatibility? Also, since TBL_ID is dependent on backing database, aren't we leaking implementation into APIs here? http://www.datanucleus.org/products/accessplatform_4_2/jdo/value_generation.html says that the identity generation could be different based on how the datastore is configured. By exposing its type as a number i64 are we saying that we only support native identity generation? > Expose an API to retrieve the TBL_ID from TBLS in the metastore tables > -- > > Key: HIVE-20556 > URL: https://issues.apache.org/jira/browse/HIVE-20556 > Project: Hive > Issue Type: New Feature > Components: Metastore, Standalone Metastore >Reporter: Jaume M >Assignee: Jaume M >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-20556.1.patch, HIVE-20556.10.patch, > HIVE-20556.11.patch, HIVE-20556.12.patch, HIVE-20556.13.patch, > HIVE-20556.14.patch, HIVE-20556.15.patch, HIVE-20556.16.patch, > HIVE-20556.17.patch, HIVE-20556.18.patch, HIVE-20556.2.patch, > HIVE-20556.3.patch, HIVE-20556.4.patch, HIVE-20556.5.patch, > HIVE-20556.6.patch, HIVE-20556.7.patch, HIVE-20556.8.patch, HIVE-20556.9.patch > > > We have two options to do this > 1) Use the current MTable and add a field for this value > 2) Add an independent API call to the metastore that would return the TBL_ID. > Option 1 is preferable. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21149) Refactor LlapServiceDriver
[
https://issues.apache.org/jira/browse/HIVE-21149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750277#comment-16750277
]
Hive QA commented on HIVE-21149:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12956006/HIVE-21149.04.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15719 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15760/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15760/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15760/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12956006 - PreCommit-HIVE-Build
> Refactor LlapServiceDriver
> --
>
> Key: HIVE-21149
> URL: https://issues.apache.org/jira/browse/HIVE-21149
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Affects Versions: 3.1.2
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 3.1.2
>
> Attachments: HIVE-21149.01.patch, HIVE-21149.02.patch,
> HIVE-21149.03.patch, HIVE-21149.04.patch
>
>
> LlapServiceDriver is one monolith class doing several things, needs to be
> refactor in order to make it clearer how it works.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21148) Remove Use StandardCharsets Where Possible
[
https://issues.apache.org/jira/browse/HIVE-21148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
slim bouguerra updated HIVE-21148:
--
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Remove Use StandardCharsets Where Possible
> --
>
> Key: HIVE-21148
> URL: https://issues.apache.org/jira/browse/HIVE-21148
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 4.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-21148.1.patch
>
>
> Starting in Java 1.7, JDKs must support a set of standard charsets. When
> using this facility, instead of passing around the name (string) of the
> character set, one can pass around a {{Charset}} object and there is no need
> to catch a {{UnsupportedEncodingException}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21148) Remove Use StandardCharsets Where Possible
[
https://issues.apache.org/jira/browse/HIVE-21148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750321#comment-16750321
]
slim bouguerra commented on HIVE-21148:
---
https://github.com/apache/hive/commit/a7e704c679a00db68db9b9f921d133d79a32cfcc
> Remove Use StandardCharsets Where Possible
> --
>
> Key: HIVE-21148
> URL: https://issues.apache.org/jira/browse/HIVE-21148
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 4.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-21148.1.patch
>
>
> Starting in Java 1.7, JDKs must support a set of standard charsets. When
> using this facility, instead of passing around the name (string) of the
> character set, one can pass around a {{Charset}} object and there is no need
> to catch a {{UnsupportedEncodingException}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21125) Hive does not check for dependent materialized views when issuing a DROP TABLE command
[
https://issues.apache.org/jira/browse/HIVE-21125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750263#comment-16750263
]
Taylor Cox commented on HIVE-21125:
---
A check for this is in fact in place, but it seems to not be working correctly:
https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java#L1117
> Hive does not check for dependent materialized views when issuing a DROP
> TABLE command
> --
>
> Key: HIVE-21125
> URL: https://issues.apache.org/jira/browse/HIVE-21125
> Project: Hive
> Issue Type: Bug
> Components: Materialized views
>Affects Versions: 3.1.0
>Reporter: Taylor Cox
>Assignee: Taylor Cox
>Priority: Major
>
> Dropping a table leads to undefined behavior when that table is the source of
> an existing materialized view. The following behavior is observed:
>
> * Table still appears in 'show tables' despite not being in metastore
> * Actions on table hang and then display a "could not fetch table" error
> * Rebuilding any dependent materialized view has same error
> It seems that the root cause is the fact that users are allowed to issue a
> DROP TABLE command against a table even if there is a materialized view using
> this table at the time. This is not something I have seen other query
> languages permit.
> Repro steps: Launch these commands from any Hive 3 client:
> {code:java}
> create table footable (id int); insert into footable values (1), (2), (3);
> create materialized view mv_footable as select count(*) from footable;
> drop table footable;
> --These lines have unexpected behavior
> show tables;
> select * from footable;
> alter materialized view mv_footable rebuild;{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21115) Add support for object versions in metastore
[
https://issues.apache.org/jira/browse/HIVE-21115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bharathkrishna Guruvayoor Murali updated HIVE-21115:
Attachment: HIVE-21115.2.patch
> Add support for object versions in metastore
>
>
> Key: HIVE-21115
> URL: https://issues.apache.org/jira/browse/HIVE-21115
> Project: Hive
> Issue Type: Improvement
>Reporter: Vihang Karajgaonkar
>Assignee: Bharathkrishna Guruvayoor Murali
>Priority: Major
> Attachments: HIVE-21115.1.patch, HIVE-21115.2.patch
>
>
> Currently, metastore objects are identified uniquely by their names (eg.
> catName, dbName and tblName for a table is unique). Once a table or partition
> is created it could be altered in many ways. There is no good way currently
> to identify the version of the object once it is altered. For example,
> suppose there are two clients (Hive and Impala) using the same metastore.
> Once some alter operations are performed by a client, another client which
> wants to do a alter operation has no good way to know if the object which it
> has is the same as the one stored in metastore. Metastore updates the
> {{transient_lastDdlTime}} every time there is a DDL operation on the object.
> However, this value cannot be relied for all the clients since after
> HIVE-1768 metastore updates the value only when it is not set in the
> parameters. It is possible that a client which alters the object state, does
> not remove the {{transient_lastDdlTime}} and metastore will not update it.
> Secondly, if there is a clock skew between multiple HMS instances when HMS-HA
> is configured, time values cannot be relied on to find out the sequence of
> alter operations on a given object.
> This JIRA propose to use JDO versioning support by Datanucleus
> http://www.datanucleus.org/products/accessplatform_4_2/jdo/versioning.html to
> generate a incrementing sequence number every time a object is altered. The
> value of this object can be set as one of the values in the parameters. The
> advantage of using Datanucleus the versioning can be done across HMS
> instances as part of the database transaction and it should work for all the
> supported databases.
> In theory such a version can be used to detect if the client is presenting a
> object which is "stale" when issuing a alter request. Metastore can choose to
> reject such a alter request since the client may be caching a old version of
> the object and any alter operation on such stale object can potentially
> overwrite previous operations. However, this is can be done in a separate
> JIRA.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21115) Add support for object versions in metastore
[
https://issues.apache.org/jira/browse/HIVE-21115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750269#comment-16750269
]
Alan Gates commented on HIVE-21115:
---
I'm surprised to see a patch here already, as we don't have consensus yet on
how to proceed. Given this is a major change, we need to get consensus.
> Add support for object versions in metastore
>
>
> Key: HIVE-21115
> URL: https://issues.apache.org/jira/browse/HIVE-21115
> Project: Hive
> Issue Type: Improvement
>Reporter: Vihang Karajgaonkar
>Assignee: Bharathkrishna Guruvayoor Murali
>Priority: Major
> Attachments: HIVE-21115.1.patch, HIVE-21115.2.patch
>
>
> Currently, metastore objects are identified uniquely by their names (eg.
> catName, dbName and tblName for a table is unique). Once a table or partition
> is created it could be altered in many ways. There is no good way currently
> to identify the version of the object once it is altered. For example,
> suppose there are two clients (Hive and Impala) using the same metastore.
> Once some alter operations are performed by a client, another client which
> wants to do a alter operation has no good way to know if the object which it
> has is the same as the one stored in metastore. Metastore updates the
> {{transient_lastDdlTime}} every time there is a DDL operation on the object.
> However, this value cannot be relied for all the clients since after
> HIVE-1768 metastore updates the value only when it is not set in the
> parameters. It is possible that a client which alters the object state, does
> not remove the {{transient_lastDdlTime}} and metastore will not update it.
> Secondly, if there is a clock skew between multiple HMS instances when HMS-HA
> is configured, time values cannot be relied on to find out the sequence of
> alter operations on a given object.
> This JIRA propose to use JDO versioning support by Datanucleus
> http://www.datanucleus.org/products/accessplatform_4_2/jdo/versioning.html to
> generate a incrementing sequence number every time a object is altered. The
> value of this object can be set as one of the values in the parameters. The
> advantage of using Datanucleus the versioning can be done across HMS
> instances as part of the database transaction and it should work for all the
> supported databases.
> In theory such a version can be used to detect if the client is presenting a
> object which is "stale" when issuing a alter request. Metastore can choose to
> reject such a alter request since the client may be caching a old version of
> the object and any alter operation on such stale object can potentially
> overwrite previous operations. However, this is can be done in a separate
> JIRA.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21115) Add support for object versions in metastore
[
https://issues.apache.org/jira/browse/HIVE-21115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750280#comment-16750280
]
Bharathkrishna Guruvayoor Murali commented on HIVE-21115:
-
Hi Alan, sure we can proceed after discussing and getting the consensus. The
main idea of putting this patch is for everyone to get an idea of how we are
currently thinking of implementing this and to see if any unexpected tests fail
to detect early if there are any problems with the approach.
Updating the versions via datanucleus has a problem that the updated version
number is not reflected in the MetaStoreConf notifications for transactional
listeners because the notifications are issued before commitTransaction().
Hence, the logic in the prototype patch attached involves select..for update on
the respective table/partition row to update version. As I mentioned, let's go
ahead and put more thoughts into this, and the patch just serves to give more
clarity on the idea.
> Add support for object versions in metastore
>
>
> Key: HIVE-21115
> URL: https://issues.apache.org/jira/browse/HIVE-21115
> Project: Hive
> Issue Type: Improvement
>Reporter: Vihang Karajgaonkar
>Assignee: Bharathkrishna Guruvayoor Murali
>Priority: Major
> Attachments: HIVE-21115.1.patch, HIVE-21115.2.patch
>
>
> Currently, metastore objects are identified uniquely by their names (eg.
> catName, dbName and tblName for a table is unique). Once a table or partition
> is created it could be altered in many ways. There is no good way currently
> to identify the version of the object once it is altered. For example,
> suppose there are two clients (Hive and Impala) using the same metastore.
> Once some alter operations are performed by a client, another client which
> wants to do a alter operation has no good way to know if the object which it
> has is the same as the one stored in metastore. Metastore updates the
> {{transient_lastDdlTime}} every time there is a DDL operation on the object.
> However, this value cannot be relied for all the clients since after
> HIVE-1768 metastore updates the value only when it is not set in the
> parameters. It is possible that a client which alters the object state, does
> not remove the {{transient_lastDdlTime}} and metastore will not update it.
> Secondly, if there is a clock skew between multiple HMS instances when HMS-HA
> is configured, time values cannot be relied on to find out the sequence of
> alter operations on a given object.
> This JIRA propose to use JDO versioning support by Datanucleus
> http://www.datanucleus.org/products/accessplatform_4_2/jdo/versioning.html to
> generate a incrementing sequence number every time a object is altered. The
> value of this object can be set as one of the values in the parameters. The
> advantage of using Datanucleus the versioning can be done across HMS
> instances as part of the database transaction and it should work for all the
> supported databases.
> In theory such a version can be used to detect if the client is presenting a
> object which is "stale" when issuing a alter request. Metastore can choose to
> reject such a alter request since the client may be caching a old version of
> the object and any alter operation on such stale object can potentially
> overwrite previous operations. However, this is can be done in a separate
> JIRA.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21149) Refactor LlapServiceDriver
[
https://issues.apache.org/jira/browse/HIVE-21149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750254#comment-16750254
]
Hive QA commented on HIVE-21149:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
19s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
25s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
37s{color} | {color:blue} llap-server in master has 81 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 6m
12s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
24s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m
49s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
11s{color} | {color:red} llap-server: The patch generated 3 new + 3 unchanged -
44 fixed = 6 total (was 47) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
44s{color} | {color:green} llap-server generated 0 new + 79 unchanged - 2 fixed
= 79 total (was 81) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 6m
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
11s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 41m 18s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15760/dev-support/hive-personality.sh
|
| git revision | master / 268a6e5 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15760/yetus/diff-checkstyle-llap-server.txt
|
| modules | C: llap-server . U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15760/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Refactor LlapServiceDriver
> --
>
> Key: HIVE-21149
> URL: https://issues.apache.org/jira/browse/HIVE-21149
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Affects Versions: 3.1.2
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 3.1.2
>
> Attachments: HIVE-21149.01.patch, HIVE-21149.02.patch,
> HIVE-21149.03.patch, HIVE-21149.04.patch
>
>
> LlapServiceDriver is one monolith class doing several things, needs to be
> refactor in order to make it clearer how it works.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20255) Review LevelOrderWalker.java
[
https://issues.apache.org/jira/browse/HIVE-20255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750294#comment-16750294
]
Hive QA commented on HIVE-20255:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
35s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
33s{color} | {color:blue} ql in master has 2305 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
35s{color} | {color:green} ql: The patch generated 0 new + 1 unchanged - 2
fixed = 1 total (was 3) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 21m 38s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15761/dev-support/hive-personality.sh
|
| git revision | master / 268a6e5 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15761/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Review LevelOrderWalker.java
>
>
> Key: HIVE-20255
> URL: https://issues.apache.org/jira/browse/HIVE-20255
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Affects Versions: 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HIVE-20255.1.patch, HIVE-20255.10.patch,
> HIVE-20255.2.patch, HIVE-20255.3.patch, HIVE-20255.4.patch,
> HIVE-20255.5.patch, HIVE-20255.6.patch, HIVE-20255.7.patch,
> HIVE-20255.8.patch, HIVE-20255.9.patch
>
>
> https://github.com/apache/hive/blob/6d890faf22fd1ede3658a5eed097476eab3c67e9/ql/src/java/org/apache/hadoop/hive/ql/lib/LevelOrderWalker.java
> * Make code more concise
> * Fix some check style issues
> {code}
> if (toWalk.get(index).getChildren() != null) {
> for(Node child : toWalk.get(index).getChildren()) {
> {code}
> Actually, the underlying implementation of {{getChildren()}} has to do some
> real work, so do not throw away the work after checking for null. Simply
> call once and store the results.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21148) Remove Use StandardCharsets Where Possible
[
https://issues.apache.org/jira/browse/HIVE-21148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750306#comment-16750306
]
slim bouguerra commented on HIVE-21148:
---
[~belugabehr] Looks good to me thanks for doing that. +1
> Remove Use StandardCharsets Where Possible
> --
>
> Key: HIVE-21148
> URL: https://issues.apache.org/jira/browse/HIVE-21148
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 4.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-21148.1.patch
>
>
> Starting in Java 1.7, JDKs must support a set of standard charsets. When
> using this facility, instead of passing around the name (string) of the
> character set, one can pass around a {{Charset}} object and there is no need
> to catch a {{UnsupportedEncodingException}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21001) Upgrade to calcite-1.18
[
https://issues.apache.org/jira/browse/HIVE-21001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750403#comment-16750403
]
Hive QA commented on HIVE-21001:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
14s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
12s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
40s{color} | {color:blue} ql in master has 2304 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
25s{color} | {color:blue} accumulo-handler in master has 21 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
29s{color} | {color:blue} hbase-handler in master has 15 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 7m
19s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
21s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
12s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
38s{color} | {color:red} ql: The patch generated 4 new + 135 unchanged - 2
fixed = 139 total (was 137) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
1s{color} | {color:red} The patch has 3 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
1s{color} | {color:red} The patch 14 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 7m
47s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 59m 29s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc xml compile findbugs
checkstyle |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15763/dev-support/hive-personality.sh
|
| git revision | master / a7e704c |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15763/yetus/diff-checkstyle-ql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15763/yetus/whitespace-eol.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15763/yetus/whitespace-tabs.txt
|
| modules | C: ql accumulo-handler hbase-handler . U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15763/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Upgrade to calcite-1.18
> ---
>
> Key: HIVE-21001
> URL: https://issues.apache.org/jira/browse/HIVE-21001
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>
[jira] [Commented] (HIVE-21001) Upgrade to calcite-1.18
[
https://issues.apache.org/jira/browse/HIVE-21001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750406#comment-16750406
]
Hive QA commented on HIVE-21001:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12956010/HIVE-21001.16.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 155 failed/errored test(s), 15709 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_predicate_pushdown]
(batchId=267)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ambiguitycheck]
(batchId=79)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[complex_alias]
(batchId=18)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[constant_prop_3]
(batchId=47)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[correlationoptimizer8]
(batchId=14)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[decimal_udf] (batchId=10)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[filter_numeric]
(batchId=87)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[fold_case] (batchId=16)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[fold_eq_with_case_when]
(batchId=88)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[fold_when] (batchId=30)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[in_typecheck_char]
(batchId=51)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[infer_const_type]
(batchId=74)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[infer_join_preds]
(batchId=26)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[innerjoin1] (batchId=26)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input8] (batchId=9)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join34] (batchId=76)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join35] (batchId=70)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join45] (batchId=21)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join47] (batchId=34)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[macro] (batchId=4)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin47] (batchId=64)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[order3] (batchId=69)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_5]
(batchId=76)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_8]
(batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[partition_wise_fileformat2]
(batchId=94)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[pcr] (batchId=65)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[pcs] (batchId=54)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd2] (batchId=28)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_constant_expr]
(batchId=63)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_udf_case]
(batchId=47)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppr_allchildsarenull]
(batchId=75)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[rand_partitionpruner3]
(batchId=86)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[select_unquote_or]
(batchId=41)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[smb_mapjoin_47]
(batchId=31)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[subquery_notin_having]
(batchId=54)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[subquery_unqualcolumnrefs]
(batchId=19)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[timestamp_ints_casts]
(batchId=1)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_between] (batchId=78)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union_offcbo]
(batchId=49)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_between_columns]
(batchId=75)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_case_when_1]
(batchId=91)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_case_when_2]
(batchId=58)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_decimal_math_funcs]
(batchId=25)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_groupby_mapjoin]
(batchId=81)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorization_5]
(batchId=62)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorization_8]
(batchId=51)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorized_casts]
(batchId=89)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorized_math_funcs]
(batchId=22)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorized_string_funcs]
(batchId=63)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorized_timestamp_ints_casts]
(batchId=53)
org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver[hbase_ppd_key_range]
(batchId=104)
[jira] [Commented] (HIVE-21115) Add support for object versions in metastore
[
https://issues.apache.org/jira/browse/HIVE-21115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750428#comment-16750428
]
Hive QA commented on HIVE-21115:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
26s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
21s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
11s{color} | {color:blue} standalone-metastore/metastore-common in master has
29 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
1s{color} | {color:blue} standalone-metastore/metastore-server in master has
184 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
19s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
29s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 4 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
14s{color} | {color:red} standalone-metastore/metastore-server generated 1 new
+ 184 unchanged - 0 fixed = 185 total (was 184) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 51s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:standalone-metastore/metastore-server |
| | Nullcheck of newt at line 4172 of value previously dereferenced in
org.apache.hadoop.hive.metastore.ObjectStore.alterTable(String, String, String,
Table, String) At ObjectStore.java:4172 of value previously dereferenced in
org.apache.hadoop.hive.metastore.ObjectStore.alterTable(String, String, String,
Table, String) At ObjectStore.java:[line 4172] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15764/dev-support/hive-personality.sh
|
| git revision | master / a7e704c |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15764/yetus/whitespace-eol.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15764/yetus/new-findbugs-standalone-metastore_metastore-server.html
|
| modules | C: standalone-metastore/metastore-common metastore
standalone-metastore/metastore-server U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15764/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Add support for object versions in metastore
>
>
> Key: HIVE-21115
> URL: https://issues.apache.org/jira/browse/HIVE-21115
> Project: Hive
> Issue Type: Improvement
>
[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called
[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750429#comment-16750429 ] Jaume M commented on HIVE-21052: Can you take a new look [~ekoifman], I've updated the rb. > Make sure transactions get cleaned if they are aborted before addPartitions > is called > - > > Key: HIVE-21052 > URL: https://issues.apache.org/jira/browse/HIVE-21052 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 3.0.0 >Reporter: Jaume M >Assignee: Jaume M >Priority: Critical > Attachments: Aborted Txn w_Direct Write.pdf, HIVE-21052.1.patch, > HIVE-21052.2.patch, HIVE-21052.3.patch, HIVE-21052.4.patch, > HIVE-21052.5.patch, HIVE-21052.6.patch, HIVE-21052.7.patch, HIVE-21052.8.patch > > > If the transaction is aborted between openTxn and addPartitions and data has > been written on the table the transaction manager will think it's an empty > transaction and no cleaning will be done. > This is currently an issue in the streaming API and in micromanaged tables. > As proposed by [~ekoifman] this can be solved by: > * Writing an entry with a special marker to TXN_COMPONENTS at openTxn and > when addPartitions is called remove this entry from TXN_COMPONENTS and add > the corresponding partition entry to TXN_COMPONENTS. > * If the cleaner finds and entry with a special marker in TXN_COMPONENTS that > specifies that a transaction was opened and it was aborted it must generate > jobs for the worker for every possible partition available. > cc [~ewohlstadter] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-21155) create_time datatype for database and catalog should be int in sql server schema scripts
[ https://issues.apache.org/jira/browse/HIVE-21155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vihang Karajgaonkar reassigned HIVE-21155: -- > create_time datatype for database and catalog should be int in sql server > schema scripts > > > Key: HIVE-21155 > URL: https://issues.apache.org/jira/browse/HIVE-21155 > Project: Hive > Issue Type: Bug >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar >Priority: Trivial > > HIVE-21077 added create_time field to database and catalogs. However, the > data type of this field was set to bigint instead of int like in case of > other create_time fields for tbls and partitions. We should change it to int > from bigint to be consistent with other create_time fields. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21077) Database and catalogs should have creation time
[
https://issues.apache.org/jira/browse/HIVE-21077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vihang Karajgaonkar updated HIVE-21077:
---
Attachment: HIVE-21077.addendum.patch
> Database and catalogs should have creation time
> ---
>
> Key: HIVE-21077
> URL: https://issues.apache.org/jira/browse/HIVE-21077
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Major
> Fix For: 4.0.0, 3.2.0
>
> Attachments: HIVE-21077.01.patch, HIVE-21077.02.patch,
> HIVE-21077.03.patch, HIVE-21077.04.patch, HIVE-21077.05.patch,
> HIVE-21077.06.patch, HIVE-21077.07.patch, HIVE-21077.08.branch-3.patch,
> HIVE-21077.09.patch, HIVE-21077.10.patch, HIVE-21077.addendum.patch
>
>
> Currently, database do not have creation time like we have for tables and
> partitions.
> {noformat}
> // namespace for tables
> struct Database {
> 1: string name,
> 2: string description,
> 3: string locationUri,
> 4: map parameters, // properties associated with the
> database
> 5: optional PrincipalPrivilegeSet privileges,
> 6: optional string ownerName,
> 7: optional PrincipalType ownerType,
> 8: optional string catalogName
> }
> {noformat}
> Currently, without creationTime there is no way to identify if the copy of
> Database which a client has is the same as the one on the server if the name
> is same. Without object ids creationTime value is the only way currently to
> identify uniquely a instance of metastore object. It would be good to have
> Database creation time as well.
> Same applies for catalogs as well.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (HIVE-21155) create_time datatype for database and catalog should be int in sql server schema scripts
[ https://issues.apache.org/jira/browse/HIVE-21155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vihang Karajgaonkar resolved HIVE-21155. Resolution: Invalid > create_time datatype for database and catalog should be int in sql server > schema scripts > > > Key: HIVE-21155 > URL: https://issues.apache.org/jira/browse/HIVE-21155 > Project: Hive > Issue Type: Bug >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar >Priority: Trivial > > HIVE-21077 added create_time field to database and catalogs. However, the > data type of this field was set to bigint instead of int like in case of > other create_time fields for tbls and partitions. We should change it to int > from bigint to be consistent with other create_time fields. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21155) create_time datatype for database and catalog should be int in sql server schema scripts
[ https://issues.apache.org/jira/browse/HIVE-21155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750461#comment-16750461 ] Vihang Karajgaonkar commented on HIVE-21155: actually I think its better to add this as a addendum to HIVE-21077. Closing this as invalid and will provide a addedum patch on HIVE-21077 > create_time datatype for database and catalog should be int in sql server > schema scripts > > > Key: HIVE-21155 > URL: https://issues.apache.org/jira/browse/HIVE-21155 > Project: Hive > Issue Type: Bug >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar >Priority: Trivial > > HIVE-21077 added create_time field to database and catalogs. However, the > data type of this field was set to bigint instead of int like in case of > other create_time fields for tbls and partitions. We should change it to int > from bigint to be consistent with other create_time fields. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21077) Database and catalogs should have creation time
[
https://issues.apache.org/jira/browse/HIVE-21077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750462#comment-16750462
]
Vihang Karajgaonkar commented on HIVE-21077:
Attaching a addendum patch which changes datatype of CREATE_TIME field in mssql
scripts for database and catalog to INT from BIGINT to make it consistent with
other create_time fields.
> Database and catalogs should have creation time
> ---
>
> Key: HIVE-21077
> URL: https://issues.apache.org/jira/browse/HIVE-21077
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Major
> Fix For: 4.0.0, 3.2.0
>
> Attachments: HIVE-21077.01.patch, HIVE-21077.02.patch,
> HIVE-21077.03.patch, HIVE-21077.04.patch, HIVE-21077.05.patch,
> HIVE-21077.06.patch, HIVE-21077.07.patch, HIVE-21077.08.branch-3.patch,
> HIVE-21077.09.patch, HIVE-21077.10.patch, HIVE-21077.addendum.patch
>
>
> Currently, database do not have creation time like we have for tables and
> partitions.
> {noformat}
> // namespace for tables
> struct Database {
> 1: string name,
> 2: string description,
> 3: string locationUri,
> 4: map parameters, // properties associated with the
> database
> 5: optional PrincipalPrivilegeSet privileges,
> 6: optional string ownerName,
> 7: optional PrincipalType ownerType,
> 8: optional string catalogName
> }
> {noformat}
> Currently, without creationTime there is no way to identify if the copy of
> Database which a client has is the same as the one on the server if the name
> is same. Without object ids creationTime value is the only way currently to
> identify uniquely a instance of metastore object. It would be good to have
> Database creation time as well.
> Same applies for catalogs as well.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20556) Expose an API to retrieve the TBL_ID from TBLS in the metastore tables
[ https://issues.apache.org/jira/browse/HIVE-20556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750473#comment-16750473 ] Jaume M commented on HIVE-20556: bq. Isn't that breaking on-the-wire-compatibility? What is the compatibility that should be preserved? Hive can be upgraded while the metastore remain the same? So new client should be able to speak to old server. bq. Also, since TBL_ID is dependent on backing database, aren't we leaking implementation into APIs here? The purpose of exposing this was for the client to be able to submit the table id in this ticket HIVE-20538. What would be the problem here? The client could figure out what database is being used by analyzing the ids? bq. By exposing its type as a number i64 are we saying that we only support native identity generation. I assumed since before the strategy wasn't specified, native identity was the one being used, like now. > Expose an API to retrieve the TBL_ID from TBLS in the metastore tables > -- > > Key: HIVE-20556 > URL: https://issues.apache.org/jira/browse/HIVE-20556 > Project: Hive > Issue Type: New Feature > Components: Metastore, Standalone Metastore >Reporter: Jaume M >Assignee: Jaume M >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-20556.1.patch, HIVE-20556.10.patch, > HIVE-20556.11.patch, HIVE-20556.12.patch, HIVE-20556.13.patch, > HIVE-20556.14.patch, HIVE-20556.15.patch, HIVE-20556.16.patch, > HIVE-20556.17.patch, HIVE-20556.18.patch, HIVE-20556.2.patch, > HIVE-20556.3.patch, HIVE-20556.4.patch, HIVE-20556.5.patch, > HIVE-20556.6.patch, HIVE-20556.7.patch, HIVE-20556.8.patch, HIVE-20556.9.patch > > > We have two options to do this > 1) Use the current MTable and add a field for this value > 2) Add an independent API call to the metastore that would return the TBL_ID. > Option 1 is preferable. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20776) Run HMS filterHooks on server-side in addition to client-side
[ https://issues.apache.org/jira/browse/HIVE-20776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated HIVE-20776: - Attachment: HIVE-20776.015.branch-3.patch > Run HMS filterHooks on server-side in addition to client-side > - > > Key: HIVE-20776 > URL: https://issues.apache.org/jira/browse/HIVE-20776 > Project: Hive > Issue Type: Improvement > Components: Standalone Metastore >Reporter: Karthik Manamcheri >Assignee: Na Li >Priority: Major > Attachments: HIVE-20776.001.patch, HIVE-20776.003.patch, > HIVE-20776.004.patch, HIVE-20776.005.patch, HIVE-20776.006.patch, > HIVE-20776.007.patch, HIVE-20776.007.patch, HIVE-20776.008.patch, > HIVE-20776.009.patch, HIVE-20776.009.patch, HIVE-20776.010.patch, > HIVE-20776.011.patch, HIVE-20776.011.patch, HIVE-20776.012.patch, > HIVE-20776.013.patch, HIVE-20776.014.patch, HIVE-20776.015.branch-3.patch, > HIVE-20776.015.patch > > > In HMS, I noticed that all the filter hooks are applied on the client side > (in HiveMetaStoreClient.java). Is there any reason why we can't apply the > filters on the server-side? > Motivation: Some newer apache projects such as Kudu use HMS for metadata > storage. Kudu is not completely Java-based and there are interaction points > where they have C++ clients. In such cases, it would be ideal to have > consistent behavior from HMS side as far as filters, etc are concerned. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (HIVE-21156) SharedWorkOptimizer may preserve filter in TS incorrectly
[ https://issues.apache.org/jira/browse/HIVE-21156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-21156 started by Jesus Camacho Rodriguez. -- > SharedWorkOptimizer may preserve filter in TS incorrectly > - > > Key: HIVE-21156 > URL: https://issues.apache.org/jira/browse/HIVE-21156 > Project: Hive > Issue Type: Bug > Components: Physical Optimizer >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > > For some scan reutilizations, we may end up with a Filter expression > associated with the scan that should be removed by the optimizer. This can > lead to incorrect results. Repro case is part of the patch. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21156) SharedWorkOptimizer may preserve filter in TS incorrectly
[ https://issues.apache.org/jira/browse/HIVE-21156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-21156: --- Status: Patch Available (was: In Progress) > SharedWorkOptimizer may preserve filter in TS incorrectly > - > > Key: HIVE-21156 > URL: https://issues.apache.org/jira/browse/HIVE-21156 > Project: Hive > Issue Type: Bug > Components: Physical Optimizer >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > > For some scan reutilizations, we may end up with a Filter expression > associated with the scan that should be removed by the optimizer. This can > lead to incorrect results. Repro case is part of the patch. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21156) SharedWorkOptimizer may preserve filter in TS incorrectly
[ https://issues.apache.org/jira/browse/HIVE-21156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-21156: --- Attachment: HIVE-21156.patch > SharedWorkOptimizer may preserve filter in TS incorrectly > - > > Key: HIVE-21156 > URL: https://issues.apache.org/jira/browse/HIVE-21156 > Project: Hive > Issue Type: Bug > Components: Physical Optimizer >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-21156.patch > > > For some scan reutilizations, we may end up with a Filter expression > associated with the scan that should be removed by the optimizer. This can > lead to incorrect results. Repro case is part of the patch. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-21156) SharedWorkOptimizer may preserve filter in TS incorrectly
[ https://issues.apache.org/jira/browse/HIVE-21156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez reassigned HIVE-21156: -- > SharedWorkOptimizer may preserve filter in TS incorrectly > - > > Key: HIVE-21156 > URL: https://issues.apache.org/jira/browse/HIVE-21156 > Project: Hive > Issue Type: Bug >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > > For some scan reutilizations, we may end up with a Filter expression > associated with the scan that should be removed by the optimizer. This can > lead to incorrect results. Repro case is part of the patch. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21045) Add HMS total api count stats and connection pool stats to metrics
[
https://issues.apache.org/jira/browse/HIVE-21045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karthik Manamcheri updated HIVE-21045:
--
Attachment: HIVE-21045.4.patch
> Add HMS total api count stats and connection pool stats to metrics
> --
>
> Key: HIVE-21045
> URL: https://issues.apache.org/jira/browse/HIVE-21045
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Reporter: Karthik Manamcheri
>Assignee: Karthik Manamcheri
>Priority: Minor
> Attachments: HIVE-21045.1.patch, HIVE-21045.2.patch,
> HIVE-21045.3.patch, HIVE-21045.4.patch
>
>
> There are two key metrics which I think we lack and which would be really
> great to help with scaling visibility in HMS.
> *Total API calls duration stats*
> We already compute and log the duration of API calls in the {{PerfLogger}}.
> We don't have any gauge or timer on what the average duration of an API call
> is for the past some bucket of time. This will give us an insight into if
> there is load on the server which is increasing the average API response time.
>
> *Connection Pool stats*
> We can use different connection pooling libraries such as bonecp or hikaricp.
> These pool managers expose statistics such as average time waiting to get a
> connection, number of connections active, etc. We should expose this as a
> metric so that we can track if the the connection pool size configured is too
> small and we are saturating!
> These metrics would help catch problems with HMS resource contention before
> they actually have jobs failing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21156) SharedWorkOptimizer may preserve filter in TS incorrectly
[ https://issues.apache.org/jira/browse/HIVE-21156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-21156: --- Component/s: Physical Optimizer > SharedWorkOptimizer may preserve filter in TS incorrectly > - > > Key: HIVE-21156 > URL: https://issues.apache.org/jira/browse/HIVE-21156 > Project: Hive > Issue Type: Bug > Components: Physical Optimizer >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > > For some scan reutilizations, we may end up with a Filter expression > associated with the scan that should be removed by the optimizer. This can > lead to incorrect results. Repro case is part of the patch. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21132) Semi join edge is not being removed despite max bloomfilter entries set to 1
[
https://issues.apache.org/jira/browse/HIVE-21132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-21132:
---
Attachment: HIVE-21132.2.patch
> Semi join edge is not being removed despite max bloomfilter entries set to 1
>
>
> Key: HIVE-21132
> URL: https://issues.apache.org/jira/browse/HIVE-21132
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Affects Versions: 4.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-21132.1.patch, HIVE-21132.2.patch
>
>
> * Reproducer
> {code:sql}
> --! qt:dataset:lineitem
> --! qt:dataset:part
> --! qt:dataset:src
> set hive.support.concurrency=true;
> set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
> --set hive.compute.query.using.stats=false;
> set hive.mapred.mode=nonstrict;
> set hive.explain.user=false;
> set hive.optimize.ppd=true;
> set hive.ppd.remove.duplicatefilters=true;
> set hive.tez.dynamic.partition.pruning=true;
> set hive.tez.dynamic.semijoin.reduction=true;
> set hive.optimize.metadataonly=false;
> set hive.optimize.index.filter=true;
> set hive.stats.autogather=true;
> set hive.tez.bigtable.minsize.semijoin.reduction=1;
> set hive.tez.min.bloom.filter.entries=1;
> set hive.stats.fetch.column.stats=true;
> set hive.tez.bloom.filter.factor=1.0f;
> set hive.auto.convert.join=false;
> set hive.optimize.shared.work=false;
> create database tpch_test;
> use tpch_test;
> CREATE TABLE `customer`(
> `c_custkey` bigint,
> `c_name` string,
> `c_address` string,
> `c_nationkey` bigint,
> `c_phone` string,
> `c_acctbal` double,
> `c_mktsegment` string,
> `c_comment` string)
> ROW FORMAT SERDE
> 'org.apache.hadoop.hive.ql.io.orc.OrcSerde'
> STORED AS INPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'
> OUTPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat'
> TBLPROPERTIES (
> 'bucketing_version'='2',
> 'transactional'='true',
> 'transactional_properties'='default',
> 'transient_lastDdlTime'='1543026723');
> CREATE TABLE `lineitem`(
> `l_orderkey` bigint,
> `l_partkey` bigint,
> `l_suppkey` bigint,
> `l_linenumber` int,
> `l_quantity` double,
> `l_extendedprice` double,
> `l_discount` double,
> `l_tax` double,
> `l_returnflag` string,
> `l_linestatus` string,
> `l_shipdate` string,
> `l_commitdate` string,
> `l_receiptdate` string,
> `l_shipinstruct` string,
> `l_shipmode` string,
> `l_comment` string)
> ROW FORMAT SERDE
> 'org.apache.hadoop.hive.ql.io.orc.OrcSerde'
> STORED AS INPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'
> OUTPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat'
> TBLPROPERTIES (
> 'bucketing_version'='2',
> 'transactional'='true',
> 'transactional_properties'='default',
> 'transient_lastDdlTime'='1543027179');
> CREATE TABLE `orders`(
> `o_orderkey` bigint,
> `o_custkey` bigint,
> `o_orderstatus` string,
> `o_totalprice` double,
> `o_orderdate` string,
> `o_orderpriority` string,
> `o_clerk` string,
> `o_shippriority` int,
> `o_comment` string)
> ROW FORMAT SERDE
> 'org.apache.hadoop.hive.ql.io.orc.OrcSerde'
> STORED AS INPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'
> OUTPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat'
> TBLPROPERTIES (
> 'bucketing_version'='2',
> 'transactional'='true',
> 'transactional_properties'='default',
> 'transient_lastDdlTime'='1543026824');
> alter table customer update statistics
> set('numRows'='15000','rawDataSize'='8633707142');
> alter table lineitem update statistics
> set('numRows'='589709','rawDataSize'='184245066955');
> alter table orders update statistics
> set('numRows'='15','rawDataSize'='46741318253');
> create view q18_tmp_cached as
> select l_orderkey, sum(l_quantity) as t_sum_quantity
> from lineitem
> where l_orderkey is not null
> group by l_orderkey;
> -- Set bloom filter size to huge number so we get any possible semijoin
> reductions
> set hive.tez.min.bloom.filter.entries=0;
> set hive.tez.max.bloom.filter.entries=1;
> create table q18_large_volume_customer_cached stored as orc tblproperties
> ('transactional'='true', 'transactional_properties'='default') as
> select c_name, c_custkey, o_orderkey, o_orderdate, o_totalprice,
> sum(l_quantity)
> from customer, orders, q18_tmp_cached t, lineitem l
> where
> c_custkey = o_custkey and o_orderkey = t.l_orderkey
> and o_orderkey is not null and t.t_sum_quantity > 300
> and o_orderkey = l.l_orderkey and l.l_orderkey is not null
> group by c_name, c_custkey, o_orderkey, o_orderdate, o_totalprice
> order by o_totalprice desc, o_orderdate
> limit 100;
>
[jira] [Updated] (HIVE-21132) Semi join edge is not being removed despite max bloomfilter entries set to 1
[
https://issues.apache.org/jira/browse/HIVE-21132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-21132:
---
Status: Open (was: Patch Available)
> Semi join edge is not being removed despite max bloomfilter entries set to 1
>
>
> Key: HIVE-21132
> URL: https://issues.apache.org/jira/browse/HIVE-21132
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Affects Versions: 4.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-21132.1.patch, HIVE-21132.2.patch
>
>
> * Reproducer
> {code:sql}
> --! qt:dataset:lineitem
> --! qt:dataset:part
> --! qt:dataset:src
> set hive.support.concurrency=true;
> set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
> --set hive.compute.query.using.stats=false;
> set hive.mapred.mode=nonstrict;
> set hive.explain.user=false;
> set hive.optimize.ppd=true;
> set hive.ppd.remove.duplicatefilters=true;
> set hive.tez.dynamic.partition.pruning=true;
> set hive.tez.dynamic.semijoin.reduction=true;
> set hive.optimize.metadataonly=false;
> set hive.optimize.index.filter=true;
> set hive.stats.autogather=true;
> set hive.tez.bigtable.minsize.semijoin.reduction=1;
> set hive.tez.min.bloom.filter.entries=1;
> set hive.stats.fetch.column.stats=true;
> set hive.tez.bloom.filter.factor=1.0f;
> set hive.auto.convert.join=false;
> set hive.optimize.shared.work=false;
> create database tpch_test;
> use tpch_test;
> CREATE TABLE `customer`(
> `c_custkey` bigint,
> `c_name` string,
> `c_address` string,
> `c_nationkey` bigint,
> `c_phone` string,
> `c_acctbal` double,
> `c_mktsegment` string,
> `c_comment` string)
> ROW FORMAT SERDE
> 'org.apache.hadoop.hive.ql.io.orc.OrcSerde'
> STORED AS INPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'
> OUTPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat'
> TBLPROPERTIES (
> 'bucketing_version'='2',
> 'transactional'='true',
> 'transactional_properties'='default',
> 'transient_lastDdlTime'='1543026723');
> CREATE TABLE `lineitem`(
> `l_orderkey` bigint,
> `l_partkey` bigint,
> `l_suppkey` bigint,
> `l_linenumber` int,
> `l_quantity` double,
> `l_extendedprice` double,
> `l_discount` double,
> `l_tax` double,
> `l_returnflag` string,
> `l_linestatus` string,
> `l_shipdate` string,
> `l_commitdate` string,
> `l_receiptdate` string,
> `l_shipinstruct` string,
> `l_shipmode` string,
> `l_comment` string)
> ROW FORMAT SERDE
> 'org.apache.hadoop.hive.ql.io.orc.OrcSerde'
> STORED AS INPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'
> OUTPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat'
> TBLPROPERTIES (
> 'bucketing_version'='2',
> 'transactional'='true',
> 'transactional_properties'='default',
> 'transient_lastDdlTime'='1543027179');
> CREATE TABLE `orders`(
> `o_orderkey` bigint,
> `o_custkey` bigint,
> `o_orderstatus` string,
> `o_totalprice` double,
> `o_orderdate` string,
> `o_orderpriority` string,
> `o_clerk` string,
> `o_shippriority` int,
> `o_comment` string)
> ROW FORMAT SERDE
> 'org.apache.hadoop.hive.ql.io.orc.OrcSerde'
> STORED AS INPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'
> OUTPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat'
> TBLPROPERTIES (
> 'bucketing_version'='2',
> 'transactional'='true',
> 'transactional_properties'='default',
> 'transient_lastDdlTime'='1543026824');
> alter table customer update statistics
> set('numRows'='15000','rawDataSize'='8633707142');
> alter table lineitem update statistics
> set('numRows'='589709','rawDataSize'='184245066955');
> alter table orders update statistics
> set('numRows'='15','rawDataSize'='46741318253');
> create view q18_tmp_cached as
> select l_orderkey, sum(l_quantity) as t_sum_quantity
> from lineitem
> where l_orderkey is not null
> group by l_orderkey;
> -- Set bloom filter size to huge number so we get any possible semijoin
> reductions
> set hive.tez.min.bloom.filter.entries=0;
> set hive.tez.max.bloom.filter.entries=1;
> create table q18_large_volume_customer_cached stored as orc tblproperties
> ('transactional'='true', 'transactional_properties'='default') as
> select c_name, c_custkey, o_orderkey, o_orderdate, o_totalprice,
> sum(l_quantity)
> from customer, orders, q18_tmp_cached t, lineitem l
> where
> c_custkey = o_custkey and o_orderkey = t.l_orderkey
> and o_orderkey is not null and t.t_sum_quantity > 300
> and o_orderkey = l.l_orderkey and l.l_orderkey is not null
> group by c_name, c_custkey, o_orderkey, o_orderdate, o_totalprice
> order by o_totalprice desc, o_orderdate
> limit
[jira] [Updated] (HIVE-21132) Semi join edge is not being removed despite max bloomfilter entries set to 1
[
https://issues.apache.org/jira/browse/HIVE-21132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-21132:
---
Status: Patch Available (was: Open)
> Semi join edge is not being removed despite max bloomfilter entries set to 1
>
>
> Key: HIVE-21132
> URL: https://issues.apache.org/jira/browse/HIVE-21132
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Affects Versions: 4.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-21132.1.patch, HIVE-21132.2.patch
>
>
> * Reproducer
> {code:sql}
> --! qt:dataset:lineitem
> --! qt:dataset:part
> --! qt:dataset:src
> set hive.support.concurrency=true;
> set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
> --set hive.compute.query.using.stats=false;
> set hive.mapred.mode=nonstrict;
> set hive.explain.user=false;
> set hive.optimize.ppd=true;
> set hive.ppd.remove.duplicatefilters=true;
> set hive.tez.dynamic.partition.pruning=true;
> set hive.tez.dynamic.semijoin.reduction=true;
> set hive.optimize.metadataonly=false;
> set hive.optimize.index.filter=true;
> set hive.stats.autogather=true;
> set hive.tez.bigtable.minsize.semijoin.reduction=1;
> set hive.tez.min.bloom.filter.entries=1;
> set hive.stats.fetch.column.stats=true;
> set hive.tez.bloom.filter.factor=1.0f;
> set hive.auto.convert.join=false;
> set hive.optimize.shared.work=false;
> create database tpch_test;
> use tpch_test;
> CREATE TABLE `customer`(
> `c_custkey` bigint,
> `c_name` string,
> `c_address` string,
> `c_nationkey` bigint,
> `c_phone` string,
> `c_acctbal` double,
> `c_mktsegment` string,
> `c_comment` string)
> ROW FORMAT SERDE
> 'org.apache.hadoop.hive.ql.io.orc.OrcSerde'
> STORED AS INPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'
> OUTPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat'
> TBLPROPERTIES (
> 'bucketing_version'='2',
> 'transactional'='true',
> 'transactional_properties'='default',
> 'transient_lastDdlTime'='1543026723');
> CREATE TABLE `lineitem`(
> `l_orderkey` bigint,
> `l_partkey` bigint,
> `l_suppkey` bigint,
> `l_linenumber` int,
> `l_quantity` double,
> `l_extendedprice` double,
> `l_discount` double,
> `l_tax` double,
> `l_returnflag` string,
> `l_linestatus` string,
> `l_shipdate` string,
> `l_commitdate` string,
> `l_receiptdate` string,
> `l_shipinstruct` string,
> `l_shipmode` string,
> `l_comment` string)
> ROW FORMAT SERDE
> 'org.apache.hadoop.hive.ql.io.orc.OrcSerde'
> STORED AS INPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'
> OUTPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat'
> TBLPROPERTIES (
> 'bucketing_version'='2',
> 'transactional'='true',
> 'transactional_properties'='default',
> 'transient_lastDdlTime'='1543027179');
> CREATE TABLE `orders`(
> `o_orderkey` bigint,
> `o_custkey` bigint,
> `o_orderstatus` string,
> `o_totalprice` double,
> `o_orderdate` string,
> `o_orderpriority` string,
> `o_clerk` string,
> `o_shippriority` int,
> `o_comment` string)
> ROW FORMAT SERDE
> 'org.apache.hadoop.hive.ql.io.orc.OrcSerde'
> STORED AS INPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'
> OUTPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat'
> TBLPROPERTIES (
> 'bucketing_version'='2',
> 'transactional'='true',
> 'transactional_properties'='default',
> 'transient_lastDdlTime'='1543026824');
> alter table customer update statistics
> set('numRows'='15000','rawDataSize'='8633707142');
> alter table lineitem update statistics
> set('numRows'='589709','rawDataSize'='184245066955');
> alter table orders update statistics
> set('numRows'='15','rawDataSize'='46741318253');
> create view q18_tmp_cached as
> select l_orderkey, sum(l_quantity) as t_sum_quantity
> from lineitem
> where l_orderkey is not null
> group by l_orderkey;
> -- Set bloom filter size to huge number so we get any possible semijoin
> reductions
> set hive.tez.min.bloom.filter.entries=0;
> set hive.tez.max.bloom.filter.entries=1;
> create table q18_large_volume_customer_cached stored as orc tblproperties
> ('transactional'='true', 'transactional_properties'='default') as
> select c_name, c_custkey, o_orderkey, o_orderdate, o_totalprice,
> sum(l_quantity)
> from customer, orders, q18_tmp_cached t, lineitem l
> where
> c_custkey = o_custkey and o_orderkey = t.l_orderkey
> and o_orderkey is not null and t.t_sum_quantity > 300
> and o_orderkey = l.l_orderkey and l.l_orderkey is not null
> group by c_name, c_custkey, o_orderkey, o_orderdate, o_totalprice
> order by o_totalprice desc, o_orderdate
> limit
[jira] [Commented] (HIVE-21143) Add rewrite rules to open/close Between operators
[
https://issues.apache.org/jira/browse/HIVE-21143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750493#comment-16750493
]
Hive QA commented on HIVE-21143:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
54s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
38s{color} | {color:blue} ql in master has 2304 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
37s{color} | {color:red} ql: The patch generated 4 new + 119 unchanged - 0
fixed = 123 total (was 119) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 2 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch 14 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 38s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15765/dev-support/hive-personality.sh
|
| git revision | master / a7e704c |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15765/yetus/diff-checkstyle-ql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15765/yetus/whitespace-eol.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15765/yetus/whitespace-tabs.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15765/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Add rewrite rules to open/close Between operators
> -
>
> Key: HIVE-21143
> URL: https://issues.apache.org/jira/browse/HIVE-21143
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-21143.01.patch, HIVE-21143.02.patch,
> HIVE-21143.03.patch, HIVE-21143.03.patch, HIVE-21143.04.patch,
> HIVE-21143.05.patch
>
>
> During query compilation it's better to have BETWEEN statements in open form,
> as Calcite current not considering them during simplification.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-16924) Support distinct in presence Gby
[
https://issues.apache.org/jira/browse/HIVE-16924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Miklos Gergely reassigned HIVE-16924:
-
Assignee: Miklos Gergely (was: Julian Hyde)
> Support distinct in presence Gby
> -
>
> Key: HIVE-16924
> URL: https://issues.apache.org/jira/browse/HIVE-16924
> Project: Hive
> Issue Type: New Feature
> Components: Query Planning
>Reporter: Carter Shanklin
>Assignee: Miklos Gergely
>Priority: Major
> Attachments: HIVE-16924.01.patch, HIVE-16924.02.patch,
> HIVE-16924.03.patch, HIVE-16924.04.patch, HIVE-16924.05.patch,
> HIVE-16924.06.patch
>
>
> {code:sql}
> create table e011_01 (c1 int, c2 smallint);
> insert into e011_01 values (1, 1), (2, 2);
> {code}
> These queries should work:
> {code:sql}
> select distinct c1, count(*) from e011_01 group by c1;
> select distinct c1, avg(c2) from e011_01 group by c1;
> {code}
> Currently, you get :
> FAILED: SemanticException 1:52 SELECT DISTINCT and GROUP BY can not be in the
> same query. Error encountered near token 'c1'
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21001) Upgrade to calcite-1.18
[
https://issues.apache.org/jira/browse/HIVE-21001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749980#comment-16749980
]
Hive QA commented on HIVE-21001:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12955948/HIVE-21001.15.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 157 failed/errored test(s), 15695 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_predicate_pushdown]
(batchId=267)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ambiguitycheck]
(batchId=79)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[complex_alias]
(batchId=18)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[constant_prop_3]
(batchId=47)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[correlationoptimizer8]
(batchId=14)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[decimal_udf] (batchId=10)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[filter_numeric]
(batchId=87)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[fold_case] (batchId=16)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[fold_eq_with_case_when]
(batchId=88)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[fold_when] (batchId=30)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[in_typecheck_char]
(batchId=51)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[infer_const_type]
(batchId=74)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[infer_join_preds]
(batchId=26)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[innerjoin1] (batchId=26)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input8] (batchId=9)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join34] (batchId=76)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join35] (batchId=70)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join45] (batchId=21)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join47] (batchId=34)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[macro] (batchId=4)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin47] (batchId=64)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_memcheck]
(batchId=45)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[order3] (batchId=69)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_5]
(batchId=76)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_8]
(batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[partition_wise_fileformat2]
(batchId=94)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[pcr] (batchId=65)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[pcs] (batchId=54)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd2] (batchId=28)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_constant_expr]
(batchId=63)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_udf_case]
(batchId=47)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppr_allchildsarenull]
(batchId=75)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[rand_partitionpruner3]
(batchId=86)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[select_unquote_or]
(batchId=41)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[smb_mapjoin_47]
(batchId=31)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[subquery_notin_having]
(batchId=54)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[subquery_unqualcolumnrefs]
(batchId=19)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[timestamp_ints_casts]
(batchId=1)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_between] (batchId=78)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union_offcbo]
(batchId=49)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_between_columns]
(batchId=75)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_case_when_1]
(batchId=91)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_case_when_2]
(batchId=58)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_decimal_math_funcs]
(batchId=25)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_groupby_mapjoin]
(batchId=81)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorization_5]
(batchId=62)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorization_8]
(batchId=51)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorized_casts]
(batchId=89)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorized_math_funcs]
(batchId=22)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorized_string_funcs]
(batchId=63)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorized_timestamp_ints_casts]
(batchId=53)
[jira] [Commented] (HIVE-21001) Upgrade to calcite-1.18
[
https://issues.apache.org/jira/browse/HIVE-21001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749983#comment-16749983
]
Hive QA commented on HIVE-21001:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
31s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
58s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
31s{color} | {color:blue} common in master has 65 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
40s{color} | {color:blue} ql in master has 2305 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
26s{color} | {color:blue} accumulo-handler in master has 21 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
21s{color} | {color:blue} contrib in master has 10 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
28s{color} | {color:blue} hbase-handler in master has 15 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 7m
55s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
25s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 8m
1s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
40s{color} | {color:red} ql: The patch generated 4 new + 135 unchanged - 2
fixed = 139 total (was 137) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
1s{color} | {color:red} The patch has 3 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
1s{color} | {color:red} The patch 14 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
5s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 8m
3s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
28s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 72m 10s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile
xml |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15757/dev-support/hive-personality.sh
|
| git revision | master / 2ffca04 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15757/yetus/diff-checkstyle-ql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15757/yetus/whitespace-eol.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15757/yetus/whitespace-tabs.txt
|
| modules | C: common ql accumulo-handler contrib hbase-handler .
itests/hive-blobstore U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15757/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This
[jira] [Updated] (HIVE-21116) HADOOP_CREDSTORE_PASSWORD is not populated under yarn.app.mapreduce.am.admin.user.env
[ https://issues.apache.org/jira/browse/HIVE-21116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Vary updated HIVE-21116: -- Resolution: Fixed Fix Version/s: 4.0.0 Status: Resolved (was: Patch Available) Pushed to master. Thanks for the patch [~dkuzmenko]! > HADOOP_CREDSTORE_PASSWORD is not populated under > yarn.app.mapreduce.am.admin.user.env > -- > > Key: HIVE-21116 > URL: https://issues.apache.org/jira/browse/HIVE-21116 > Project: Hive > Issue Type: Bug >Reporter: Denys Kuzmenko >Assignee: Denys Kuzmenko >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-21116.1.patch, HIVE-21116.10.patch, > HIVE-21116.11.patch, HIVE-21116.12.patch, HIVE-21116.2.patch, > HIVE-21116.3.patch, HIVE-21116.4.patch, HIVE-21116.5.patch, > HIVE-21116.6.patch, HIVE-21116.7.patch, HIVE-21116.8.patch, HIVE-21116.9.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-17084) Turn on hive.stats.fetch.column.stats configuration flag
[
https://issues.apache.org/jira/browse/HIVE-17084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749720#comment-16749720
]
Hive QA commented on HIVE-17084:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
45s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
54s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
55s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
30s{color} | {color:blue} common in master has 65 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
34s{color} | {color:blue} ql in master has 2310 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
27s{color} | {color:blue} accumulo-handler in master has 21 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
21s{color} | {color:blue} contrib in master has 10 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
28s{color} | {color:blue} hbase-handler in master has 15 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 7m
58s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
23s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
4s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 7m
50s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
11s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 69m 58s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile
xml |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15749/dev-support/hive-personality.sh
|
| git revision | master / dfd63d9 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: common ql accumulo-handler contrib hbase-handler .
itests/hive-blobstore U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15749/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Turn on hive.stats.fetch.column.stats configuration flag
>
>
> Key: HIVE-17084
> URL: https://issues.apache.org/jira/browse/HIVE-17084
> Project: Hive
> Issue Type: Task
> Components: Statistics
>Reporter: Vineet Garg
>Assignee: Zoltan Haindrich
>Priority: Major
> Fix For: 3.2.0
>
> Attachments: HIVE-17084.08.patch, HIVE-17084.09.patch,
> HIVE-17084.1.patch,
[jira] [Updated] (HIVE-21153) task run in parallel cause ArrayIndexOutOfBoundsException
[ https://issues.apache.org/jira/browse/HIVE-21153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yx91490 updated HIVE-21153: --- Description: java.lang.ArrayIndexOutOfBoundsException: 22 at java.util.ArrayList.add(ArrayList.java:459) at org.apache.hadoop.hive.conf.HiveConf.setupRestrictList(HiveConf.java:3216) at org.apache.hadoop.hive.conf.HiveConf.addToRestrictList(HiveConf.java:3191) at org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:651) at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:547) at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:78) > task run in parallel cause ArrayIndexOutOfBoundsException > - > > Key: HIVE-21153 > URL: https://issues.apache.org/jira/browse/HIVE-21153 > Project: Hive > Issue Type: Bug >Reporter: yx91490 >Priority: Major > > java.lang.ArrayIndexOutOfBoundsException: 22 > at java.util.ArrayList.add(ArrayList.java:459) > at org.apache.hadoop.hive.conf.HiveConf.setupRestrictList(HiveConf.java:3216) > at org.apache.hadoop.hive.conf.HiveConf.addToRestrictList(HiveConf.java:3191) > at > org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:651) > at > org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:547) > at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:78) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21078) Replicate column and table level statistics for unpartitioned Hive tables
[ https://issues.apache.org/jira/browse/HIVE-21078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749807#comment-16749807 ] Sankar Hariappan commented on HIVE-21078: - 11.patch committed to master. Thanks [~ashutosh.bapat] for the contribution! > Replicate column and table level statistics for unpartitioned Hive tables > - > > Key: HIVE-21078 > URL: https://issues.apache.org/jira/browse/HIVE-21078 > Project: Hive > Issue Type: Sub-task >Reporter: Ashutosh Bapat >Assignee: Ashutosh Bapat >Priority: Major > Labels: pull-request-available > Attachments: HIVE-21078.01.patch, HIVE-21078.02.patch, > HIVE-21078.03.patch, HIVE-21078.04.patch, HIVE-21078.05.patch, > HIVE-21078.06.patch, HIVE-21078.07.patch, HIVE-21078.08.patch, > HIVE-21078.09.patch, HIVE-21078.10.patch, HIVE-21078.11.patch, > HIVE-21078.sameas.05.patch > > > This task is for replicating column and table level statistics for > unpartitioned tables. The same for partitioned tables will be worked upon in > a separate sub-task. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-16907) "INSERT INTO" overwrite old data when destination table encapsulated by backquote
[
https://issues.apache.org/jira/browse/HIVE-16907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Haindrich updated HIVE-16907:
Resolution: Fixed
Fix Version/s: 4.0.0
Status: Resolved (was: Patch Available)
pushed to master. Thank you [~jcamachorodriguez] for reviewing the changes!
> "INSERT INTO" overwrite old data when destination table encapsulated by
> backquote
>
>
> Key: HIVE-16907
> URL: https://issues.apache.org/jira/browse/HIVE-16907
> Project: Hive
> Issue Type: Bug
> Components: Parser
>Affects Versions: 1.1.0, 2.1.1
>Reporter: Nemon Lou
>Assignee: Zoltan Haindrich
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-16907.02.patch, HIVE-16907.03.patch,
> HIVE-16907.03.patch, HIVE-16907.03.patch, HIVE-16907.03.patch,
> HIVE-16907.1.patch
>
>
> A way to reproduce:
> {noformat}
> create database tdb;
> use tdb;
> create table t1(id int);
> create table t2(id int);
> explain insert into `tdb.t1` select * from t2;
> {noformat}
> {noformat}
> +---+
> |
> Explain |
> +---+
> | STAGE DEPENDENCIES:
> |
> | Stage-1 is a root stage
> |
> | Stage-6 depends on stages: Stage-1 , consists of Stage-3, Stage-2,
> Stage-4 |
> | Stage-3
> |
> | Stage-0 depends on stages: Stage-3, Stage-2, Stage-5
> |
> | Stage-2
> |
> | Stage-4
> |
> | Stage-5 depends on stages: Stage-4
> |
> |
> |
> | STAGE PLANS:
> |
> | Stage: Stage-1
> |
> | Map Reduce
> |
> | Map Operator Tree:
> |
> | TableScan
> |
> | alias: t2
> |
> | Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column
> stats: NONE |
> | Select Operator
> |
> | expressions: id (type: int)
> |
> | outputColumnNames: _col0
> |
> | Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column
> stats: NONE |
> | File Output Operator
> |
> |
[jira] [Updated] (HIVE-21153) task run in parallel cause ArrayIndexOutOfBoundsException
[ https://issues.apache.org/jira/browse/HIVE-21153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yx91490 updated HIVE-21153: --- Description: It happens occasionally when we run sql in parallel.the reason is HiveConf class is not thread safe, and SessionState.createSessionDirs() is not Synchronized. the error message: java.lang.ArrayIndexOutOfBoundsException: 22 at java.util.ArrayList.add(ArrayList.java:459) at org.apache.hadoop.hive.conf.HiveConf.setupRestrictList(HiveConf.java:3216) at org.apache.hadoop.hive.conf.HiveConf.addToRestrictList(HiveConf.java:3191) at org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:651) at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:547) at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:78) was: java.lang.ArrayIndexOutOfBoundsException: 22 at java.util.ArrayList.add(ArrayList.java:459) at org.apache.hadoop.hive.conf.HiveConf.setupRestrictList(HiveConf.java:3216) at org.apache.hadoop.hive.conf.HiveConf.addToRestrictList(HiveConf.java:3191) at org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:651) at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:547) at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:78) > task run in parallel cause ArrayIndexOutOfBoundsException > - > > Key: HIVE-21153 > URL: https://issues.apache.org/jira/browse/HIVE-21153 > Project: Hive > Issue Type: Bug >Reporter: yx91490 >Priority: Major > > It happens occasionally when we run sql in parallel.the reason is HiveConf > class is not thread safe, and SessionState.createSessionDirs() is not > Synchronized. > the error message: > java.lang.ArrayIndexOutOfBoundsException: 22 > at java.util.ArrayList.add(ArrayList.java:459) > at org.apache.hadoop.hive.conf.HiveConf.setupRestrictList(HiveConf.java:3216) > at org.apache.hadoop.hive.conf.HiveConf.addToRestrictList(HiveConf.java:3191) > at > org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:651) > at > org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:547) > at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:78) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21149) Refactor LlapServiceDriver
[ https://issues.apache.org/jira/browse/HIVE-21149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-21149: -- Attachment: HIVE-21149.02.patch > Refactor LlapServiceDriver > -- > > Key: HIVE-21149 > URL: https://issues.apache.org/jira/browse/HIVE-21149 > Project: Hive > Issue Type: Improvement > Components: Hive >Affects Versions: 3.1.2 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 3.1.2 > > Attachments: HIVE-21149.01.patch, HIVE-21149.02.patch > > > LlapServiceDriver is one monolith class doing several things, needs to be > refactor in order to make it clearer how it works. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21149) Refactor LlapServiceDriver
[ https://issues.apache.org/jira/browse/HIVE-21149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-21149: -- Status: Patch Available (was: Open) > Refactor LlapServiceDriver > -- > > Key: HIVE-21149 > URL: https://issues.apache.org/jira/browse/HIVE-21149 > Project: Hive > Issue Type: Improvement > Components: Hive >Affects Versions: 3.1.2 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 3.1.2 > > Attachments: HIVE-21149.01.patch, HIVE-21149.02.patch > > > LlapServiceDriver is one monolith class doing several things, needs to be > refactor in order to make it clearer how it works. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21149) Refactor LlapServiceDriver
[ https://issues.apache.org/jira/browse/HIVE-21149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-21149: -- Status: Open (was: Patch Available) > Refactor LlapServiceDriver > -- > > Key: HIVE-21149 > URL: https://issues.apache.org/jira/browse/HIVE-21149 > Project: Hive > Issue Type: Improvement > Components: Hive >Affects Versions: 3.1.2 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 3.1.2 > > Attachments: HIVE-21149.01.patch, HIVE-21149.02.patch > > > LlapServiceDriver is one monolith class doing several things, needs to be > refactor in order to make it clearer how it works. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-17084) Turn on hive.stats.fetch.column.stats configuration flag
[ https://issues.apache.org/jira/browse/HIVE-17084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749765#comment-16749765 ] Zoltan Haindrich commented on HIVE-17084: - [~ashutoshc], [~vgarg], [~jcamachorodriguez]: Could you please take a look? > Turn on hive.stats.fetch.column.stats configuration flag > > > Key: HIVE-17084 > URL: https://issues.apache.org/jira/browse/HIVE-17084 > Project: Hive > Issue Type: Task > Components: Statistics >Reporter: Vineet Garg >Assignee: Zoltan Haindrich >Priority: Major > Fix For: 3.2.0 > > Attachments: HIVE-17084.08.patch, HIVE-17084.09.patch, > HIVE-17084.1.patch, HIVE-17084.10.patch, HIVE-17084.11.patch, > HIVE-17084.12.patch, HIVE-17084.13.patch, HIVE-17084.14.patch, > HIVE-17084.15.patch, HIVE-17084.16.patch, HIVE-17084.17.patch, > HIVE-17084.18.patch, HIVE-17084.19.patch, HIVE-17084.20.patch, > HIVE-17084.21.patch, HIVE-17084.22.patch, HIVE-17084.23.patch, > HIVE-17084.24.patch, HIVE-17084.25.patch, HIVE-17084.26.patch, > HIVE-17084.27.patch, HIVE-17084.28.patch, HIVE-17084.29.patch, > HIVE-17084.30.patch, HIVE-17084.31.patch, HIVE-17084.31.patch, > HIVE-17084.31.patch, HIVE-170884.4.patch, HIVE-170884.5.patch, > HIVE-170884.7.patch > > > This flag is off by default and could result in bad plans due to missing > column statistics. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21153) task run in parallel cause ArrayIndexOutOfBoundsException
[ https://issues.apache.org/jira/browse/HIVE-21153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749790#comment-16749790 ] Zoltan Haindrich commented on HIVE-21153: - which version of Hive are you using? > task run in parallel cause ArrayIndexOutOfBoundsException > - > > Key: HIVE-21153 > URL: https://issues.apache.org/jira/browse/HIVE-21153 > Project: Hive > Issue Type: Bug >Reporter: yx91490 >Priority: Major > > It happens occasionally when we run sql in parallel.the reason is HiveConf > class is not thread safe, and SessionState.createSessionDirs() is not > Synchronized. > the error message: > java.lang.ArrayIndexOutOfBoundsException: 22 > at java.util.ArrayList.add(ArrayList.java:459) > at org.apache.hadoop.hive.conf.HiveConf.setupRestrictList(HiveConf.java:3216) > at org.apache.hadoop.hive.conf.HiveConf.addToRestrictList(HiveConf.java:3191) > at > org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:651) > at > org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:547) > at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:78) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21078) Replicate column and table level statistics for unpartitioned Hive tables
[ https://issues.apache.org/jira/browse/HIVE-21078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-21078: Component/s: repl > Replicate column and table level statistics for unpartitioned Hive tables > - > > Key: HIVE-21078 > URL: https://issues.apache.org/jira/browse/HIVE-21078 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 4.0.0 >Reporter: Ashutosh Bapat >Assignee: Ashutosh Bapat >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-21078.01.patch, HIVE-21078.02.patch, > HIVE-21078.03.patch, HIVE-21078.04.patch, HIVE-21078.05.patch, > HIVE-21078.06.patch, HIVE-21078.07.patch, HIVE-21078.08.patch, > HIVE-21078.09.patch, HIVE-21078.10.patch, HIVE-21078.11.patch, > HIVE-21078.sameas.05.patch > > > This task is for replicating column and table level statistics for > unpartitioned tables. The same for partitioned tables will be worked upon in > a separate sub-task. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21078) Replicate column and table level statistics for unpartitioned Hive tables
[ https://issues.apache.org/jira/browse/HIVE-21078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-21078: Fix Version/s: 4.0.0 > Replicate column and table level statistics for unpartitioned Hive tables > - > > Key: HIVE-21078 > URL: https://issues.apache.org/jira/browse/HIVE-21078 > Project: Hive > Issue Type: Sub-task >Reporter: Ashutosh Bapat >Assignee: Ashutosh Bapat >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-21078.01.patch, HIVE-21078.02.patch, > HIVE-21078.03.patch, HIVE-21078.04.patch, HIVE-21078.05.patch, > HIVE-21078.06.patch, HIVE-21078.07.patch, HIVE-21078.08.patch, > HIVE-21078.09.patch, HIVE-21078.10.patch, HIVE-21078.11.patch, > HIVE-21078.sameas.05.patch > > > This task is for replicating column and table level statistics for > unpartitioned tables. The same for partitioned tables will be worked upon in > a separate sub-task. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-17084) Turn on hive.stats.fetch.column.stats configuration flag
[
https://issues.apache.org/jira/browse/HIVE-17084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749718#comment-16749718
]
Hive QA commented on HIVE-17084:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12955903/HIVE-17084.31.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15701 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15749/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15749/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15749/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12955903 - PreCommit-HIVE-Build
> Turn on hive.stats.fetch.column.stats configuration flag
>
>
> Key: HIVE-17084
> URL: https://issues.apache.org/jira/browse/HIVE-17084
> Project: Hive
> Issue Type: Task
> Components: Statistics
>Reporter: Vineet Garg
>Assignee: Zoltan Haindrich
>Priority: Major
> Fix For: 3.2.0
>
> Attachments: HIVE-17084.08.patch, HIVE-17084.09.patch,
> HIVE-17084.1.patch, HIVE-17084.10.patch, HIVE-17084.11.patch,
> HIVE-17084.12.patch, HIVE-17084.13.patch, HIVE-17084.14.patch,
> HIVE-17084.15.patch, HIVE-17084.16.patch, HIVE-17084.17.patch,
> HIVE-17084.18.patch, HIVE-17084.19.patch, HIVE-17084.20.patch,
> HIVE-17084.21.patch, HIVE-17084.22.patch, HIVE-17084.23.patch,
> HIVE-17084.24.patch, HIVE-17084.25.patch, HIVE-17084.26.patch,
> HIVE-17084.27.patch, HIVE-17084.28.patch, HIVE-17084.29.patch,
> HIVE-17084.30.patch, HIVE-17084.31.patch, HIVE-17084.31.patch,
> HIVE-17084.31.patch, HIVE-170884.4.patch, HIVE-170884.5.patch,
> HIVE-170884.7.patch
>
>
> This flag is off by default and could result in bad plans due to missing
> column statistics.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21138) Fix some of the alerts raised by lgtm.com
[ https://issues.apache.org/jira/browse/HIVE-21138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-21138: Resolution: Fixed Fix Version/s: 4.0.0 Status: Resolved (was: Patch Available) pushed to master. Thank you [~malcolmt] for taking care of this! > Fix some of the alerts raised by lgtm.com > - > > Key: HIVE-21138 > URL: https://issues.apache.org/jira/browse/HIVE-21138 > Project: Hive > Issue Type: Improvement >Reporter: Malcolm Taylor >Assignee: Malcolm Taylor >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-21138.2.patch, HIVE-21138.patch > > > lgtm.com has identified a number of issues with the hive code base > ([https://lgtm.com/projects/g/apache/hive/alerts/?mode=tree).] This issue is > to address some of the more straightforward ones. Specifically, > * Array index out of bounds inOpProcFactory > * Dereferenced variable is always null in ObjectStore > * Self assignment in CheckConstraint and Vectorizer > * several cases of Contradictory type checks > * several cases of Missing format argument -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21138) Fix some of the alerts raised by lgtm.com
[ https://issues.apache.org/jira/browse/HIVE-21138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749742#comment-16749742 ] Malcolm Taylor commented on HIVE-21138: --- Thank you [~kgyrtkirk] for reviewing and merging. > Fix some of the alerts raised by lgtm.com > - > > Key: HIVE-21138 > URL: https://issues.apache.org/jira/browse/HIVE-21138 > Project: Hive > Issue Type: Improvement >Reporter: Malcolm Taylor >Assignee: Malcolm Taylor >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-21138.2.patch, HIVE-21138.patch > > > lgtm.com has identified a number of issues with the hive code base > ([https://lgtm.com/projects/g/apache/hive/alerts/?mode=tree).] This issue is > to address some of the more straightforward ones. Specifically, > * Array index out of bounds inOpProcFactory > * Dereferenced variable is always null in ObjectStore > * Self assignment in CheckConstraint and Vectorizer > * several cases of Contradictory type checks > * several cases of Missing format argument -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21079) Replicate column statistics for partitions of partitioned Hive table.
[
https://issues.apache.org/jira/browse/HIVE-21079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749788#comment-16749788
]
Hive QA commented on HIVE-21079:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
29s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
34s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
10s{color} | {color:blue} standalone-metastore/metastore-common in master has
29 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
0s{color} | {color:blue} standalone-metastore/metastore-server in master has
188 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
31s{color} | {color:blue} ql in master has 2310 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
25s{color} | {color:blue} hcatalog/server-extensions in master has 3 extant
Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
38s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
43s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
25s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
0s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
40s{color} | {color:red} ql: The patch generated 28 new + 784 unchanged - 6
fixed = 812 total (was 790) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
21s{color} | {color:red} itests/hive-unit: The patch generated 2 new + 771
unchanged - 0 fixed = 773 total (was 771) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 8 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 8m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
51s{color} | {color:green} metastore-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} standalone-metastore_metastore-server generated 0
new + 49 unchanged - 1 fixed = 49 total (was 50) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
52s{color} | {color:green} ql in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
12s{color} | {color:green} server-extensions in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
9s{color} | {color:green} tools-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
22s{color} | {color:green} hive-unit in the patch passed. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 44m 56s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1
[jira] [Commented] (HIVE-21079) Replicate column statistics for partitions of partitioned Hive table.
[
https://issues.apache.org/jira/browse/HIVE-21079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749808#comment-16749808
]
Hive QA commented on HIVE-21079:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12955904/HIVE-21079.01.patch
{color:green}SUCCESS:{color} +1 due to 8 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 15707 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_memcheck]
(batchId=45)
org.apache.hadoop.hive.metastore.TestFilterHooks.testHMSClientWithoutFilter
(batchId=220)
org.apache.hadoop.hive.metastore.TestFilterHooks.testHMSServerWithoutFilter
(batchId=220)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15750/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15750/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15750/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12955904 - PreCommit-HIVE-Build
> Replicate column statistics for partitions of partitioned Hive table.
> -
>
> Key: HIVE-21079
> URL: https://issues.apache.org/jira/browse/HIVE-21079
> Project: Hive
> Issue Type: Sub-task
>Reporter: Ashutosh Bapat
>Assignee: Ashutosh Bapat
>Priority: Major
> Attachments: HIVE-21079.01.patch
>
>
> This task is for replicating statistics for partitions of a partitioned Hive
> table.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21078) Replicate column and table level statistics for unpartitioned Hive tables
[ https://issues.apache.org/jira/browse/HIVE-21078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-21078: Resolution: Fixed Status: Resolved (was: Patch Available) > Replicate column and table level statistics for unpartitioned Hive tables > - > > Key: HIVE-21078 > URL: https://issues.apache.org/jira/browse/HIVE-21078 > Project: Hive > Issue Type: Sub-task >Reporter: Ashutosh Bapat >Assignee: Ashutosh Bapat >Priority: Major > Labels: pull-request-available > Attachments: HIVE-21078.01.patch, HIVE-21078.02.patch, > HIVE-21078.03.patch, HIVE-21078.04.patch, HIVE-21078.05.patch, > HIVE-21078.06.patch, HIVE-21078.07.patch, HIVE-21078.08.patch, > HIVE-21078.09.patch, HIVE-21078.10.patch, HIVE-21078.11.patch, > HIVE-21078.sameas.05.patch > > > This task is for replicating column and table level statistics for > unpartitioned tables. The same for partitioned tables will be worked upon in > a separate sub-task. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20758) Constraints: Show create table does not show constraints
[
https://issues.apache.org/jira/browse/HIVE-20758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Barnabas Maidics updated HIVE-20758:
Status: Open (was: Patch Available)
> Constraints: Show create table does not show constraints
>
>
> Key: HIVE-20758
> URL: https://issues.apache.org/jira/browse/HIVE-20758
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0
>Reporter: Gopal V
>Assignee: Barnabas Maidics
>Priority: Major
> Attachments: HIVE-20758.1.patch, Screen Shot 2019-01-23 at
> 11.52.04.png
>
>
> Even though the desc formatted shows the constraints, the show create table
> does not
> {code}
> | # Primary Key | NULL
> | NULL
>|
> | Table: |
> tpcds_bin_partitioned_orc_1.inventory | NULL
> |
> | Constraint Name: | pk_in
> | NULL
>|
> | Column Names: | inv_date_sk
> | inv_item_sk
>|
> || NULL
> | NULL
>|
> | # Foreign Keys | NULL
> | NULL
>|
> | Table: |
> tpcds_bin_partitioned_orc_1.inventory | NULL
> |
> | Constraint Name: | inv_d
> | NULL
>|
> | Parent Column Name:tpcds_bin_partitioned_orc_1.date_dim.d_date_sk |
> Column Name:inv_date_sk| Key Sequence:1
> |
> || NULL
> | NULL
>|
> | Constraint Name: | inv_i
> | NULL
>|
> | Parent Column Name:tpcds_bin_partitioned_orc_1.item.i_item_sk | Column
> Name:inv_item_sk| Key Sequence:1
>|
> || NULL
> | NULL
>|
> | Constraint Name: | inv_w
> | NULL
>|
> | Parent Column Name:tpcds_bin_partitioned_orc_1.warehouse.w_warehouse_sk
> | Column Name:inv_warehouse_sk | Key Sequence:1
> |
> || NULL
> | NULL
>|
> {code}
> But
> {code}
> ++
> | createtab_stmt |
> ++
> | CREATE TABLE `inventory`( |
> | `inv_item_sk` bigint,|
> | `inv_warehouse_sk` bigint, |
> | `inv_quantity_on_hand` int, |
> | `inv_date_sk` bigint)|
> | ROW FORMAT SERDE |
> | 'org.apache.hadoop.hive.ql.io.orc.OrcSerde' |
> | STORED AS INPUTFORMAT |
> | 'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat' |
> | OUTPUTFORMAT |
> | 'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat' |
> | LOCATION |
> |
> 'hdfs:///warehouse/tablespace/managed/hive/tpcds_bin_partitioned_orc_1.db/inventory'
> |
> | TBLPROPERTIES (|
> | 'bucketing_version'='2', |
> | 'transactional'='true', |
> | 'transactional_properties'='default',|
> | 'transient_lastDdlTime'='1539710410')|
> ++
> {code}
--
This message was sent by Atlassian JIRA
[jira] [Updated] (HIVE-20758) Constraints: Show create table does not show constraints
[
https://issues.apache.org/jira/browse/HIVE-20758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Barnabas Maidics updated HIVE-20758:
Attachment: HIVE-20758.1.patch
Status: Patch Available (was: Open)
Resubmitting the patch, since the Precommit test was triggered by an image :)
> Constraints: Show create table does not show constraints
>
>
> Key: HIVE-20758
> URL: https://issues.apache.org/jira/browse/HIVE-20758
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0
>Reporter: Gopal V
>Assignee: Barnabas Maidics
>Priority: Major
> Attachments: HIVE-20758.1.patch, Screen Shot 2019-01-23 at
> 11.52.04.png
>
>
> Even though the desc formatted shows the constraints, the show create table
> does not
> {code}
> | # Primary Key | NULL
> | NULL
>|
> | Table: |
> tpcds_bin_partitioned_orc_1.inventory | NULL
> |
> | Constraint Name: | pk_in
> | NULL
>|
> | Column Names: | inv_date_sk
> | inv_item_sk
>|
> || NULL
> | NULL
>|
> | # Foreign Keys | NULL
> | NULL
>|
> | Table: |
> tpcds_bin_partitioned_orc_1.inventory | NULL
> |
> | Constraint Name: | inv_d
> | NULL
>|
> | Parent Column Name:tpcds_bin_partitioned_orc_1.date_dim.d_date_sk |
> Column Name:inv_date_sk| Key Sequence:1
> |
> || NULL
> | NULL
>|
> | Constraint Name: | inv_i
> | NULL
>|
> | Parent Column Name:tpcds_bin_partitioned_orc_1.item.i_item_sk | Column
> Name:inv_item_sk| Key Sequence:1
>|
> || NULL
> | NULL
>|
> | Constraint Name: | inv_w
> | NULL
>|
> | Parent Column Name:tpcds_bin_partitioned_orc_1.warehouse.w_warehouse_sk
> | Column Name:inv_warehouse_sk | Key Sequence:1
> |
> || NULL
> | NULL
>|
> {code}
> But
> {code}
> ++
> | createtab_stmt |
> ++
> | CREATE TABLE `inventory`( |
> | `inv_item_sk` bigint,|
> | `inv_warehouse_sk` bigint, |
> | `inv_quantity_on_hand` int, |
> | `inv_date_sk` bigint)|
> | ROW FORMAT SERDE |
> | 'org.apache.hadoop.hive.ql.io.orc.OrcSerde' |
> | STORED AS INPUTFORMAT |
> | 'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat' |
> | OUTPUTFORMAT |
> | 'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat' |
> | LOCATION |
> |
> 'hdfs:///warehouse/tablespace/managed/hive/tpcds_bin_partitioned_orc_1.db/inventory'
> |
> | TBLPROPERTIES (|
> | 'bucketing_version'='2', |
> | 'transactional'='true', |
> | 'transactional_properties'='default',|
> | 'transient_lastDdlTime'='1539710410')|
>
[jira] [Updated] (HIVE-20758) Constraints: Show create table does not show constraints
[
https://issues.apache.org/jira/browse/HIVE-20758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Barnabas Maidics updated HIVE-20758:
Attachment: (was: HIVE-20758.1.patch)
> Constraints: Show create table does not show constraints
>
>
> Key: HIVE-20758
> URL: https://issues.apache.org/jira/browse/HIVE-20758
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0
>Reporter: Gopal V
>Assignee: Barnabas Maidics
>Priority: Major
> Attachments: HIVE-20758.1.patch, Screen Shot 2019-01-23 at
> 11.52.04.png
>
>
> Even though the desc formatted shows the constraints, the show create table
> does not
> {code}
> | # Primary Key | NULL
> | NULL
>|
> | Table: |
> tpcds_bin_partitioned_orc_1.inventory | NULL
> |
> | Constraint Name: | pk_in
> | NULL
>|
> | Column Names: | inv_date_sk
> | inv_item_sk
>|
> || NULL
> | NULL
>|
> | # Foreign Keys | NULL
> | NULL
>|
> | Table: |
> tpcds_bin_partitioned_orc_1.inventory | NULL
> |
> | Constraint Name: | inv_d
> | NULL
>|
> | Parent Column Name:tpcds_bin_partitioned_orc_1.date_dim.d_date_sk |
> Column Name:inv_date_sk| Key Sequence:1
> |
> || NULL
> | NULL
>|
> | Constraint Name: | inv_i
> | NULL
>|
> | Parent Column Name:tpcds_bin_partitioned_orc_1.item.i_item_sk | Column
> Name:inv_item_sk| Key Sequence:1
>|
> || NULL
> | NULL
>|
> | Constraint Name: | inv_w
> | NULL
>|
> | Parent Column Name:tpcds_bin_partitioned_orc_1.warehouse.w_warehouse_sk
> | Column Name:inv_warehouse_sk | Key Sequence:1
> |
> || NULL
> | NULL
>|
> {code}
> But
> {code}
> ++
> | createtab_stmt |
> ++
> | CREATE TABLE `inventory`( |
> | `inv_item_sk` bigint,|
> | `inv_warehouse_sk` bigint, |
> | `inv_quantity_on_hand` int, |
> | `inv_date_sk` bigint)|
> | ROW FORMAT SERDE |
> | 'org.apache.hadoop.hive.ql.io.orc.OrcSerde' |
> | STORED AS INPUTFORMAT |
> | 'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat' |
> | OUTPUTFORMAT |
> | 'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat' |
> | LOCATION |
> |
> 'hdfs:///warehouse/tablespace/managed/hive/tpcds_bin_partitioned_orc_1.db/inventory'
> |
> | TBLPROPERTIES (|
> | 'bucketing_version'='2', |
> | 'transactional'='true', |
> | 'transactional_properties'='default',|
> | 'transient_lastDdlTime'='1539710410')|
> ++
> {code}
--
This message was sent by Atlassian JIRA
[jira] [Updated] (HIVE-21149) Refactor LlapServiceDriver
[ https://issues.apache.org/jira/browse/HIVE-21149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-21149: -- Status: Open (was: Patch Available) > Refactor LlapServiceDriver > -- > > Key: HIVE-21149 > URL: https://issues.apache.org/jira/browse/HIVE-21149 > Project: Hive > Issue Type: Improvement > Components: Hive >Affects Versions: 3.1.2 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 3.1.2 > > Attachments: HIVE-21149.01.patch, HIVE-21149.02.patch, > HIVE-21149.03.patch > > > LlapServiceDriver is one monolith class doing several things, needs to be > refactor in order to make it clearer how it works. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21149) Refactor LlapServiceDriver
[ https://issues.apache.org/jira/browse/HIVE-21149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-21149: -- Attachment: HIVE-21149.03.patch > Refactor LlapServiceDriver > -- > > Key: HIVE-21149 > URL: https://issues.apache.org/jira/browse/HIVE-21149 > Project: Hive > Issue Type: Improvement > Components: Hive >Affects Versions: 3.1.2 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 3.1.2 > > Attachments: HIVE-21149.01.patch, HIVE-21149.02.patch, > HIVE-21149.03.patch > > > LlapServiceDriver is one monolith class doing several things, needs to be > refactor in order to make it clearer how it works. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21149) Refactor LlapServiceDriver
[ https://issues.apache.org/jira/browse/HIVE-21149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-21149: -- Status: Patch Available (was: Open) > Refactor LlapServiceDriver > -- > > Key: HIVE-21149 > URL: https://issues.apache.org/jira/browse/HIVE-21149 > Project: Hive > Issue Type: Improvement > Components: Hive >Affects Versions: 3.1.2 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 3.1.2 > > Attachments: HIVE-21149.01.patch, HIVE-21149.02.patch, > HIVE-21149.03.patch > > > LlapServiceDriver is one monolith class doing several things, needs to be > refactor in order to make it clearer how it works. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20758) Constraints: Show create table does not show constraints
[
https://issues.apache.org/jira/browse/HIVE-20758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Barnabas Maidics updated HIVE-20758:
Attachment: HIVE-20758.1.patch
Status: Patch Available (was: Open)
> Constraints: Show create table does not show constraints
>
>
> Key: HIVE-20758
> URL: https://issues.apache.org/jira/browse/HIVE-20758
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0
>Reporter: Gopal V
>Assignee: Barnabas Maidics
>Priority: Major
> Attachments: HIVE-20758.1.patch
>
>
> Even though the desc formatted shows the constraints, the show create table
> does not
> {code}
> | # Primary Key | NULL
> | NULL
>|
> | Table: |
> tpcds_bin_partitioned_orc_1.inventory | NULL
> |
> | Constraint Name: | pk_in
> | NULL
>|
> | Column Names: | inv_date_sk
> | inv_item_sk
>|
> || NULL
> | NULL
>|
> | # Foreign Keys | NULL
> | NULL
>|
> | Table: |
> tpcds_bin_partitioned_orc_1.inventory | NULL
> |
> | Constraint Name: | inv_d
> | NULL
>|
> | Parent Column Name:tpcds_bin_partitioned_orc_1.date_dim.d_date_sk |
> Column Name:inv_date_sk| Key Sequence:1
> |
> || NULL
> | NULL
>|
> | Constraint Name: | inv_i
> | NULL
>|
> | Parent Column Name:tpcds_bin_partitioned_orc_1.item.i_item_sk | Column
> Name:inv_item_sk| Key Sequence:1
>|
> || NULL
> | NULL
>|
> | Constraint Name: | inv_w
> | NULL
>|
> | Parent Column Name:tpcds_bin_partitioned_orc_1.warehouse.w_warehouse_sk
> | Column Name:inv_warehouse_sk | Key Sequence:1
> |
> || NULL
> | NULL
>|
> {code}
> But
> {code}
> ++
> | createtab_stmt |
> ++
> | CREATE TABLE `inventory`( |
> | `inv_item_sk` bigint,|
> | `inv_warehouse_sk` bigint, |
> | `inv_quantity_on_hand` int, |
> | `inv_date_sk` bigint)|
> | ROW FORMAT SERDE |
> | 'org.apache.hadoop.hive.ql.io.orc.OrcSerde' |
> | STORED AS INPUTFORMAT |
> | 'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat' |
> | OUTPUTFORMAT |
> | 'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat' |
> | LOCATION |
> |
> 'hdfs:///warehouse/tablespace/managed/hive/tpcds_bin_partitioned_orc_1.db/inventory'
> |
> | TBLPROPERTIES (|
> | 'bucketing_version'='2', |
> | 'transactional'='true', |
> | 'transactional_properties'='default',|
> | 'transient_lastDdlTime'='1539710410')|
> ++
> {code}
--
This message was sent by Atlassian JIRA
[jira] [Commented] (HIVE-21134) Hive Build Version as UDF
[ https://issues.apache.org/jira/browse/HIVE-21134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749822#comment-16749822 ] ASF GitHub Bot commented on HIVE-21134: --- GitHub user rmsmani opened a pull request: https://github.com/apache/hive/pull/521 Hive 21134 HIVE-21134 UDFBuildVersion source & test case You can merge this pull request into a Git repository by running: $ git pull https://github.com/rmsmani/hive HIVE-21134 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/hive/pull/521.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #521 commit c8ab66b17f32a6f70d379e3db3ebe9f402aa5a0e Author: Mani M Date: 2019-01-23T11:04:08Z HIVE-21134 UDFBuildVersion source commit d36bbf0c4a477f8cbea3a0418e2d01fa59a1deba Author: Mani M Date: 2019-01-23T11:05:02Z HIVE-21134 UDFBuildVersion TestCase > Hive Build Version as UDF > - > > Key: HIVE-21134 > URL: https://issues.apache.org/jira/browse/HIVE-21134 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0 >Reporter: Mani M >Assignee: Mani M >Priority: Minor > Labels: UDF, pull-request-available > Attachments: HIVE.21134.02.PATCH, HIVE.21134.03.PATCH, > HIVE.21134.04.PATCH, HIVE.21134.05.PATCH, HIVE.21134.06.PATCH, > HIVE.21134.07.PATCH, HIVE.21134.PATCH > > > This Jira is to get the Hive Build Version as UDF. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20758) Constraints: Show create table does not show constraints
[
https://issues.apache.org/jira/browse/HIVE-20758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Barnabas Maidics updated HIVE-20758:
Attachment: Screen Shot 2019-01-23 at 11.52.04.png
> Constraints: Show create table does not show constraints
>
>
> Key: HIVE-20758
> URL: https://issues.apache.org/jira/browse/HIVE-20758
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0
>Reporter: Gopal V
>Assignee: Barnabas Maidics
>Priority: Major
> Attachments: HIVE-20758.1.patch, Screen Shot 2019-01-23 at
> 11.52.04.png
>
>
> Even though the desc formatted shows the constraints, the show create table
> does not
> {code}
> | # Primary Key | NULL
> | NULL
>|
> | Table: |
> tpcds_bin_partitioned_orc_1.inventory | NULL
> |
> | Constraint Name: | pk_in
> | NULL
>|
> | Column Names: | inv_date_sk
> | inv_item_sk
>|
> || NULL
> | NULL
>|
> | # Foreign Keys | NULL
> | NULL
>|
> | Table: |
> tpcds_bin_partitioned_orc_1.inventory | NULL
> |
> | Constraint Name: | inv_d
> | NULL
>|
> | Parent Column Name:tpcds_bin_partitioned_orc_1.date_dim.d_date_sk |
> Column Name:inv_date_sk| Key Sequence:1
> |
> || NULL
> | NULL
>|
> | Constraint Name: | inv_i
> | NULL
>|
> | Parent Column Name:tpcds_bin_partitioned_orc_1.item.i_item_sk | Column
> Name:inv_item_sk| Key Sequence:1
>|
> || NULL
> | NULL
>|
> | Constraint Name: | inv_w
> | NULL
>|
> | Parent Column Name:tpcds_bin_partitioned_orc_1.warehouse.w_warehouse_sk
> | Column Name:inv_warehouse_sk | Key Sequence:1
> |
> || NULL
> | NULL
>|
> {code}
> But
> {code}
> ++
> | createtab_stmt |
> ++
> | CREATE TABLE `inventory`( |
> | `inv_item_sk` bigint,|
> | `inv_warehouse_sk` bigint, |
> | `inv_quantity_on_hand` int, |
> | `inv_date_sk` bigint)|
> | ROW FORMAT SERDE |
> | 'org.apache.hadoop.hive.ql.io.orc.OrcSerde' |
> | STORED AS INPUTFORMAT |
> | 'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat' |
> | OUTPUTFORMAT |
> | 'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat' |
> | LOCATION |
> |
> 'hdfs:///warehouse/tablespace/managed/hive/tpcds_bin_partitioned_orc_1.db/inventory'
> |
> | TBLPROPERTIES (|
> | 'bucketing_version'='2', |
> | 'transactional'='true', |
> | 'transactional_properties'='default',|
> | 'transient_lastDdlTime'='1539710410')|
> ++
> {code}
--
This message was sent by Atlassian
[jira] [Updated] (HIVE-21078) Replicate column and table level statistics for unpartitioned Hive tables
[ https://issues.apache.org/jira/browse/HIVE-21078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-21078: Affects Version/s: 4.0.0 > Replicate column and table level statistics for unpartitioned Hive tables > - > > Key: HIVE-21078 > URL: https://issues.apache.org/jira/browse/HIVE-21078 > Project: Hive > Issue Type: Sub-task >Affects Versions: 4.0.0 >Reporter: Ashutosh Bapat >Assignee: Ashutosh Bapat >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-21078.01.patch, HIVE-21078.02.patch, > HIVE-21078.03.patch, HIVE-21078.04.patch, HIVE-21078.05.patch, > HIVE-21078.06.patch, HIVE-21078.07.patch, HIVE-21078.08.patch, > HIVE-21078.09.patch, HIVE-21078.10.patch, HIVE-21078.11.patch, > HIVE-21078.sameas.05.patch > > > This task is for replicating column and table level statistics for > unpartitioned tables. The same for partitioned tables will be worked upon in > a separate sub-task. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20758) Constraints: Show create table does not show constraints
[
https://issues.apache.org/jira/browse/HIVE-20758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749824#comment-16749824
]
Barnabas Maidics commented on HIVE-20758:
-
Hi [~gopalv] ,
I created a solution for this. My patch does not add new functionality to
Metastore as [~vgarg] suggested but solves the problem.
The only limitation which I couldn't solve is that I think we don't store
additional keywords in Metastore, which belongs to a constraint (like DISABLE
or RELY), so I couldn't add it to the result. Do you think these should be
added to Metastore, or for now it's okay without them as well?
Otherwise, show create table now showing all the constraints (without
additional keywords) mentioned in the Jira in the following format:
!Screen Shot 2019-01-23 at 11.52.04.png!
> Constraints: Show create table does not show constraints
>
>
> Key: HIVE-20758
> URL: https://issues.apache.org/jira/browse/HIVE-20758
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0
>Reporter: Gopal V
>Assignee: Barnabas Maidics
>Priority: Major
> Attachments: HIVE-20758.1.patch, Screen Shot 2019-01-23 at
> 11.52.04.png
>
>
> Even though the desc formatted shows the constraints, the show create table
> does not
> {code}
> | # Primary Key | NULL
> | NULL
>|
> | Table: |
> tpcds_bin_partitioned_orc_1.inventory | NULL
> |
> | Constraint Name: | pk_in
> | NULL
>|
> | Column Names: | inv_date_sk
> | inv_item_sk
>|
> || NULL
> | NULL
>|
> | # Foreign Keys | NULL
> | NULL
>|
> | Table: |
> tpcds_bin_partitioned_orc_1.inventory | NULL
> |
> | Constraint Name: | inv_d
> | NULL
>|
> | Parent Column Name:tpcds_bin_partitioned_orc_1.date_dim.d_date_sk |
> Column Name:inv_date_sk| Key Sequence:1
> |
> || NULL
> | NULL
>|
> | Constraint Name: | inv_i
> | NULL
>|
> | Parent Column Name:tpcds_bin_partitioned_orc_1.item.i_item_sk | Column
> Name:inv_item_sk| Key Sequence:1
>|
> || NULL
> | NULL
>|
> | Constraint Name: | inv_w
> | NULL
>|
> | Parent Column Name:tpcds_bin_partitioned_orc_1.warehouse.w_warehouse_sk
> | Column Name:inv_warehouse_sk | Key Sequence:1
> |
> || NULL
> | NULL
>|
> {code}
> But
> {code}
> ++
> | createtab_stmt |
> ++
> | CREATE TABLE `inventory`( |
> | `inv_item_sk` bigint,|
> | `inv_warehouse_sk` bigint, |
> | `inv_quantity_on_hand` int, |
> | `inv_date_sk` bigint)|
> | ROW FORMAT SERDE |
> | 'org.apache.hadoop.hive.ql.io.orc.OrcSerde' |
> | STORED AS INPUTFORMAT |
> | 'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat' |
> | OUTPUTFORMAT
[jira] [Updated] (HIVE-21143) Add rewrite rules to open/close Between operators
[ https://issues.apache.org/jira/browse/HIVE-21143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-21143: Attachment: HIVE-21143.03.patch > Add rewrite rules to open/close Between operators > - > > Key: HIVE-21143 > URL: https://issues.apache.org/jira/browse/HIVE-21143 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-21143.01.patch, HIVE-21143.02.patch, > HIVE-21143.03.patch, HIVE-21143.03.patch > > > During query compilation it's better to have BETWEEN statements in open form, > as Calcite current not considering them during simplification. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21143) Add rewrite rules to open/close Between operators
[
https://issues.apache.org/jira/browse/HIVE-21143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749836#comment-16749836
]
Hive QA commented on HIVE-21143:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
40s{color} | {color:blue} ql in master has 2305 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
35s{color} | {color:red} ql: The patch generated 4 new + 119 unchanged - 0
fixed = 123 total (was 119) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 2 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch 14 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 21m 53s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15751/dev-support/hive-personality.sh
|
| git revision | master / 2ffca04 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15751/yetus/diff-checkstyle-ql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15751/yetus/whitespace-eol.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15751/yetus/whitespace-tabs.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15751/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Add rewrite rules to open/close Between operators
> -
>
> Key: HIVE-21143
> URL: https://issues.apache.org/jira/browse/HIVE-21143
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-21143.01.patch, HIVE-21143.02.patch,
> HIVE-21143.03.patch, HIVE-21143.03.patch
>
>
> During query compilation it's better to have BETWEEN statements in open form,
> as Calcite current not considering them during simplification.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20758) Constraints: Show create table does not show constraints
[
https://issues.apache.org/jira/browse/HIVE-20758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749887#comment-16749887
]
Hive QA commented on HIVE-20758:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12955945/Screen%20Shot%202019-01-23%20at%2011.52.04.png
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15754/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15754/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15754/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2019-01-23 12:15:37.812
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-15754/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2019-01-23 12:15:37.814
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 2ffca04 HIVE-21078: Replicate column and table level statistics
for unpartitioned Hive tables (Ashutosh Bapat, reviewed by Sankar Hariappan)
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 2ffca04 HIVE-21078: Replicate column and table level statistics
for unpartitioned Hive tables (Ashutosh Bapat, reviewed by Sankar Hariappan)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2019-01-23 12:15:38.374
+ rm -rf ../yetus_PreCommit-HIVE-Build-15754
+ mkdir ../yetus_PreCommit-HIVE-Build-15754
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-15754
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-15754/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
fatal: unrecognized input
fatal: unrecognized input
fatal: unrecognized input
The patch does not appear to apply with p0, p1, or p2
+ result=1
+ '[' 1 -ne 0 ']'
+ rm -rf yetus_PreCommit-HIVE-Build-15754
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12955945 - PreCommit-HIVE-Build
> Constraints: Show create table does not show constraints
>
>
> Key: HIVE-20758
> URL: https://issues.apache.org/jira/browse/HIVE-20758
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0
>Reporter: Gopal V
>Assignee: Barnabas Maidics
>Priority: Major
> Attachments: HIVE-20758.1.patch, Screen Shot 2019-01-23 at
> 11.52.04.png
>
>
> Even though the desc formatted shows the constraints, the show create table
> does not
> {code}
> | # Primary Key | NULL
> | NULL
>|
> | Table: |
> tpcds_bin_partitioned_orc_1.inventory | NULL
> |
> | Constraint Name: | pk_in
> | NULL
>|
> | Column Names: | inv_date_sk
> | inv_item_sk
>|
> || NULL
> | NULL
>|
> | # Foreign Keys | NULL
> | NULL
[jira] [Commented] (HIVE-21143) Add rewrite rules to open/close Between operators
[
https://issues.apache.org/jira/browse/HIVE-21143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749890#comment-16749890
]
Hive QA commented on HIVE-21143:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12955918/HIVE-21143.03.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15756/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15756/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15756/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12955918/HIVE-21143.03.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12955918 - PreCommit-HIVE-Build
> Add rewrite rules to open/close Between operators
> -
>
> Key: HIVE-21143
> URL: https://issues.apache.org/jira/browse/HIVE-21143
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-21143.01.patch, HIVE-21143.02.patch,
> HIVE-21143.03.patch
>
>
> During query compilation it's better to have BETWEEN statements in open form,
> as Calcite current not considering them during simplification.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-20758) Constraints: Show create table does not show constraints
[
https://issues.apache.org/jira/browse/HIVE-20758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Barnabas Maidics reassigned HIVE-20758:
---
Assignee: Barnabas Maidics
> Constraints: Show create table does not show constraints
>
>
> Key: HIVE-20758
> URL: https://issues.apache.org/jira/browse/HIVE-20758
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0
>Reporter: Gopal V
>Assignee: Barnabas Maidics
>Priority: Major
>
> Even though the desc formatted shows the constraints, the show create table
> does not
> {code}
> | # Primary Key | NULL
> | NULL
>|
> | Table: |
> tpcds_bin_partitioned_orc_1.inventory | NULL
> |
> | Constraint Name: | pk_in
> | NULL
>|
> | Column Names: | inv_date_sk
> | inv_item_sk
>|
> || NULL
> | NULL
>|
> | # Foreign Keys | NULL
> | NULL
>|
> | Table: |
> tpcds_bin_partitioned_orc_1.inventory | NULL
> |
> | Constraint Name: | inv_d
> | NULL
>|
> | Parent Column Name:tpcds_bin_partitioned_orc_1.date_dim.d_date_sk |
> Column Name:inv_date_sk| Key Sequence:1
> |
> || NULL
> | NULL
>|
> | Constraint Name: | inv_i
> | NULL
>|
> | Parent Column Name:tpcds_bin_partitioned_orc_1.item.i_item_sk | Column
> Name:inv_item_sk| Key Sequence:1
>|
> || NULL
> | NULL
>|
> | Constraint Name: | inv_w
> | NULL
>|
> | Parent Column Name:tpcds_bin_partitioned_orc_1.warehouse.w_warehouse_sk
> | Column Name:inv_warehouse_sk | Key Sequence:1
> |
> || NULL
> | NULL
>|
> {code}
> But
> {code}
> ++
> | createtab_stmt |
> ++
> | CREATE TABLE `inventory`( |
> | `inv_item_sk` bigint,|
> | `inv_warehouse_sk` bigint, |
> | `inv_quantity_on_hand` int, |
> | `inv_date_sk` bigint)|
> | ROW FORMAT SERDE |
> | 'org.apache.hadoop.hive.ql.io.orc.OrcSerde' |
> | STORED AS INPUTFORMAT |
> | 'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat' |
> | OUTPUTFORMAT |
> | 'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat' |
> | LOCATION |
> |
> 'hdfs:///warehouse/tablespace/managed/hive/tpcds_bin_partitioned_orc_1.db/inventory'
> |
> | TBLPROPERTIES (|
> | 'bucketing_version'='2', |
> | 'transactional'='true', |
> | 'transactional_properties'='default',|
> | 'transient_lastDdlTime'='1539710410')|
> ++
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21134) Hive Build Version as UDF
[ https://issues.apache.org/jira/browse/HIVE-21134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-21134: -- Labels: UDF pull-request-available (was: UDF) > Hive Build Version as UDF > - > > Key: HIVE-21134 > URL: https://issues.apache.org/jira/browse/HIVE-21134 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0 >Reporter: Mani M >Assignee: Mani M >Priority: Minor > Labels: UDF, pull-request-available > Attachments: HIVE.21134.02.PATCH, HIVE.21134.03.PATCH, > HIVE.21134.04.PATCH, HIVE.21134.05.PATCH, HIVE.21134.06.PATCH, > HIVE.21134.07.PATCH, HIVE.21134.PATCH > > > This Jira is to get the Hive Build Version as UDF. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21001) Upgrade to calcite-1.18
[ https://issues.apache.org/jira/browse/HIVE-21001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-21001: Attachment: HIVE-21001.15.patch > Upgrade to calcite-1.18 > --- > > Key: HIVE-21001 > URL: https://issues.apache.org/jira/browse/HIVE-21001 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-21001.01.patch, HIVE-21001.01.patch, > HIVE-21001.02.patch, HIVE-21001.03.patch, HIVE-21001.04.patch, > HIVE-21001.05.patch, HIVE-21001.06.patch, HIVE-21001.06.patch, > HIVE-21001.07.patch, HIVE-21001.08.patch, HIVE-21001.08.patch, > HIVE-21001.08.patch, HIVE-21001.09.patch, HIVE-21001.09.patch, > HIVE-21001.09.patch, HIVE-21001.10.patch, HIVE-21001.11.patch, > HIVE-21001.12.patch, HIVE-21001.13.patch, HIVE-21001.15.patch > > > CLEAR LIBRARY CACHE -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21143) Add rewrite rules to open/close Between operators
[ https://issues.apache.org/jira/browse/HIVE-21143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-21143: Attachment: (was: HIVE-21143.03.patch) > Add rewrite rules to open/close Between operators > - > > Key: HIVE-21143 > URL: https://issues.apache.org/jira/browse/HIVE-21143 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-21143.01.patch, HIVE-21143.02.patch, > HIVE-21143.03.patch > > > During query compilation it's better to have BETWEEN statements in open form, > as Calcite current not considering them during simplification. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21143) Add rewrite rules to open/close Between operators
[
https://issues.apache.org/jira/browse/HIVE-21143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749882#comment-16749882
]
Hive QA commented on HIVE-21143:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12955918/HIVE-21143.03.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15752/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15752/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15752/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12955918/HIVE-21143.03.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12955918 - PreCommit-HIVE-Build
> Add rewrite rules to open/close Between operators
> -
>
> Key: HIVE-21143
> URL: https://issues.apache.org/jira/browse/HIVE-21143
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-21143.01.patch, HIVE-21143.02.patch,
> HIVE-21143.03.patch
>
>
> During query compilation it's better to have BETWEEN statements in open form,
> as Calcite current not considering them during simplification.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21149) Refactor LlapServiceDriver
[
https://issues.apache.org/jira/browse/HIVE-21149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749884#comment-16749884
]
Hive QA commented on HIVE-21149:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12955934/HIVE-21149.02.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15753/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15753/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15753/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2019-01-23 12:14:24.391
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-15753/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2019-01-23 12:14:24.394
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 2ffca04 HIVE-21078: Replicate column and table level statistics
for unpartitioned Hive tables (Ashutosh Bapat, reviewed by Sankar Hariappan)
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 2ffca04 HIVE-21078: Replicate column and table level statistics
for unpartitioned Hive tables (Ashutosh Bapat, reviewed by Sankar Hariappan)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2019-01-23 12:14:25.508
+ rm -rf ../yetus_PreCommit-HIVE-Build-15753
+ mkdir ../yetus_PreCommit-HIVE-Build-15753
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-15753
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-15753/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error: a/bin/ext/llap.sh: does not exist in index
error:
a/llap-server/src/java/org/apache/hadoop/hive/llap/cli/LlapOptionsProcessor.java:
does not exist in index
error:
a/llap-server/src/java/org/apache/hadoop/hive/llap/cli/LlapServiceDriver.java:
does not exist in index
error:
a/llap-server/src/java/org/apache/hadoop/hive/llap/cli/LlapSliderUtils.java:
does not exist in index
error: patch failed:
llap-server/src/java/org/apache/hadoop/hive/llap/cli/LlapServiceDriver.java:1
error:
llap-server/src/java/org/apache/hadoop/hive/llap/cli/LlapServiceDriver.java:
patch does not apply
error: ext/llap.sh: does not exist in index
error: src/java/org/apache/hadoop/hive/llap/cli/LlapOptionsProcessor.java: does
not exist in index
error: src/java/org/apache/hadoop/hive/llap/cli/LlapServiceDriver.java: does
not exist in index
error: src/java/org/apache/hadoop/hive/llap/cli/LlapSliderUtils.java: does not
exist in index
The patch does not appear to apply with p0, p1, or p2
+ result=1
+ '[' 1 -ne 0 ']'
+ rm -rf yetus_PreCommit-HIVE-Build-15753
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12955934 - PreCommit-HIVE-Build
> Refactor LlapServiceDriver
> --
>
> Key: HIVE-21149
> URL: https://issues.apache.org/jira/browse/HIVE-21149
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Affects Versions: 3.1.2
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 3.1.2
>
> Attachments: HIVE-21149.01.patch, HIVE-21149.02.patch
>
>
> LlapServiceDriver is one monolith class doing several things, needs to be
> refactor in order to make it clearer how it works.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20758) Constraints: Show create table does not show constraints
[
https://issues.apache.org/jira/browse/HIVE-20758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749888#comment-16749888
]
Hive QA commented on HIVE-20758:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12955945/Screen%20Shot%202019-01-23%20at%2011.52.04.png
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15755/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15755/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15755/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12955945/Screen%20Shot%202019-01-23%20at%2011.52.04.png
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12955945 - PreCommit-HIVE-Build
> Constraints: Show create table does not show constraints
>
>
> Key: HIVE-20758
> URL: https://issues.apache.org/jira/browse/HIVE-20758
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0
>Reporter: Gopal V
>Assignee: Barnabas Maidics
>Priority: Major
> Attachments: HIVE-20758.1.patch, Screen Shot 2019-01-23 at
> 11.52.04.png
>
>
> Even though the desc formatted shows the constraints, the show create table
> does not
> {code}
> | # Primary Key | NULL
> | NULL
>|
> | Table: |
> tpcds_bin_partitioned_orc_1.inventory | NULL
> |
> | Constraint Name: | pk_in
> | NULL
>|
> | Column Names: | inv_date_sk
> | inv_item_sk
>|
> || NULL
> | NULL
>|
> | # Foreign Keys | NULL
> | NULL
>|
> | Table: |
> tpcds_bin_partitioned_orc_1.inventory | NULL
> |
> | Constraint Name: | inv_d
> | NULL
>|
> | Parent Column Name:tpcds_bin_partitioned_orc_1.date_dim.d_date_sk |
> Column Name:inv_date_sk| Key Sequence:1
> |
> || NULL
> | NULL
>|
> | Constraint Name: | inv_i
> | NULL
>|
> | Parent Column Name:tpcds_bin_partitioned_orc_1.item.i_item_sk | Column
> Name:inv_item_sk| Key Sequence:1
>|
> || NULL
> | NULL
>|
> | Constraint Name: | inv_w
> | NULL
>|
> | Parent Column Name:tpcds_bin_partitioned_orc_1.warehouse.w_warehouse_sk
> | Column Name:inv_warehouse_sk | Key Sequence:1
> |
> || NULL
> | NULL
>|
> {code}
> But
> {code}
> ++
> | createtab_stmt |
> ++
> | CREATE TABLE `inventory`( |
> | `inv_item_sk` bigint,|
> | `inv_warehouse_sk` bigint, |
> | `inv_quantity_on_hand` int, |
> | `inv_date_sk` bigint)|
> | ROW FORMAT SERDE
[jira] [Commented] (HIVE-21143) Add rewrite rules to open/close Between operators
[
https://issues.apache.org/jira/browse/HIVE-21143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749879#comment-16749879
]
Hive QA commented on HIVE-21143:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12955918/HIVE-21143.03.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 15709 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[partition_wise_fileformat2]
(batchId=94)
org.apache.hadoop.hive.cli.TestMiniDruidKafkaCliDriver.testCliDriver[druidkafkamini_basic]
(batchId=275)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15751/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15751/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15751/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12955918 - PreCommit-HIVE-Build
> Add rewrite rules to open/close Between operators
> -
>
> Key: HIVE-21143
> URL: https://issues.apache.org/jira/browse/HIVE-21143
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-21143.01.patch, HIVE-21143.02.patch,
> HIVE-21143.03.patch
>
>
> During query compilation it's better to have BETWEEN statements in open form,
> as Calcite current not considering them during simplification.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
