[jira] [Work logged] (HIVE-21173) Upgrade Apache Thrift to 0.9.3-1
[ https://issues.apache.org/jira/browse/HIVE-21173?focusedWorklogId=439386=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-439386 ] ASF GitHub Bot logged work on HIVE-21173: - Author: ASF GitHub Bot Created on: 01/Jun/20 07:09 Start Date: 01/Jun/20 07:09 Worklog Time Spent: 10m Work Description: dlavati closed pull request #730: URL: https://github.com/apache/hive/pull/730 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 439386) Time Spent: 50m (was: 40m) > Upgrade Apache Thrift to 0.9.3-1 > > > Key: HIVE-21173 > URL: https://issues.apache.org/jira/browse/HIVE-21173 > Project: Hive > Issue Type: Bug > Components: Thrift API >Reporter: James E. King III >Assignee: David Lavati >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-21173.01.patch > > Time Spent: 50m > Remaining Estimate: 0h > > The project currently depends on libthrift-0.9.3, however thrift released > 0.12.0 on 2019-JAN-04. This release includes a security fix for THRIFT-4506 > (CVE-2018-1320). Updating thrift to the latest version will remove that > vulnerability. > Also note the Apache Thrift project does not publish "libfb303" any longer. > fb303 is contributed code (in '/contrib') and it has not been maintained. > > Ps.: 0.9.3.1 also addresses the CVE, see THRIFT-4506 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-21173) Upgrade Apache Thrift to 0.9.3-1
[ https://issues.apache.org/jira/browse/HIVE-21173?focusedWorklogId=439387=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-439387 ] ASF GitHub Bot logged work on HIVE-21173: - Author: ASF GitHub Bot Created on: 01/Jun/20 07:09 Start Date: 01/Jun/20 07:09 Worklog Time Spent: 10m Work Description: dlavati commented on pull request #730: URL: https://github.com/apache/hive/pull/730#issuecomment-636655154 Merged in e1090a8aeb52a9e114f7053b77b4cfdec2329490 a while ago. See HIVE-21000 for any updates on newer versions. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 439387) Time Spent: 1h (was: 50m) > Upgrade Apache Thrift to 0.9.3-1 > > > Key: HIVE-21173 > URL: https://issues.apache.org/jira/browse/HIVE-21173 > Project: Hive > Issue Type: Bug > Components: Thrift API >Reporter: James E. King III >Assignee: David Lavati >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-21173.01.patch > > Time Spent: 1h > Remaining Estimate: 0h > > The project currently depends on libthrift-0.9.3, however thrift released > 0.12.0 on 2019-JAN-04. This release includes a security fix for THRIFT-4506 > (CVE-2018-1320). Updating thrift to the latest version will remove that > vulnerability. > Also note the Apache Thrift project does not publish "libfb303" any longer. > fb303 is contributed code (in '/contrib') and it has not been maintained. > > Ps.: 0.9.3.1 also addresses the CVE, see THRIFT-4506 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-21890) Fix alter_partition_change_col.q qtest inclusion in minillaplocal.query.files
[
https://issues.apache.org/jira/browse/HIVE-21890?focusedWorklogId=439389=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-439389
]
ASF GitHub Bot logged work on HIVE-21890:
-
Author: ASF GitHub Bot
Created on: 01/Jun/20 07:10
Start Date: 01/Jun/20 07:10
Worklog Time Spent: 10m
Work Description: dlavati closed pull request #676:
URL: https://github.com/apache/hive/pull/676
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 439389)
Time Spent: 0.5h (was: 20m)
> Fix alter_partition_change_col.q qtest inclusion in minillaplocal.query.files
> -
>
> Key: HIVE-21890
> URL: https://issues.apache.org/jira/browse/HIVE-21890
> Project: Hive
> Issue Type: Bug
>Reporter: David Lavati
>Assignee: David Lavati
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-21890.01.patch, HIVE-21890.01.patch
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> HIVE-20833 introduced alter_partition_change_col.q under
> minillaplocal.query.files, however it was named only
> {{alter_partition_change_col}} without the postfix.
> Looking at the recent precommit tests, it looks like this test never gets
> called.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-21890) Fix alter_partition_change_col.q qtest inclusion in minillaplocal.query.files
[
https://issues.apache.org/jira/browse/HIVE-21890?focusedWorklogId=439388=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-439388
]
ASF GitHub Bot logged work on HIVE-21890:
-
Author: ASF GitHub Bot
Created on: 01/Jun/20 07:10
Start Date: 01/Jun/20 07:10
Worklog Time Spent: 10m
Work Description: dlavati commented on pull request #676:
URL: https://github.com/apache/hive/pull/676#issuecomment-636655385
Merged in 959c430fcb60173ddad6d684ec2e7199c1f059f6
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 439388)
Time Spent: 20m (was: 10m)
> Fix alter_partition_change_col.q qtest inclusion in minillaplocal.query.files
> -
>
> Key: HIVE-21890
> URL: https://issues.apache.org/jira/browse/HIVE-21890
> Project: Hive
> Issue Type: Bug
>Reporter: David Lavati
>Assignee: David Lavati
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-21890.01.patch, HIVE-21890.01.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> HIVE-20833 introduced alter_partition_change_col.q under
> minillaplocal.query.files, however it was named only
> {{alter_partition_change_col}} without the postfix.
> Looking at the recent precommit tests, it looks like this test never gets
> called.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22308) Add missing support of Azure Blobstore schemes
[
https://issues.apache.org/jira/browse/HIVE-22308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17120835#comment-17120835
]
Hive QA commented on HIVE-22308:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12985752/HIVE-22308.02.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17219 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22699/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22699/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22699/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12985752 - PreCommit-HIVE-Build
> Add missing support of Azure Blobstore schemes
> --
>
> Key: HIVE-22308
> URL: https://issues.apache.org/jira/browse/HIVE-22308
> Project: Hive
> Issue Type: Improvement
>Reporter: David Lavati
>Assignee: David Lavati
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22308.02.patch, HIVE-22308.02.patch,
> HIVE-22308.patch
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Azure has been used as a filesystem for Hive, but its various schemes aren't
> registered under
> {{HiveConf.HIVE_BLOBSTORE_SUPPORTED_SCHEMES.}}
> Found the list of elements in:
> https://github.com/apache/hadoop/blob/trunk/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/constants/FileSystemUriSchemes.java
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23526) Beeline may throw the misleading exception
[ https://issues.apache.org/jira/browse/HIVE-23526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihua Deng updated HIVE-23526: --- Description: Sometimes we can see 'out of sequence response' message in beeline, for example: Error: org.apache.thrift.TApplicationException: CloseOperation failed: out of sequence response (state=08S01,code=0) java.sql.SQLException: org.apache.thrift.TApplicationException: CloseOperation failed: out of sequence response at org.apache.hive.jdbc.HiveStatement.closeClientOperation(HiveStatement.java:198) at org.apache.hive.jdbc.HiveStatement.close(HiveStatement.java:217) at org.apache.hive.beeline.Commands.execute(Commands.java:891) at org.apache.hive.beeline.Commands.sql(Commands.java:713) at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:976) at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:816) at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:774) at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:487) at org.apache.hive.beeline.BeeLine.main(BeeLine.java:470) and there is no other usage message to figured it out, even with --verbose, this makes problem puzzled as beeline does not have concurrency problem on underlying thrift transport. was: Sometimes we can see 'out of sequence response' message in beeline, for example: Error: org.apache.thrift.TApplicationException: CloseOperation failed: out of sequence response (state=08S01,code=0) java.sql.SQLException: org.apache.thrift.TApplicationException: CloseOperation failed: out of sequence response at org.apache.hive.jdbc.HiveStatement.closeClientOperation(HiveStatement.java:198) at org.apache.hive.jdbc.HiveStatement.close(HiveStatement.java:217) at org.apache.hive.beeline.Commands.execute(Commands.java:891) at org.apache.hive.beeline.Commands.sql(Commands.java:713) at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:976) at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:816) at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:774) at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:487) at org.apache.hive.beeline.BeeLine.main(BeeLine.java:470) and there is no other usage messages to figured it out, even with --verbose, this makes problem puzzled as beeline does not have concurrency problem on underlying thrift transport. > Beeline may throw the misleading exception > -- > > Key: HIVE-23526 > URL: https://issues.apache.org/jira/browse/HIVE-23526 > Project: Hive > Issue Type: Bug > Components: Beeline > Environment: Hive 1.2.2 >Reporter: Zhihua Deng >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23526.2.patch, HIVE-23526.3.patch, > HIVE-23526.patch, outofsequence.log > > Time Spent: 10m > Remaining Estimate: 0h > > Sometimes we can see 'out of sequence response' message in beeline, for > example: > Error: org.apache.thrift.TApplicationException: CloseOperation failed: out of > sequence response (state=08S01,code=0) > java.sql.SQLException: org.apache.thrift.TApplicationException: > CloseOperation failed: out of sequence response > at > org.apache.hive.jdbc.HiveStatement.closeClientOperation(HiveStatement.java:198) > at org.apache.hive.jdbc.HiveStatement.close(HiveStatement.java:217) > at org.apache.hive.beeline.Commands.execute(Commands.java:891) > at org.apache.hive.beeline.Commands.sql(Commands.java:713) > at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:976) > at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:816) > at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:774) > at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:487) > at org.apache.hive.beeline.BeeLine.main(BeeLine.java:470) > and there is no other usage message to figured it out, even with --verbose, > this makes problem puzzled as beeline does not have concurrency problem on > underlying thrift transport. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23516) Store hive replication policy execution metrics in the relational DB
[ https://issues.apache.org/jira/browse/HIVE-23516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-23516: --- Description: Details documented in the attached doc > Store hive replication policy execution metrics in the relational DB > > > Key: HIVE-23516 > URL: https://issues.apache.org/jira/browse/HIVE-23516 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Attachments: Replication Metrics.pdf > > > Details documented in the attached doc -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23585) Retrieve replication instance metrics details
[ https://issues.apache.org/jira/browse/HIVE-23585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-23585: --- Attachment: Replication Metrics.pdf > Retrieve replication instance metrics details > - > > Key: HIVE-23585 > URL: https://issues.apache.org/jira/browse/HIVE-23585 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Attachments: Replication Metrics.pdf > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23526) Beeline may throw the misleading exception
[ https://issues.apache.org/jira/browse/HIVE-23526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17120771#comment-17120771 ] Zhihua Deng commented on HIVE-23526: [~jcamachorodriguez] [~hashutosh] [~anishek] any thought about this? > Beeline may throw the misleading exception > -- > > Key: HIVE-23526 > URL: https://issues.apache.org/jira/browse/HIVE-23526 > Project: Hive > Issue Type: Bug > Components: Beeline > Environment: Hive 1.2.2 >Reporter: Zhihua Deng >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23526.2.patch, HIVE-23526.3.patch, > HIVE-23526.patch, outofsequence.log > > Time Spent: 10m > Remaining Estimate: 0h > > Sometimes we can see 'out of sequence response' message in beeline, for > example: > Error: org.apache.thrift.TApplicationException: CloseOperation failed: out of > sequence response (state=08S01,code=0) > java.sql.SQLException: org.apache.thrift.TApplicationException: > CloseOperation failed: out of sequence response > at > org.apache.hive.jdbc.HiveStatement.closeClientOperation(HiveStatement.java:198) > at org.apache.hive.jdbc.HiveStatement.close(HiveStatement.java:217) > at org.apache.hive.beeline.Commands.execute(Commands.java:891) > at org.apache.hive.beeline.Commands.sql(Commands.java:713) > at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:976) > at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:816) > at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:774) > at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:487) > at org.apache.hive.beeline.BeeLine.main(BeeLine.java:470) > and there is no other usage message to figured it out, even with --verbose, > this makes problem puzzled as beeline does not have concurrency problem on > underlying thrift transport. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22015) [CachedStore] Cache table constraints in CachedStore
[ https://issues.apache.org/jira/browse/HIVE-22015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17120780#comment-17120780 ] Adesh Kumar Rao commented on HIVE-22015: Discussed with [~kishendas] , picking this up. > [CachedStore] Cache table constraints in CachedStore > > > Key: HIVE-22015 > URL: https://issues.apache.org/jira/browse/HIVE-22015 > Project: Hive > Issue Type: Sub-task >Reporter: Daniel Dai >Priority: Major > > Currently table constraints are not cached. Hive will pull all constraints > from tables involved in query, which results multiple db reads (including > get_primary_keys, get_foreign_keys, get_unique_constraints, etc). The effort > to cache this is small as it's just another table component. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-22015) [CachedStore] Cache table constraints in CachedStore
[ https://issues.apache.org/jira/browse/HIVE-22015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adesh Kumar Rao reassigned HIVE-22015: -- Assignee: Adesh Kumar Rao > [CachedStore] Cache table constraints in CachedStore > > > Key: HIVE-22015 > URL: https://issues.apache.org/jira/browse/HIVE-22015 > Project: Hive > Issue Type: Sub-task >Reporter: Daniel Dai >Assignee: Adesh Kumar Rao >Priority: Major > > Currently table constraints are not cached. Hive will pull all constraints > from tables involved in query, which results multiple db reads (including > get_primary_keys, get_foreign_keys, get_unique_constraints, etc). The effort > to cache this is small as it's just another table component. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23570) [CachedStore ] Prewarm HMS cache during bootstrap with ValidWriteIdList for all the tables
[ https://issues.apache.org/jira/browse/HIVE-23570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17120779#comment-17120779 ] Adesh Kumar Rao commented on HIVE-23570: Discussed with [~kishendas] , picking this up. > [CachedStore ] Prewarm HMS cache during bootstrap with ValidWriteIdList for > all the tables > -- > > Key: HIVE-23570 > URL: https://issues.apache.org/jira/browse/HIVE-23570 > Project: Hive > Issue Type: Sub-task >Reporter: Kishen Das >Priority: Major > > Since we will be caching additional ValidWriteIdList for all the tables to > provide cache consistency, cache should be prewarmed during bootstrap. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-23570) [CachedStore ] Prewarm HMS cache during bootstrap with ValidWriteIdList for all the tables
[ https://issues.apache.org/jira/browse/HIVE-23570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adesh Kumar Rao reassigned HIVE-23570: -- Assignee: Adesh Kumar Rao > [CachedStore ] Prewarm HMS cache during bootstrap with ValidWriteIdList for > all the tables > -- > > Key: HIVE-23570 > URL: https://issues.apache.org/jira/browse/HIVE-23570 > Project: Hive > Issue Type: Sub-task >Reporter: Kishen Das >Assignee: Adesh Kumar Rao >Priority: Major > > Since we will be caching additional ValidWriteIdList for all the tables to > provide cache consistency, cache should be prewarmed during bootstrap. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-23571) [CachedStore] Add ValidWriteIdList to SharedCache.TableWrapper
[ https://issues.apache.org/jira/browse/HIVE-23571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adesh Kumar Rao reassigned HIVE-23571: -- Assignee: Adesh Kumar Rao > [CachedStore] Add ValidWriteIdList to SharedCache.TableWrapper > -- > > Key: HIVE-23571 > URL: https://issues.apache.org/jira/browse/HIVE-23571 > Project: Hive > Issue Type: Sub-task >Reporter: Kishen Das >Assignee: Adesh Kumar Rao >Priority: Major > > Add ValidWriteIdList to SharedCache.TableWrapper. This would be used in > deciding whether a given read request can be served from the cache or we have > to reload it from the backing database. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23571) [CachedStore] Add ValidWriteIdList to SharedCache.TableWrapper
[ https://issues.apache.org/jira/browse/HIVE-23571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17120777#comment-17120777 ] Adesh Kumar Rao commented on HIVE-23571: Discussed with [~kishendas] , picking this up. > [CachedStore] Add ValidWriteIdList to SharedCache.TableWrapper > -- > > Key: HIVE-23571 > URL: https://issues.apache.org/jira/browse/HIVE-23571 > Project: Hive > Issue Type: Sub-task >Reporter: Kishen Das >Priority: Major > > Add ValidWriteIdList to SharedCache.TableWrapper. This would be used in > deciding whether a given read request can be served from the cache or we have > to reload it from the backing database. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-22308) Add missing support of Azure Blobstore schemes
[
https://issues.apache.org/jira/browse/HIVE-22308?focusedWorklogId=439383=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-439383
]
ASF GitHub Bot logged work on HIVE-22308:
-
Author: ASF GitHub Bot
Created on: 01/Jun/20 07:07
Start Date: 01/Jun/20 07:07
Worklog Time Spent: 10m
Work Description: dlavati commented on pull request #805:
URL: https://github.com/apache/hive/pull/805#issuecomment-636653977
Closing this while blocked by HIVE-22916
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 439383)
Time Spent: 0.5h (was: 20m)
> Add missing support of Azure Blobstore schemes
> --
>
> Key: HIVE-22308
> URL: https://issues.apache.org/jira/browse/HIVE-22308
> Project: Hive
> Issue Type: Improvement
>Reporter: David Lavati
>Assignee: David Lavati
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22308.02.patch, HIVE-22308.02.patch,
> HIVE-22308.patch
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Azure has been used as a filesystem for Hive, but its various schemes aren't
> registered under
> {{HiveConf.HIVE_BLOBSTORE_SUPPORTED_SCHEMES.}}
> Found the list of elements in:
> https://github.com/apache/hadoop/blob/trunk/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/constants/FileSystemUriSchemes.java
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-22308) Add missing support of Azure Blobstore schemes
[
https://issues.apache.org/jira/browse/HIVE-22308?focusedWorklogId=439382=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-439382
]
ASF GitHub Bot logged work on HIVE-22308:
-
Author: ASF GitHub Bot
Created on: 01/Jun/20 07:07
Start Date: 01/Jun/20 07:07
Worklog Time Spent: 10m
Work Description: dlavati closed pull request #805:
URL: https://github.com/apache/hive/pull/805
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 439382)
Time Spent: 20m (was: 10m)
> Add missing support of Azure Blobstore schemes
> --
>
> Key: HIVE-22308
> URL: https://issues.apache.org/jira/browse/HIVE-22308
> Project: Hive
> Issue Type: Improvement
>Reporter: David Lavati
>Assignee: David Lavati
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22308.02.patch, HIVE-22308.02.patch,
> HIVE-22308.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Azure has been used as a filesystem for Hive, but its various schemes aren't
> registered under
> {{HiveConf.HIVE_BLOBSTORE_SUPPORTED_SCHEMES.}}
> Found the list of elements in:
> https://github.com/apache/hadoop/blob/trunk/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/constants/FileSystemUriSchemes.java
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-22243) Align Apache Thrift version to 0.9.3-1 in standalone-metastore as well
[ https://issues.apache.org/jira/browse/HIVE-22243?focusedWorklogId=439385=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-439385 ] ASF GitHub Bot logged work on HIVE-22243: - Author: ASF GitHub Bot Created on: 01/Jun/20 07:08 Start Date: 01/Jun/20 07:08 Worklog Time Spent: 10m Work Description: dlavati closed pull request #790: URL: https://github.com/apache/hive/pull/790 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 439385) Time Spent: 0.5h (was: 20m) > Align Apache Thrift version to 0.9.3-1 in standalone-metastore as well > -- > > Key: HIVE-22243 > URL: https://issues.apache.org/jira/browse/HIVE-22243 > Project: Hive > Issue Type: Bug >Reporter: David Lavati >Assignee: David Lavati >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-22243.patch > > Time Spent: 0.5h > Remaining Estimate: 0h > > Thrift was bumped to 0.9.3-1 in HIVE-21173, but standalone-metastore was left > out of this. > Thanks for pointing this out [~ashutosh.bapat]! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-22243) Align Apache Thrift version to 0.9.3-1 in standalone-metastore as well
[ https://issues.apache.org/jira/browse/HIVE-22243?focusedWorklogId=439384=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-439384 ] ASF GitHub Bot logged work on HIVE-22243: - Author: ASF GitHub Bot Created on: 01/Jun/20 07:08 Start Date: 01/Jun/20 07:08 Worklog Time Spent: 10m Work Description: dlavati commented on pull request #790: URL: https://github.com/apache/hive/pull/790#issuecomment-636654216 Merged in aacc83010b1c9dc3c5b68e6d75503bb72066456b This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 439384) Time Spent: 20m (was: 10m) > Align Apache Thrift version to 0.9.3-1 in standalone-metastore as well > -- > > Key: HIVE-22243 > URL: https://issues.apache.org/jira/browse/HIVE-22243 > Project: Hive > Issue Type: Bug >Reporter: David Lavati >Assignee: David Lavati >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-22243.patch > > Time Spent: 20m > Remaining Estimate: 0h > > Thrift was bumped to 0.9.3-1 in HIVE-21173, but standalone-metastore was left > out of this. > Thanks for pointing this out [~ashutosh.bapat]! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (HIVE-23526) Beeline may throw the misleading exception
[ https://issues.apache.org/jira/browse/HIVE-23526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17120771#comment-17120771 ] Zhihua Deng edited comment on HIVE-23526 at 6/1/20, 6:12 AM: - [~jcamachorodriguez] [~ashutoshc] [~anishek] any thought about this? was (Author: dengzh): [~jcamachorodriguez] [~hashutosh] [~anishek] any thought about this? > Beeline may throw the misleading exception > -- > > Key: HIVE-23526 > URL: https://issues.apache.org/jira/browse/HIVE-23526 > Project: Hive > Issue Type: Bug > Components: Beeline > Environment: Hive 1.2.2 >Reporter: Zhihua Deng >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23526.2.patch, HIVE-23526.3.patch, > HIVE-23526.patch, outofsequence.log > > Time Spent: 10m > Remaining Estimate: 0h > > Sometimes we can see 'out of sequence response' message in beeline, for > example: > Error: org.apache.thrift.TApplicationException: CloseOperation failed: out of > sequence response (state=08S01,code=0) > java.sql.SQLException: org.apache.thrift.TApplicationException: > CloseOperation failed: out of sequence response > at > org.apache.hive.jdbc.HiveStatement.closeClientOperation(HiveStatement.java:198) > at org.apache.hive.jdbc.HiveStatement.close(HiveStatement.java:217) > at org.apache.hive.beeline.Commands.execute(Commands.java:891) > at org.apache.hive.beeline.Commands.sql(Commands.java:713) > at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:976) > at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:816) > at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:774) > at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:487) > at org.apache.hive.beeline.BeeLine.main(BeeLine.java:470) > and there is no other usage message to figured it out, even with --verbose, > this makes problem puzzled as beeline does not have concurrency problem on > underlying thrift transport. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-21569) Bump guava version to 28.1-jre
[ https://issues.apache.org/jira/browse/HIVE-21569?focusedWorklogId=439380=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-439380 ] ASF GitHub Bot logged work on HIVE-21569: - Author: ASF GitHub Bot Created on: 01/Jun/20 07:04 Start Date: 01/Jun/20 07:04 Worklog Time Spent: 10m Work Description: dlavati closed pull request #861: URL: https://github.com/apache/hive/pull/861 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 439380) Time Spent: 20m (was: 10m) > Bump guava version to 28.1-jre > -- > > Key: HIVE-21569 > URL: https://issues.apache.org/jira/browse/HIVE-21569 > Project: Hive > Issue Type: Task > Components: Hive >Reporter: László Pintér >Assignee: László Bodor >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-21569.01.patch, HIVE-21569.02.patch, > HIVE-21569.03.patch, HIVE-21569.04.patch, HIVE-21569.05.patch > > Time Spent: 20m > Remaining Estimate: 0h > > Changes required to bump guava to 28.1-jre -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-22585) Clean up catalog/db/table name usage
[ https://issues.apache.org/jira/browse/HIVE-22585?focusedWorklogId=439379=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-439379 ] ASF GitHub Bot logged work on HIVE-22585: - Author: ASF GitHub Bot Created on: 01/Jun/20 07:03 Start Date: 01/Jun/20 07:03 Worklog Time Spent: 10m Work Description: dlavati closed pull request #876: URL: https://github.com/apache/hive/pull/876 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 439379) Time Spent: 1.5h (was: 1h 20m) > Clean up catalog/db/table name usage > > > Key: HIVE-22585 > URL: https://issues.apache.org/jira/browse/HIVE-22585 > Project: Hive > Issue Type: Sub-task >Reporter: David Lavati >Assignee: David Lavati >Priority: Major > Labels: pull-request-available, refactor > Attachments: HIVE-22585.01.patch, HIVE-22585.02.patch, > HIVE-22585.03.patch, HIVE-22585.04.patch, HIVE-22585.05.patch > > Time Spent: 1.5h > Remaining Estimate: 0h > > This is a followup to HIVE-21198 to address some additional improvement ideas > for the TableName object mentioned in > [https://github.com/apache/hive/pull/550] and attempt to remove all the fishy > usages of db/tablenames, as a number of places still rely on certain state > changes/black magic. > This change includes the following: > * Remove reliance on Utilities.getDbTableName > * Introduce a contract in TableName to always store catalog.dbname.tablename > in lowercase > * Make TableName a parsable type for ExplainTask > * Additional refactors to leverage the TableName object -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-22585) Clean up catalog/db/table name usage
[ https://issues.apache.org/jira/browse/HIVE-22585?focusedWorklogId=439378=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-439378 ] ASF GitHub Bot logged work on HIVE-22585: - Author: ASF GitHub Bot Created on: 01/Jun/20 07:03 Start Date: 01/Jun/20 07:03 Worklog Time Spent: 10m Work Description: dlavati commented on pull request #876: URL: https://github.com/apache/hive/pull/876#issuecomment-636652467 Whatever nice progress I had, currently I don't have the bandwidth to keep up with master and fix the remaining issues, so I'm closing this for now. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 439378) Time Spent: 1h 20m (was: 1h 10m) > Clean up catalog/db/table name usage > > > Key: HIVE-22585 > URL: https://issues.apache.org/jira/browse/HIVE-22585 > Project: Hive > Issue Type: Sub-task >Reporter: David Lavati >Assignee: David Lavati >Priority: Major > Labels: pull-request-available, refactor > Attachments: HIVE-22585.01.patch, HIVE-22585.02.patch, > HIVE-22585.03.patch, HIVE-22585.04.patch, HIVE-22585.05.patch > > Time Spent: 1h 20m > Remaining Estimate: 0h > > This is a followup to HIVE-21198 to address some additional improvement ideas > for the TableName object mentioned in > [https://github.com/apache/hive/pull/550] and attempt to remove all the fishy > usages of db/tablenames, as a number of places still rely on certain state > changes/black magic. > This change includes the following: > * Remove reliance on Utilities.getDbTableName > * Introduce a contract in TableName to always store catalog.dbname.tablename > in lowercase > * Make TableName a parsable type for ExplainTask > * Additional refactors to leverage the TableName object -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-22780) Upgrade slf4j version to 1.7.30
[ https://issues.apache.org/jira/browse/HIVE-22780?focusedWorklogId=439374=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-439374 ] ASF GitHub Bot logged work on HIVE-22780: - Author: ASF GitHub Bot Created on: 01/Jun/20 07:01 Start Date: 01/Jun/20 07:01 Worklog Time Spent: 10m Work Description: dlavati closed pull request #890: URL: https://github.com/apache/hive/pull/890 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 439374) Time Spent: 20m (was: 10m) > Upgrade slf4j version to 1.7.30 > --- > > Key: HIVE-22780 > URL: https://issues.apache.org/jira/browse/HIVE-22780 > Project: Hive > Issue Type: Improvement >Reporter: David Lavati >Assignee: David Lavati >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-22780.01.patch > > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-20359) Update protobuf version in pom.xml
[ https://issues.apache.org/jira/browse/HIVE-20359?focusedWorklogId=439373=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-439373 ] ASF GitHub Bot logged work on HIVE-20359: - Author: ASF GitHub Bot Created on: 01/Jun/20 07:01 Start Date: 01/Jun/20 07:01 Worklog Time Spent: 10m Work Description: dlavati closed pull request #920: URL: https://github.com/apache/hive/pull/920 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 439373) Time Spent: 0.5h (was: 20m) > Update protobuf version in pom.xml > -- > > Key: HIVE-20359 > URL: https://issues.apache.org/jira/browse/HIVE-20359 > Project: Hive > Issue Type: Improvement > Components: Hive >Affects Versions: 3.1.0 > Environment: ARM64 Server machines with CentOS/Debian OS. >Reporter: Naresh Bhat >Priority: Major > Labels: pull-request-available > Fix For: 3.1.0, 4.0.0 > > Attachments: HIVE-20359.01.branch-3.1.patch, > HIVE-20359.01.branch-3.1.patch, HIVE-20359.01.patch, HIVE-20359.01.patch, > HIVE-20359.01.patch > > Time Spent: 0.5h > Remaining Estimate: 0h > > The following protoc version's used in hive is very old i.e. 2.5.0 > [https://repo.maven.apache.org/maven2/com/google/protobuf/protoc/] . The > v2.5.0 does not have aarch64 support. But the AArch64 support started from > v3.5.0 on-words in google's protobuf project release. The hive release > version branch v3/3.0/3.1/master branch build fails on AArch64 because of old > protoc version 2.5.0, in which there is no AArch64 support/exe available. > After I replaced to latest available protoc version 3.6.1 the hive master > branch pom.xml files, The hive compiled without any issues on ARM64 machine. > This Jira ticket is used to propose a patch on Hive mailing list > [https://git.linaro.org/people/naresh.bhat/apache/hive.git/] to update > pom.xml files protobuf version. I will also cherry-pick patches for other > branches v3/3.0/3.1, and create a pull request. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-22780) Upgrade slf4j version to 1.7.30
[ https://issues.apache.org/jira/browse/HIVE-22780?focusedWorklogId=439375=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-439375 ] ASF GitHub Bot logged work on HIVE-22780: - Author: ASF GitHub Bot Created on: 01/Jun/20 07:01 Start Date: 01/Jun/20 07:01 Worklog Time Spent: 10m Work Description: dlavati commented on pull request #890: URL: https://github.com/apache/hive/pull/890#issuecomment-636651581 Merged in 7abc5f71d56628064caf16a22f72c6674ef40df2 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 439375) Time Spent: 0.5h (was: 20m) > Upgrade slf4j version to 1.7.30 > --- > > Key: HIVE-22780 > URL: https://issues.apache.org/jira/browse/HIVE-22780 > Project: Hive > Issue Type: Improvement >Reporter: David Lavati >Assignee: David Lavati >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-22780.01.patch > > Time Spent: 0.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-22653) Remove commons-lang leftovers
[ https://issues.apache.org/jira/browse/HIVE-22653?focusedWorklogId=439376=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-439376 ] ASF GitHub Bot logged work on HIVE-22653: - Author: ASF GitHub Bot Created on: 01/Jun/20 07:01 Start Date: 01/Jun/20 07:01 Worklog Time Spent: 10m Work Description: dlavati commented on pull request #886: URL: https://github.com/apache/hive/pull/886#issuecomment-636651737 Merged in 037eacea46371015a7f9894c5a9ccfb9708d5c56 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 439376) Time Spent: 40m (was: 0.5h) > Remove commons-lang leftovers > - > > Key: HIVE-22653 > URL: https://issues.apache.org/jira/browse/HIVE-22653 > Project: Hive > Issue Type: Bug >Reporter: David Lavati >Assignee: David Lavati >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-22653.01.patch, HIVE-22653.01.patch, > HIVE-22653.02.patch, HIVE-22653.03.patch, HIVE-22653.04.patch, > HIVE-22653.04.patch, HIVE-22653.04.patch, HIVE-22653.05.patch, > HIVE-22653.06.patch, HIVE-22653.07.patch, HIVE-22653.07.patch, > HIVE-22653.08.patch, HIVE-22653.08.patch > > Time Spent: 40m > Remaining Estimate: 0h > > HIVE-7145 removed commons-lang - in favor of commons-lang3 - as a direct > dependency, however a high number of files still refer to commons-lang, which > is transitively brought in either way. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-22653) Remove commons-lang leftovers
[ https://issues.apache.org/jira/browse/HIVE-22653?focusedWorklogId=439377=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-439377 ] ASF GitHub Bot logged work on HIVE-22653: - Author: ASF GitHub Bot Created on: 01/Jun/20 07:01 Start Date: 01/Jun/20 07:01 Worklog Time Spent: 10m Work Description: dlavati closed pull request #886: URL: https://github.com/apache/hive/pull/886 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 439377) Time Spent: 50m (was: 40m) > Remove commons-lang leftovers > - > > Key: HIVE-22653 > URL: https://issues.apache.org/jira/browse/HIVE-22653 > Project: Hive > Issue Type: Bug >Reporter: David Lavati >Assignee: David Lavati >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-22653.01.patch, HIVE-22653.01.patch, > HIVE-22653.02.patch, HIVE-22653.03.patch, HIVE-22653.04.patch, > HIVE-22653.04.patch, HIVE-22653.04.patch, HIVE-22653.05.patch, > HIVE-22653.06.patch, HIVE-22653.07.patch, HIVE-22653.07.patch, > HIVE-22653.08.patch, HIVE-22653.08.patch > > Time Spent: 50m > Remaining Estimate: 0h > > HIVE-7145 removed commons-lang - in favor of commons-lang3 - as a direct > dependency, however a high number of files still refer to commons-lang, which > is transitively brought in either way. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-20359) Update protobuf version in pom.xml
[ https://issues.apache.org/jira/browse/HIVE-20359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Lavati updated HIVE-20359: Status: Open (was: Patch Available) > Update protobuf version in pom.xml > -- > > Key: HIVE-20359 > URL: https://issues.apache.org/jira/browse/HIVE-20359 > Project: Hive > Issue Type: Improvement > Components: Hive >Affects Versions: 3.1.0 > Environment: ARM64 Server machines with CentOS/Debian OS. >Reporter: Naresh Bhat >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0, 3.1.0 > > Attachments: HIVE-20359.01.branch-3.1.patch, > HIVE-20359.01.branch-3.1.patch, HIVE-20359.01.patch, HIVE-20359.01.patch, > HIVE-20359.01.patch > > Time Spent: 10m > Remaining Estimate: 0h > > The following protoc version's used in hive is very old i.e. 2.5.0 > [https://repo.maven.apache.org/maven2/com/google/protobuf/protoc/] . The > v2.5.0 does not have aarch64 support. But the AArch64 support started from > v3.5.0 on-words in google's protobuf project release. The hive release > version branch v3/3.0/3.1/master branch build fails on AArch64 because of old > protoc version 2.5.0, in which there is no AArch64 support/exe available. > After I replaced to latest available protoc version 3.6.1 the hive master > branch pom.xml files, The hive compiled without any issues on ARM64 machine. > This Jira ticket is used to propose a patch on Hive mailing list > [https://git.linaro.org/people/naresh.bhat/apache/hive.git/] to update > pom.xml files protobuf version. I will also cherry-pick patches for other > branches v3/3.0/3.1, and create a pull request. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-20359) Update protobuf version in pom.xml
[ https://issues.apache.org/jira/browse/HIVE-20359?focusedWorklogId=439372=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-439372 ] ASF GitHub Bot logged work on HIVE-20359: - Author: ASF GitHub Bot Created on: 01/Jun/20 07:00 Start Date: 01/Jun/20 07:00 Worklog Time Spent: 10m Work Description: dlavati commented on pull request #920: URL: https://github.com/apache/hive/pull/920#issuecomment-636651412 Blocked by HBASE-22394 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 439372) Time Spent: 20m (was: 10m) > Update protobuf version in pom.xml > -- > > Key: HIVE-20359 > URL: https://issues.apache.org/jira/browse/HIVE-20359 > Project: Hive > Issue Type: Improvement > Components: Hive >Affects Versions: 3.1.0 > Environment: ARM64 Server machines with CentOS/Debian OS. >Reporter: Naresh Bhat >Priority: Major > Labels: pull-request-available > Fix For: 3.1.0, 4.0.0 > > Attachments: HIVE-20359.01.branch-3.1.patch, > HIVE-20359.01.branch-3.1.patch, HIVE-20359.01.patch, HIVE-20359.01.patch, > HIVE-20359.01.patch > > Time Spent: 20m > Remaining Estimate: 0h > > The following protoc version's used in hive is very old i.e. 2.5.0 > [https://repo.maven.apache.org/maven2/com/google/protobuf/protoc/] . The > v2.5.0 does not have aarch64 support. But the AArch64 support started from > v3.5.0 on-words in google's protobuf project release. The hive release > version branch v3/3.0/3.1/master branch build fails on AArch64 because of old > protoc version 2.5.0, in which there is no AArch64 support/exe available. > After I replaced to latest available protoc version 3.6.1 the hive master > branch pom.xml files, The hive compiled without any issues on ARM64 machine. > This Jira ticket is used to propose a patch on Hive mailing list > [https://git.linaro.org/people/naresh.bhat/apache/hive.git/] to update > pom.xml files protobuf version. I will also cherry-pick patches for other > branches v3/3.0/3.1, and create a pull request. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23088) Using Strings from log4j breaks non-log4j users
[
https://issues.apache.org/jira/browse/HIVE-23088?focusedWorklogId=439370=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-439370
]
ASF GitHub Bot logged work on HIVE-23088:
-
Author: ASF GitHub Bot
Created on: 01/Jun/20 06:57
Start Date: 01/Jun/20 06:57
Worklog Time Spent: 10m
Work Description: dlavati closed pull request #986:
URL: https://github.com/apache/hive/pull/986
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 439370)
Time Spent: 0.5h (was: 20m)
> Using Strings from log4j breaks non-log4j users
> ---
>
> Key: HIVE-23088
> URL: https://issues.apache.org/jira/browse/HIVE-23088
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.2
>Reporter: Vova Vysotskyi
>Assignee: David Lavati
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0, 3.2.0, 3.1.3
>
> Attachments: HIVE-23088.01.branch-3.patch,
> HIVE-23088.01.branch-3.patch, HIVE-23088.01.branch-3.patch,
> HIVE-23088.01.patch, HIVE-23088.01.patch
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> {{HookUtils}} uses explicitly {{org.apache.logging.log4j.util.Strings}} class
> from log4j, but it may break clients who use other loggers and should exclude
> log4j from the classpath.
> {{commons-lang}} has class {{StringUtils}} which may be used as a replacement
> for this one:
> {{Strings.isBlank}} -> {{StringUtils.isBlank}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-23088) Using Strings from log4j breaks non-log4j users
[
https://issues.apache.org/jira/browse/HIVE-23088?focusedWorklogId=439369=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-439369
]
ASF GitHub Bot logged work on HIVE-23088:
-
Author: ASF GitHub Bot
Created on: 01/Jun/20 06:57
Start Date: 01/Jun/20 06:57
Worklog Time Spent: 10m
Work Description: dlavati commented on pull request #986:
URL: https://github.com/apache/hive/pull/986#issuecomment-636650230
Merged in 014dafcd7ac4260f7038f969d7c8218682029b86
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 439369)
Time Spent: 20m (was: 10m)
> Using Strings from log4j breaks non-log4j users
> ---
>
> Key: HIVE-23088
> URL: https://issues.apache.org/jira/browse/HIVE-23088
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.2
>Reporter: Vova Vysotskyi
>Assignee: David Lavati
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0, 3.2.0, 3.1.3
>
> Attachments: HIVE-23088.01.branch-3.patch,
> HIVE-23088.01.branch-3.patch, HIVE-23088.01.branch-3.patch,
> HIVE-23088.01.patch, HIVE-23088.01.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> {{HookUtils}} uses explicitly {{org.apache.logging.log4j.util.Strings}} class
> from log4j, but it may break clients who use other loggers and should exclude
> log4j from the classpath.
> {{commons-lang}} has class {{StringUtils}} which may be used as a replacement
> for this one:
> {{Strings.isBlank}} -> {{StringUtils.isBlank}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23516) Store hive replication policy execution metrics in the relational DB
[ https://issues.apache.org/jira/browse/HIVE-23516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-23516: --- Summary: Store hive replication policy execution metrics in the relational DB (was: Store hive replication policy execution metrics) > Store hive replication policy execution metrics in the relational DB > > > Key: HIVE-23516 > URL: https://issues.apache.org/jira/browse/HIVE-23516 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-21569) Bump guava version to 28.1-jre
[ https://issues.apache.org/jira/browse/HIVE-21569?focusedWorklogId=439381=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-439381 ] ASF GitHub Bot logged work on HIVE-21569: - Author: ASF GitHub Bot Created on: 01/Jun/20 07:04 Start Date: 01/Jun/20 07:04 Worklog Time Spent: 10m Work Description: dlavati commented on pull request #861: URL: https://github.com/apache/hive/pull/861#issuecomment-636652944 Closing this as it's non-trivial and I don't have the bandwidth. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 439381) Time Spent: 0.5h (was: 20m) > Bump guava version to 28.1-jre > -- > > Key: HIVE-21569 > URL: https://issues.apache.org/jira/browse/HIVE-21569 > Project: Hive > Issue Type: Task > Components: Hive >Reporter: László Pintér >Assignee: László Bodor >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-21569.01.patch, HIVE-21569.02.patch, > HIVE-21569.03.patch, HIVE-21569.04.patch, HIVE-21569.05.patch > > Time Spent: 0.5h > Remaining Estimate: 0h > > Changes required to bump guava to 28.1-jre -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-21755) Backport HIVE-21462 to branch-3: Upgrading SQL server backed metastore when changing data type of a column with constraints
[ https://issues.apache.org/jira/browse/HIVE-21755?focusedWorklogId=439391=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-439391 ] ASF GitHub Bot logged work on HIVE-21755: - Author: ASF GitHub Bot Created on: 01/Jun/20 07:14 Start Date: 01/Jun/20 07:14 Worklog Time Spent: 10m Work Description: dlavati commented on pull request #638: URL: https://github.com/apache/hive/pull/638#issuecomment-636657419 Merged in 5f43c55f1e to branch-3.1 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 439391) Time Spent: 0.5h (was: 20m) > Backport HIVE-21462 to branch-3: Upgrading SQL server backed metastore when > changing data type of a column with constraints > --- > > Key: HIVE-21755 > URL: https://issues.apache.org/jira/browse/HIVE-21755 > Project: Hive > Issue Type: Bug > Components: Metastore, Standalone Metastore >Affects Versions: 3.1.1 >Reporter: David Lavati >Assignee: David Lavati >Priority: Major > Labels: pull-request-available > Fix For: 3.2.0, 3.1.2 > > Attachments: HIVE-21755.01.branch-3.patch > > Time Spent: 0.5h > Remaining Estimate: 0h > > Backport of HIVE-21462 to branch-3. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-21755) Backport HIVE-21462 to branch-3: Upgrading SQL server backed metastore when changing data type of a column with constraints
[ https://issues.apache.org/jira/browse/HIVE-21755?focusedWorklogId=439390=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-439390 ] ASF GitHub Bot logged work on HIVE-21755: - Author: ASF GitHub Bot Created on: 01/Jun/20 07:14 Start Date: 01/Jun/20 07:14 Worklog Time Spent: 10m Work Description: dlavati closed pull request #638: URL: https://github.com/apache/hive/pull/638 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 439390) Time Spent: 20m (was: 10m) > Backport HIVE-21462 to branch-3: Upgrading SQL server backed metastore when > changing data type of a column with constraints > --- > > Key: HIVE-21755 > URL: https://issues.apache.org/jira/browse/HIVE-21755 > Project: Hive > Issue Type: Bug > Components: Metastore, Standalone Metastore >Affects Versions: 3.1.1 >Reporter: David Lavati >Assignee: David Lavati >Priority: Major > Labels: pull-request-available > Fix For: 3.2.0, 3.1.2 > > Attachments: HIVE-21755.01.branch-3.patch > > Time Spent: 20m > Remaining Estimate: 0h > > Backport of HIVE-21462 to branch-3. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (HIVE-23526) Beeline may throw the misleading exception
[ https://issues.apache.org/jira/browse/HIVE-23526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113082#comment-17113082 ] Zhihua Deng edited comment on HIVE-23526 at 6/1/20, 6:07 AM: - The problem can be reproduced when there is not enough memory for thrift to allocate the message or unsuccessfully read response from server(Read timeout) , if the client now issues ResultSet::close or HiveStatement::close in finally block, the client may get the result of the last request, which causing out of sequence response problem. As beeline does not process the root cause properly, only out of sequence shows at front. was (Author: dengzh): The problem can be reproduced when there is not enough memory for thrift to allocate the message, an oom error throws as a consequence, if the client now issues ResultSet::close or HiveStatement::close in finally block, the response returned should be the result of the last request, which causing out of sequence response problem. As the beeline does not process the oom error properly, only out of sequence shows at front. > Beeline may throw the misleading exception > -- > > Key: HIVE-23526 > URL: https://issues.apache.org/jira/browse/HIVE-23526 > Project: Hive > Issue Type: Bug > Components: Beeline > Environment: Hive 1.2.2 >Reporter: Zhihua Deng >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23526.2.patch, HIVE-23526.3.patch, > HIVE-23526.patch, outofsequence.log > > Time Spent: 10m > Remaining Estimate: 0h > > Sometimes we can see 'out of sequence response' message in beeline, for > example: > Error: org.apache.thrift.TApplicationException: CloseOperation failed: out of > sequence response (state=08S01,code=0) > java.sql.SQLException: org.apache.thrift.TApplicationException: > CloseOperation failed: out of sequence response > at > org.apache.hive.jdbc.HiveStatement.closeClientOperation(HiveStatement.java:198) > at org.apache.hive.jdbc.HiveStatement.close(HiveStatement.java:217) > at org.apache.hive.beeline.Commands.execute(Commands.java:891) > at org.apache.hive.beeline.Commands.sql(Commands.java:713) > at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:976) > at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:816) > at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:774) > at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:487) > at org.apache.hive.beeline.BeeLine.main(BeeLine.java:470) > and there is no other usage message to figured it out, even with --verbose, > this makes problem puzzled as beeline does not have concurrency problem on > underlying thrift transport. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-21404) MSSQL upgrade script alters the wrong column
[
https://issues.apache.org/jira/browse/HIVE-21404?focusedWorklogId=439395=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-439395
]
ASF GitHub Bot logged work on HIVE-21404:
-
Author: ASF GitHub Bot
Created on: 01/Jun/20 07:16
Start Date: 01/Jun/20 07:16
Worklog Time Spent: 10m
Work Description: dlavati commented on pull request #560:
URL: https://github.com/apache/hive/pull/560#issuecomment-636658260
Merged in 78f85137f778330d3881e4291340f97427b04a34
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 439395)
Time Spent: 40m (was: 0.5h)
> MSSQL upgrade script alters the wrong column
>
>
> Key: HIVE-21404
> URL: https://issues.apache.org/jira/browse/HIVE-21404
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.2.0
>Reporter: David Lavati
>Assignee: David Lavati
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-21404.1.patch, HIVE-21404.2.patch,
> HIVE-21404.3.patch, HIVE-21404.4.patch, HIVE-21404.4.patch,
> HIVE-21404.4.patch, HIVE-21404.4.patch, HIVE-21404.4.patch,
> HIVE-21404.4.patch, HIVE-21404.4.patch, HIVE-21404.4.patch, HIVE-21404.4.patch
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> HIVE-20221 changes PARTITION_PARAMS, so the following command is modifying
> the wrong table:
> {{ALTER TABLE "SERDE_PARAMS" ALTER COLUMN "PARAM_VALUE" nvarchar(MAX);}}
> https://github.com/apache/hive/blob/d3b036920acde7bb04840697eb13038103b062b4/standalone-metastore/metastore-server/src/main/sql/mssql/upgrade-3.1.0-to-3.2.0.mssql.sql#L21
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-21404) MSSQL upgrade script alters the wrong column
[
https://issues.apache.org/jira/browse/HIVE-21404?focusedWorklogId=439396=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-439396
]
ASF GitHub Bot logged work on HIVE-21404:
-
Author: ASF GitHub Bot
Created on: 01/Jun/20 07:16
Start Date: 01/Jun/20 07:16
Worklog Time Spent: 10m
Work Description: dlavati closed pull request #560:
URL: https://github.com/apache/hive/pull/560
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 439396)
Time Spent: 50m (was: 40m)
> MSSQL upgrade script alters the wrong column
>
>
> Key: HIVE-21404
> URL: https://issues.apache.org/jira/browse/HIVE-21404
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.2.0
>Reporter: David Lavati
>Assignee: David Lavati
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-21404.1.patch, HIVE-21404.2.patch,
> HIVE-21404.3.patch, HIVE-21404.4.patch, HIVE-21404.4.patch,
> HIVE-21404.4.patch, HIVE-21404.4.patch, HIVE-21404.4.patch,
> HIVE-21404.4.patch, HIVE-21404.4.patch, HIVE-21404.4.patch, HIVE-21404.4.patch
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> HIVE-20221 changes PARTITION_PARAMS, so the following command is modifying
> the wrong table:
> {{ALTER TABLE "SERDE_PARAMS" ALTER COLUMN "PARAM_VALUE" nvarchar(MAX);}}
> https://github.com/apache/hive/blob/d3b036920acde7bb04840697eb13038103b062b4/standalone-metastore/metastore-server/src/main/sql/mssql/upgrade-3.1.0-to-3.2.0.mssql.sql#L21
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-21741) Backport HIVE-20221 & related fix HIVE-20833 to branch-3: Increase column width for partition_params
[ https://issues.apache.org/jira/browse/HIVE-21741?focusedWorklogId=439393=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-439393 ] ASF GitHub Bot logged work on HIVE-21741: - Author: ASF GitHub Bot Created on: 01/Jun/20 07:16 Start Date: 01/Jun/20 07:16 Worklog Time Spent: 10m Work Description: dlavati closed pull request #637: URL: https://github.com/apache/hive/pull/637 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 439393) Time Spent: 20m (was: 10m) > Backport HIVE-20221 & related fix HIVE-20833 to branch-3: Increase column > width for partition_params > > > Key: HIVE-21741 > URL: https://issues.apache.org/jira/browse/HIVE-21741 > Project: Hive > Issue Type: Bug > Components: Metastore, Standalone Metastore >Affects Versions: 3.1.1 >Reporter: David Lavati >Assignee: David Lavati >Priority: Major > Labels: pull-request-available > Fix For: 3.2.0 > > Attachments: HIVE-21741.01.branch-3.patch, > HIVE-21741.01.branch-3.patch, HIVE-21741.02.branch-3.patch, > HIVE-21741.branch-3.patch > > Time Spent: 20m > Remaining Estimate: 0h > > This is an umbrella for backporting HIVE-20221 & the related fix of > HIVE-20833 to branch-3. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-21741) Backport HIVE-20221 & related fix HIVE-20833 to branch-3: Increase column width for partition_params
[ https://issues.apache.org/jira/browse/HIVE-21741?focusedWorklogId=439394=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-439394 ] ASF GitHub Bot logged work on HIVE-21741: - Author: ASF GitHub Bot Created on: 01/Jun/20 07:16 Start Date: 01/Jun/20 07:16 Worklog Time Spent: 10m Work Description: dlavati commented on pull request #637: URL: https://github.com/apache/hive/pull/637#issuecomment-636658075 Merged in e3d5abda9118a5de4d6ef9276438bf9ea1a81ab5 to branch-3 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 439394) Time Spent: 0.5h (was: 20m) > Backport HIVE-20221 & related fix HIVE-20833 to branch-3: Increase column > width for partition_params > > > Key: HIVE-21741 > URL: https://issues.apache.org/jira/browse/HIVE-21741 > Project: Hive > Issue Type: Bug > Components: Metastore, Standalone Metastore >Affects Versions: 3.1.1 >Reporter: David Lavati >Assignee: David Lavati >Priority: Major > Labels: pull-request-available > Fix For: 3.2.0 > > Attachments: HIVE-21741.01.branch-3.patch, > HIVE-21741.01.branch-3.patch, HIVE-21741.02.branch-3.patch, > HIVE-21741.branch-3.patch > > Time Spent: 0.5h > Remaining Estimate: 0h > > This is an umbrella for backporting HIVE-20221 & the related fix of > HIVE-20833 to branch-3. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-23585) Retrieve replication instance metrics details
[ https://issues.apache.org/jira/browse/HIVE-23585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi reassigned HIVE-23585: -- > Retrieve replication instance metrics details > - > > Key: HIVE-23585 > URL: https://issues.apache.org/jira/browse/HIVE-23585 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23516) Store hive replication policy execution metrics in the relational DB
[ https://issues.apache.org/jira/browse/HIVE-23516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-23516: --- Attachment: Replication Metrics.pdf > Store hive replication policy execution metrics in the relational DB > > > Key: HIVE-23516 > URL: https://issues.apache.org/jira/browse/HIVE-23516 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Attachments: Replication Metrics.pdf > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23498) Disable HTTP Trace method on ThriftHttpCliService
[ https://issues.apache.org/jira/browse/HIVE-23498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17121519#comment-17121519 ] Rajkumar Singh commented on HIVE-23498: --- Thanks [~ngangam] for review, pushed the patch to master. > Disable HTTP Trace method on ThriftHttpCliService > - > > Key: HIVE-23498 > URL: https://issues.apache.org/jira/browse/HIVE-23498 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Affects Versions: 3.1.2 >Reporter: Rajkumar Singh >Assignee: Rajkumar Singh >Priority: Major > Attachments: HIVE-23498.01.patch, HIVE-23498.01.patch, > HIVE-23498.02.patch, HIVE-23498.03.patch, HIVE-23498.04.patch, > HIVE-23498.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23439) Hive sessions over 24 hours encounter Kerberos-related StatsTask errors
[
https://issues.apache.org/jira/browse/HIVE-23439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17123336#comment-17123336
]
Hive QA commented on HIVE-23439:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
53s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
48s{color} | {color:blue} ql in master has 1524 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
55s{color} | {color:red} ql generated 1 new + 1524 unchanged - 0 fixed = 1525
total (was 1524) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 43s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Class org.apache.hadoop.hive.ql.exec.StatsTask defines non-transient
non-serializable instance field dbb In StatsTask.java:instance field dbb In
StatsTask.java |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22706/dev-support/hive-personality.sh
|
| git revision | master / b45e152 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22706/yetus/new-findbugs-ql.html
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22706/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Hive sessions over 24 hours encounter Kerberos-related StatsTask errors
> ---
>
> Key: HIVE-23439
> URL: https://issues.apache.org/jira/browse/HIVE-23439
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2, Standalone Metastore
>Affects Versions: 3.1.0
>Reporter: Chiran Ravani
>Assignee: Rajkumar Singh
>Priority: Critical
> Attachments: HIVE-23439.patch
>
>
> We have an application that uses Hive via JDBC. The interesting thing about
> them is that they have sessions that are established with HiveServer2 for
> multiple days. After 24 hours, their queries are failing with
> StatsTask-related errors. From looking in the logs, it looks like the
> communication breaks down between HiveServer2 and the MetaStore.
> Below is error seen:
> {code}
> 2020-04-22T21:25:53,248 ERROR [Thread-1202599]: exec.StatsTask (:()) - Failed
> to run stats task
> org.apache.hadoop.hive.ql.metadata.HiveException:
> org.apache.hadoop.hive.ql.metadata.HiveException: Unable to fetch table
> tennis. Unable to instantiate
> org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
> at
>
[jira] [Updated] (HIVE-23586) load data overwrite into bucket table failed
[ https://issues.apache.org/jira/browse/HIVE-23586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhaolong updated HIVE-23586: Attachment: HIVE-23586.01.patch > load data overwrite into bucket table failed > > > Key: HIVE-23586 > URL: https://issues.apache.org/jira/browse/HIVE-23586 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.0, 3.1.2 >Reporter: zhaolong >Assignee: zhaolong >Priority: Critical > Attachments: HIVE-23586.01.patch, image-2020-06-01-21-40-21-726.png, > image-2020-06-01-21-41-28-732.png > > > load data overwrite into bucket table is failed if filename is not like > 00_0, but insert new data in the table. > > for example: > CREATE EXTERNAL TABLE IF NOT EXISTS test_hive2 (name string,account string) > PARTITIONED BY (logdate string) CLUSTERED BY (account) INTO 4 BUCKETS row > format delimited fields terminated by '|' STORED AS textfile; > load data inpath 'hdfs://hacluster/tmp/zltest' overwrite into table > default.test_hive2 partition (logdate='20200508'); > !image-2020-06-01-21-40-21-726.png! > load data inpath 'hdfs://hacluster/tmp/zltest' overwrite into table > default.test_hive2 partition (logdate='20200508');// should overwrite but > insert new data > !image-2020-06-01-21-41-28-732.png! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23586) load data overwrite into bucket table failed
[ https://issues.apache.org/jira/browse/HIVE-23586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17121487#comment-17121487 ] zhaolong commented on HIVE-23586: - [~chinnalalam] Please help review it. Thank you! this issue fixed in my enviroment. > load data overwrite into bucket table failed > > > Key: HIVE-23586 > URL: https://issues.apache.org/jira/browse/HIVE-23586 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.0, 3.1.2 >Reporter: zhaolong >Assignee: zhaolong >Priority: Critical > Attachments: HIVE-23586.01.patch, image-2020-06-01-21-40-21-726.png, > image-2020-06-01-21-41-28-732.png > > > load data overwrite into bucket table is failed if filename is not like > 00_0, but insert new data in the table. > > for example: > CREATE EXTERNAL TABLE IF NOT EXISTS test_hive2 (name string,account string) > PARTITIONED BY (logdate string) CLUSTERED BY (account) INTO 4 BUCKETS row > format delimited fields terminated by '|' STORED AS textfile; > load data inpath 'hdfs://hacluster/tmp/zltest' overwrite into table > default.test_hive2 partition (logdate='20200508'); > !image-2020-06-01-21-40-21-726.png! > load data inpath 'hdfs://hacluster/tmp/zltest' overwrite into table > default.test_hive2 partition (logdate='20200508');// should overwrite but > insert new data > !image-2020-06-01-21-41-28-732.png! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-23439) Hive sessions over 24 hours encounter Kerberos-related StatsTask errors
[
https://issues.apache.org/jira/browse/HIVE-23439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rajkumar Singh reassigned HIVE-23439:
-
Assignee: Rajkumar Singh
> Hive sessions over 24 hours encounter Kerberos-related StatsTask errors
> ---
>
> Key: HIVE-23439
> URL: https://issues.apache.org/jira/browse/HIVE-23439
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2, Standalone Metastore
>Affects Versions: 3.1.0
>Reporter: Chiran Ravani
>Assignee: Rajkumar Singh
>Priority: Critical
>
> We have an application that uses Hive via JDBC. The interesting thing about
> them is that they have sessions that are established with HiveServer2 for
> multiple days. After 24 hours, their queries are failing with
> StatsTask-related errors. From looking in the logs, it looks like the
> communication breaks down between HiveServer2 and the MetaStore.
> Below is error seen:
> {code}
> 2020-04-22T21:25:53,248 ERROR [Thread-1202599]: exec.StatsTask (:()) - Failed
> to run stats task
> org.apache.hadoop.hive.ql.metadata.HiveException:
> org.apache.hadoop.hive.ql.metadata.HiveException: Unable to fetch table
> tennis. Unable to instantiate
> org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
> at
> org.apache.hadoop.hive.ql.metadata.Hive.setPartitionColumnStatistics(Hive.java:4927)
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
> at
> org.apache.hadoop.hive.ql.stats.ColStatsProcessor.persistColumnStats(ColStatsProcessor.java:189)
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
> at
> org.apache.hadoop.hive.ql.stats.ColStatsProcessor.process(ColStatsProcessor.java:86)
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
> at org.apache.hadoop.hive.ql.exec.StatsTask.execute(StatsTask.java:108)
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:212)
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:103)
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
> at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:82)
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to fetch
> table tennis. Unable to instantiate
> org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
> at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1387)
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
> at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1336)
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
> at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1316)
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
> at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1298)
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
> at
> org.apache.hadoop.hive.ql.metadata.Hive.setPartitionColumnStatistics(Hive.java:4918)
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
> ... 6 more
> Caused by: java.lang.RuntimeException: Unable to instantiate
> org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
> at
> org.apache.hadoop.hive.metastore.utils.JavaUtils.newInstance(JavaUtils.java:86)
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.(RetryingMetaStoreClient.java:95)
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:148)
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:119)
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
> at
> org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:4790)
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
> at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:4858)
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
> at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:4838)
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
> at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1378)
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
> at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1336)
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
> at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1316)
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
> at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1298)
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
> at
>
[jira] [Updated] (HIVE-23439) Hive sessions over 24 hours encounter Kerberos-related StatsTask errors
[
https://issues.apache.org/jira/browse/HIVE-23439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rajkumar Singh updated HIVE-23439:
--
Attachment: HIVE-23439.patch
Status: Patch Available (was: Open)
> Hive sessions over 24 hours encounter Kerberos-related StatsTask errors
> ---
>
> Key: HIVE-23439
> URL: https://issues.apache.org/jira/browse/HIVE-23439
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2, Standalone Metastore
>Affects Versions: 3.1.0
>Reporter: Chiran Ravani
>Assignee: Rajkumar Singh
>Priority: Critical
> Attachments: HIVE-23439.patch
>
>
> We have an application that uses Hive via JDBC. The interesting thing about
> them is that they have sessions that are established with HiveServer2 for
> multiple days. After 24 hours, their queries are failing with
> StatsTask-related errors. From looking in the logs, it looks like the
> communication breaks down between HiveServer2 and the MetaStore.
> Below is error seen:
> {code}
> 2020-04-22T21:25:53,248 ERROR [Thread-1202599]: exec.StatsTask (:()) - Failed
> to run stats task
> org.apache.hadoop.hive.ql.metadata.HiveException:
> org.apache.hadoop.hive.ql.metadata.HiveException: Unable to fetch table
> tennis. Unable to instantiate
> org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
> at
> org.apache.hadoop.hive.ql.metadata.Hive.setPartitionColumnStatistics(Hive.java:4927)
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
> at
> org.apache.hadoop.hive.ql.stats.ColStatsProcessor.persistColumnStats(ColStatsProcessor.java:189)
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
> at
> org.apache.hadoop.hive.ql.stats.ColStatsProcessor.process(ColStatsProcessor.java:86)
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
> at org.apache.hadoop.hive.ql.exec.StatsTask.execute(StatsTask.java:108)
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:212)
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:103)
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
> at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:82)
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to fetch
> table tennis. Unable to instantiate
> org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
> at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1387)
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
> at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1336)
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
> at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1316)
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
> at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1298)
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
> at
> org.apache.hadoop.hive.ql.metadata.Hive.setPartitionColumnStatistics(Hive.java:4918)
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
> ... 6 more
> Caused by: java.lang.RuntimeException: Unable to instantiate
> org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
> at
> org.apache.hadoop.hive.metastore.utils.JavaUtils.newInstance(JavaUtils.java:86)
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.(RetryingMetaStoreClient.java:95)
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:148)
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:119)
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
> at
> org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:4790)
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
> at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:4858)
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
> at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:4838)
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
> at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1378)
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
> at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1336)
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
> at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1316)
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
> at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1298)
>
[jira] [Assigned] (HIVE-23586) load data overwrite into bucket table failed
[ https://issues.apache.org/jira/browse/HIVE-23586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhaolong reassigned HIVE-23586: --- Assignee: zhaolong > load data overwrite into bucket table failed > > > Key: HIVE-23586 > URL: https://issues.apache.org/jira/browse/HIVE-23586 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.0, 3.1.2 >Reporter: zhaolong >Assignee: zhaolong >Priority: Critical > Attachments: image-2020-06-01-21-40-21-726.png, > image-2020-06-01-21-41-28-732.png > > > load data overwrite into bucket table is failed if filename is not like > 00_0, but insert new data in the table. > > for example: > CREATE EXTERNAL TABLE IF NOT EXISTS test_hive2 (name string,account string) > PARTITIONED BY (logdate string) CLUSTERED BY (account) INTO 4 BUCKETS row > format delimited fields terminated by '|' STORED AS textfile; > load data inpath 'hdfs://hacluster/tmp/zltest' overwrite into table > default.test_hive2 partition (logdate='20200508'); > !image-2020-06-01-21-40-21-726.png! > load data inpath 'hdfs://hacluster/tmp/zltest' overwrite into table > default.test_hive2 partition (logdate='20200508');// should overwrite but > insert new data > !image-2020-06-01-21-41-28-732.png! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-21301) Show tables statement to include views and materialized views
[
https://issues.apache.org/jira/browse/HIVE-21301?focusedWorklogId=439863=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-439863
]
ASF GitHub Bot logged work on HIVE-21301:
-
Author: ASF GitHub Bot
Created on: 02/Jun/20 02:52
Start Date: 02/Jun/20 02:52
Worklog Time Spent: 10m
Work Description: jcamachor closed pull request #542:
URL: https://github.com/apache/hive/pull/542
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 439863)
Time Spent: 20m (was: 10m)
> Show tables statement to include views and materialized views
> -
>
> Key: HIVE-21301
> URL: https://issues.apache.org/jira/browse/HIVE-21301
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0, 3.2.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: TODOC3.2, pull-request-available
> Fix For: 4.0.0, 3.2.0
>
> Attachments: HIVE-21301.01.patch, HIVE-21301.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> HIVE-19974 introduced backwards incompatible change, with {{SHOW TABLES}}
> statement showing only managed/external tables in the system.
> This issue will restore old behavior, with {{SHOW TABLES}} showing all
> queryable entities, including views and materialized views.
> Instead, to provide information about table types, {{SHOW EXTENDED TABLES}}
> statement is introduced, which includes an additional column with the table
> type for each of the tables listed.
> Besides, the possibility to filter the show tables statements with a {{WHERE
> `table_type` = 'ANY_TYPE'}} clause is introduced.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-22396) CMV creating a Full ACID partitioned table fails because of no writeId
[
https://issues.apache.org/jira/browse/HIVE-22396?focusedWorklogId=439865=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-439865
]
ASF GitHub Bot logged work on HIVE-22396:
-
Author: ASF GitHub Bot
Created on: 02/Jun/20 02:53
Start Date: 02/Jun/20 02:53
Worklog Time Spent: 10m
Work Description: jcamachor closed pull request #826:
URL: https://github.com/apache/hive/pull/826
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 439865)
Time Spent: 20m (was: 10m)
> CMV creating a Full ACID partitioned table fails because of no writeId
> --
>
> Key: HIVE-22396
> URL: https://issues.apache.org/jira/browse/HIVE-22396
> Project: Hive
> Issue Type: Sub-task
> Components: HiveServer2, repl
>Affects Versions: 4.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-22396.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> create materialized view acid_cmv_part disable rewrite partitioned on (k)
> stored as orc TBLPROPERTIES ('transactional'='true')
> as select key k, value from src order by k limit 5;
> ERROR : FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.MoveTask. MoveTask : Write id is not set in
> the config by open txn task for migration
> Error: Error while processing statement: FAILED: Execution Error, return code
> 1 from org.apache.hadoop.hive.ql.exec.MoveTask. MoveTask : Write id is not
> set in the config by open txn task for migration (state=08S01,code=1)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-21696) Include partition columns and column stats in explain cbo formatted
[ https://issues.apache.org/jira/browse/HIVE-21696?focusedWorklogId=439864=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-439864 ] ASF GitHub Bot logged work on HIVE-21696: - Author: ASF GitHub Bot Created on: 02/Jun/20 02:53 Start Date: 02/Jun/20 02:53 Worklog Time Spent: 10m Work Description: jcamachor closed pull request #622: URL: https://github.com/apache/hive/pull/622 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 439864) Time Spent: 20m (was: 10m) > Include partition columns and column stats in explain cbo formatted > --- > > Key: HIVE-21696 > URL: https://issues.apache.org/jira/browse/HIVE-21696 > Project: Hive > Issue Type: Improvement > Components: CBO >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-21696.01.patch, HIVE-21696.02.patch, > HIVE-21696.02.patch, HIVE-21696.patch > > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23498) Disable HTTP Trace method on ThriftHttpCliService
[ https://issues.apache.org/jira/browse/HIVE-23498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajkumar Singh updated HIVE-23498: -- Fix Version/s: 4.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Disable HTTP Trace method on ThriftHttpCliService > - > > Key: HIVE-23498 > URL: https://issues.apache.org/jira/browse/HIVE-23498 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Affects Versions: 3.1.2 >Reporter: Rajkumar Singh >Assignee: Rajkumar Singh >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-23498.01.patch, HIVE-23498.01.patch, > HIVE-23498.02.patch, HIVE-23498.03.patch, HIVE-23498.04.patch, > HIVE-23498.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-22476) Hive datediff function provided inconsistent results when hive.fetch.task.conversion is set to none
[
https://issues.apache.org/jira/browse/HIVE-22476?focusedWorklogId=439872=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-439872
]
ASF GitHub Bot logged work on HIVE-22476:
-
Author: ASF GitHub Bot
Created on: 02/Jun/20 03:29
Start Date: 02/Jun/20 03:29
Worklog Time Spent: 10m
Work Description: asfgit closed pull request #1046:
URL: https://github.com/apache/hive/pull/1046
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 439872)
Time Spent: 20m (was: 10m)
> Hive datediff function provided inconsistent results when
> hive.fetch.task.conversion is set to none
> ---
>
> Key: HIVE-22476
> URL: https://issues.apache.org/jira/browse/HIVE-22476
> Project: Hive
> Issue Type: Bug
>Reporter: Slim Bouguerra
>Assignee: Slim Bouguerra
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22476.10.patch, HIVE-22476.2.patch,
> HIVE-22476.3.patch, HIVE-22476.5.patch, HIVE-22476.6.patch,
> HIVE-22476.7.patch, HIVE-22476.7.patch, HIVE-22476.8.patch,
> HIVE-22476.8.patch, HIVE-22476.8.patch, HIVE-22476.9.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The actual issue stems to the different date parser used by various part of
> the engine.
> Fetch task uses udfdatediff via {code}
> org.apache.hadoop.hive.ql.udf.generic.GenericUDFToDate{code} while the
> vectorized llap execution uses {code}VectorUDFDateDiffScalarCol{code}.
> This fix is meant to be not very intrusive and will add more support to the
> GenericUDFToDate by enhancing the parser.
> For the longer term will be better to use one parser for all the operators.
> Thanks [~Rajkumar Singh] for the repro example
> {code}
> create external table testdatediff(datetimecol string) stored as orc;
> insert into testdatediff values ('2019-09-09T10:45:49+02:00'),('2019-07-24');
> select datetimecol from testdatediff where datediff(cast(current_timestamp as
> string), datetimecol)<183;
> set hive.ferch.task.conversion=none;
> select datetimecol from testdatediff where datediff(cast(current_timestamp as
> string), datetimecol)<183;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22476) Hive datediff function provided inconsistent results when hive.fetch.task.conversion is set to none
[
https://issues.apache.org/jira/browse/HIVE-22476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-22476:
---
Fix Version/s: 4.0.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Hive datediff function provided inconsistent results when
> hive.fetch.task.conversion is set to none
> ---
>
> Key: HIVE-22476
> URL: https://issues.apache.org/jira/browse/HIVE-22476
> Project: Hive
> Issue Type: Bug
>Reporter: Slim Bouguerra
>Assignee: Slim Bouguerra
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-22476.10.patch, HIVE-22476.2.patch,
> HIVE-22476.3.patch, HIVE-22476.5.patch, HIVE-22476.6.patch,
> HIVE-22476.7.patch, HIVE-22476.7.patch, HIVE-22476.8.patch,
> HIVE-22476.8.patch, HIVE-22476.8.patch, HIVE-22476.9.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The actual issue stems to the different date parser used by various part of
> the engine.
> Fetch task uses udfdatediff via {code}
> org.apache.hadoop.hive.ql.udf.generic.GenericUDFToDate{code} while the
> vectorized llap execution uses {code}VectorUDFDateDiffScalarCol{code}.
> This fix is meant to be not very intrusive and will add more support to the
> GenericUDFToDate by enhancing the parser.
> For the longer term will be better to use one parser for all the operators.
> Thanks [~Rajkumar Singh] for the repro example
> {code}
> create external table testdatediff(datetimecol string) stored as orc;
> insert into testdatediff values ('2019-09-09T10:45:49+02:00'),('2019-07-24');
> select datetimecol from testdatediff where datediff(cast(current_timestamp as
> string), datetimecol)<183;
> set hive.ferch.task.conversion=none;
> select datetimecol from testdatediff where datediff(cast(current_timestamp as
> string), datetimecol)<183;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22017) [ Interface changes ] Keep HMS interfaces backward compatible with changes for HIVE-21637
[ https://issues.apache.org/jira/browse/HIVE-22017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kishen Das updated HIVE-22017: -- Attachment: HIVE-22017.2.patch > [ Interface changes ] Keep HMS interfaces backward compatible with changes > for HIVE-21637 > - > > Key: HIVE-22017 > URL: https://issues.apache.org/jira/browse/HIVE-22017 > Project: Hive > Issue Type: Sub-task >Affects Versions: 2.3.7 >Reporter: Daniel Dai >Assignee: Kishen Das >Priority: Major > Attachments: HIVE-22017.1.patch, HIVE-22017.2.patch > > > As part of HIVE-21637 we would have to introduce ValidWriteIdList in several > methods. Also, in the long term, we should deprecate and remove all the > methods that take direct arguments, as the service definition keeps changing > whenever we add/remove arguments, making it hard to maintain backward > compatibility. So, instead of adding writeId in bunch of get_xxx calls that > take direct arguments, we will create new set of methods that take Request > object and return Response object. We shall mark those deprecated and remove > in future version. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23537) RecordReader support for static row-filtering
[ https://issues.apache.org/jira/browse/HIVE-23537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Panagiotis Garefalakis updated HIVE-23537: -- Summary: RecordReader support for static row-filtering (was: RecordReader support for row-filtering) > RecordReader support for static row-filtering > - > > Key: HIVE-23537 > URL: https://issues.apache.org/jira/browse/HIVE-23537 > Project: Hive > Issue Type: Sub-task > Components: llap, Reader >Reporter: Panagiotis Garefalakis >Assignee: Panagiotis Garefalakis >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > ORC-577 enables row-level filtering for the ORC format while HIVE-23167 is > aiming to extend the existing compiler logic and push filters further down > the pipeline wherever possible. > In this jira we extend the HIVE Record readers to utilize the above filtering > functionality (similar to what we already do for PPD). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22017) [ Interface changes ] Keep HMS interfaces backward compatible with changes for HIVE-21637
[ https://issues.apache.org/jira/browse/HIVE-22017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kishen Das updated HIVE-22017: -- Attachment: (was: HIVE-216371.1.patch) > [ Interface changes ] Keep HMS interfaces backward compatible with changes > for HIVE-21637 > - > > Key: HIVE-22017 > URL: https://issues.apache.org/jira/browse/HIVE-22017 > Project: Hive > Issue Type: Sub-task >Affects Versions: 2.3.7 >Reporter: Daniel Dai >Assignee: Kishen Das >Priority: Major > Attachments: HIVE-22017.1.patch > > > As part of HIVE-21637 we would have to introduce ValidWriteIdList in several > methods. Also, in the long term, we should deprecate and remove all the > methods that take direct arguments, as the service definition keeps changing > whenever we add/remove arguments, making it hard to maintain backward > compatibility. So, instead of adding writeId in bunch of get_xxx calls that > take direct arguments, we will create new set of methods that take Request > object and return Response object. We shall mark those deprecated and remove > in future version. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22017) [ Interface changes ] Keep HMS interfaces backward compatible with changes for HIVE-21637
[ https://issues.apache.org/jira/browse/HIVE-22017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kishen Das updated HIVE-22017: -- Attachment: (was: HIVE-22017.2.patch) > [ Interface changes ] Keep HMS interfaces backward compatible with changes > for HIVE-21637 > - > > Key: HIVE-22017 > URL: https://issues.apache.org/jira/browse/HIVE-22017 > Project: Hive > Issue Type: Sub-task >Affects Versions: 2.3.7 >Reporter: Daniel Dai >Assignee: Kishen Das >Priority: Major > Attachments: HIVE-22017.1.patch > > > As part of HIVE-21637 we would have to introduce ValidWriteIdList in several > methods. Also, in the long term, we should deprecate and remove all the > methods that take direct arguments, as the service definition keeps changing > whenever we add/remove arguments, making it hard to maintain backward > compatibility. So, instead of adding writeId in bunch of get_xxx calls that > take direct arguments, we will create new set of methods that take Request > object and return Response object. We shall mark those deprecated and remove > in future version. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22017) [ Interface changes ] Keep HMS interfaces backward compatible with changes for HIVE-21637
[ https://issues.apache.org/jira/browse/HIVE-22017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kishen Das updated HIVE-22017: -- Attachment: (was: HIVE-22017.2.patch) > [ Interface changes ] Keep HMS interfaces backward compatible with changes > for HIVE-21637 > - > > Key: HIVE-22017 > URL: https://issues.apache.org/jira/browse/HIVE-22017 > Project: Hive > Issue Type: Sub-task >Affects Versions: 2.3.7 >Reporter: Daniel Dai >Assignee: Kishen Das >Priority: Major > Attachments: HIVE-22017.1.patch > > > As part of HIVE-21637 we would have to introduce ValidWriteIdList in several > methods. Also, in the long term, we should deprecate and remove all the > methods that take direct arguments, as the service definition keeps changing > whenever we add/remove arguments, making it hard to maintain backward > compatibility. So, instead of adding writeId in bunch of get_xxx calls that > take direct arguments, we will create new set of methods that take Request > object and return Response object. We shall mark those deprecated and remove > in future version. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23435) Full outer join result is missing rows
[
https://issues.apache.org/jira/browse/HIVE-23435?focusedWorklogId=439632=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-439632
]
ASF GitHub Bot logged work on HIVE-23435:
-
Author: ASF GitHub Bot
Created on: 01/Jun/20 17:43
Start Date: 01/Jun/20 17:43
Worklog Time Spent: 10m
Work Description: mustafaiman closed pull request #1039:
URL: https://github.com/apache/hive/pull/1039
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 439632)
Time Spent: 20m (was: 10m)
> Full outer join result is missing rows
> ---
>
> Key: HIVE-23435
> URL: https://issues.apache.org/jira/browse/HIVE-23435
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 3.1.0
>Reporter: Naveen Gangam
>Assignee: Mustafa Iman
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-23435.1.patch, HIVE-23435.1.patch,
> HIVE-23435.patch, HIVE-23435.patch, HIVE-23435.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Full Outer join result has missing rows. Appears to be a bug with the full
> outer join logic. Expected output is receiving when we do a left and right
> outer join.
> Reproducible steps are mentioned below.
> ~~
> SUPPORT ANALYSIS
> Steps to Reproduce:
> 1. Create a table and insert data:
> create table x (z char(5), x int, y int);
> insert into x values ('one', 1, 50),
> ('two', 2, 30),
> ('three', 3, 30),
> ('four', 4, 60),
> ('five', 5, 70),
> ('six', 6, 80);
> 2. Try full outer with the below command. The result is incomplete, it is
> missing the row:
> NULL NULL NULL three 3 30.0
> Full Outer Join:
> select x1.`z`, x1.`x`, x1.`y`, x2.`z`,
> x2.`x`, x2.`y`
> from `x` x1 full outer join
> `x` x2 on (x1.`x` > 3) and (x2.`x` < 4) and (x1.`x` =
> x2.`x`);
> Result:
> --+
> x1.z x1.x x1.y x2.z x2.x x2.y
> --+
> one 1 50 NULL NULL NULL
> NULL NULL NULL one 1 50
> two 2 30 NULL NULL NULL
> NULL NULL NULL two 2 30
> three 3 30 NULL NULL NULL
> four 4 60 NULL NULL NULL
> NULL NULL NULL four 4 60
> five 5 70 NULL NULL NULL
> NULL NULL NULL five 5 70
> six 6 80 NULL NULL NULL
> NULL NULL NULL six 6 80
> --+
> 3. Expected output is coming when we use left/right join + union:
> select x1.`z`, x1.`x`, x1.`y`, x2.`z`,
> x2.`x`, x2.`y`
> from `x` x1 left outer join
> `x` x2 on (x1.`x` > 3) and (x2.`x` < 4) and (x1.`x` =
> x2.`x`)
> union
> select x1.`z`, x1.`x`, x1.`y`, x2.`z`,
> x2.`x`, x2.`y`
> from `x` x1 right outer join
> `x` x2 on (x1.`x` > 3) and (x2.`x` < 4) and (x1.`x` =
> x2.`x`);
> Result:
> +
> z x y _col3 _col4 _col5
> +
> NULL NULL NULL five 5 70
> NULL NULL NULL four 4 60
> NULL NULL NULL one 1 50
> four 4 60 NULL NULL NULL
> one 1 50 NULL NULL NULL
> six 6 80 NULL NULL NULL
> three 3 30 NULL NULL NULL
> two 2 30 NULL NULL NULL
> NULL NULL NULL six 6 80
> NULL NULL NULL three 3 30
> NULL NULL NULL two 2 30
> five 5 70 NULL NULL NULL
> +
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22017) [ Interface changes ] Keep HMS interfaces backward compatible with changes for HIVE-21637
[
https://issues.apache.org/jira/browse/HIVE-22017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17121250#comment-17121250
]
Hive QA commented on HIVE-22017:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13004510/HIVE-22017.2.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 11 failed/errored test(s), 17236 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[results_cache_invalidation2]
(batchId=68)
org.apache.hadoop.hive.ql.metadata.TestSessionHiveMetastoreClientGetPartitionsTempTable.testGetPartitionRequest[Embedded]
(batchId=270)
org.apache.hadoop.hive.ql.metadata.TestSessionHiveMetastoreClientGetPartitionsTempTable.testGetPartitionRequest[Remote]
(batchId=270)
org.apache.hadoop.hive.ql.metadata.TestSessionHiveMetastoreClientListPartitionsTempTable.testGetPartitionsRequest[Embedded]
(batchId=270)
org.apache.hadoop.hive.ql.metadata.TestSessionHiveMetastoreClientListPartitionsTempTable.testGetPartitionsRequest[Remote]
(batchId=270)
org.apache.hadoop.hive.ql.metadata.TestSessionHiveMetastoreClientListPartitionsTempTable.testListPartitionNamesRequestByValues[Embedded]
(batchId=270)
org.apache.hadoop.hive.ql.metadata.TestSessionHiveMetastoreClientListPartitionsTempTable.testListPartitionNamesRequestByValues[Remote]
(batchId=270)
org.apache.hadoop.hive.ql.metadata.TestSessionHiveMetastoreClientListPartitionsTempTable.testListPartitionsWithAuthRequestByValues[Embedded]
(batchId=270)
org.apache.hadoop.hive.ql.metadata.TestSessionHiveMetastoreClientListPartitionsTempTable.testListPartitionsWithAuthRequestByValues[Remote]
(batchId=270)
org.apache.hadoop.hive.ql.stats.TestStatsUpdaterThread.testTxnTable
(batchId=258)
org.apache.hive.service.auth.TestImproperTrustDomainAuthenticationBinary.org.apache.hive.service.auth.TestImproperTrustDomainAuthenticationBinary
(batchId=211)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22704/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22704/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22704/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 11 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13004510 - PreCommit-HIVE-Build
> [ Interface changes ] Keep HMS interfaces backward compatible with changes
> for HIVE-21637
> -
>
> Key: HIVE-22017
> URL: https://issues.apache.org/jira/browse/HIVE-22017
> Project: Hive
> Issue Type: Sub-task
>Affects Versions: 2.3.7
>Reporter: Daniel Dai
>Assignee: Kishen Das
>Priority: Major
> Attachments: HIVE-22017.1.patch, HIVE-22017.2.patch
>
>
> As part of HIVE-21637 we would have to introduce ValidWriteIdList in several
> methods. Also, in the long term, we should deprecate and remove all the
> methods that take direct arguments, as the service definition keeps changing
> whenever we add/remove arguments, making it hard to maintain backward
> compatibility. So, instead of adding writeId in bunch of get_xxx calls that
> take direct arguments, we will create new set of methods that take Request
> object and return Response object. We shall mark those deprecated and remove
> in future version.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22017) [ Interface changes ] Keep HMS interfaces backward compatible with changes for HIVE-21637
[
https://issues.apache.org/jira/browse/HIVE-22017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17121305#comment-17121305
]
Hive QA commented on HIVE-22017:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13004511/HIVE-22017.2.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 9 failed/errored test(s), 17236 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.metadata.TestSessionHiveMetastoreClientGetPartitionsTempTable.testGetPartitionRequest[Embedded]
(batchId=270)
org.apache.hadoop.hive.ql.metadata.TestSessionHiveMetastoreClientGetPartitionsTempTable.testGetPartitionRequest[Remote]
(batchId=270)
org.apache.hadoop.hive.ql.metadata.TestSessionHiveMetastoreClientListPartitionsTempTable.testGetPartitionsRequest[Embedded]
(batchId=270)
org.apache.hadoop.hive.ql.metadata.TestSessionHiveMetastoreClientListPartitionsTempTable.testGetPartitionsRequest[Remote]
(batchId=270)
org.apache.hadoop.hive.ql.metadata.TestSessionHiveMetastoreClientListPartitionsTempTable.testListPartitionNamesRequestByValues[Embedded]
(batchId=270)
org.apache.hadoop.hive.ql.metadata.TestSessionHiveMetastoreClientListPartitionsTempTable.testListPartitionNamesRequestByValues[Remote]
(batchId=270)
org.apache.hadoop.hive.ql.metadata.TestSessionHiveMetastoreClientListPartitionsTempTable.testListPartitionsWithAuthRequestByValues[Embedded]
(batchId=270)
org.apache.hadoop.hive.ql.metadata.TestSessionHiveMetastoreClientListPartitionsTempTable.testListPartitionsWithAuthRequestByValues[Remote]
(batchId=270)
org.apache.hadoop.hive.ql.stats.TestStatsUpdaterThread.testTxnTable
(batchId=258)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22705/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22705/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22705/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 9 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13004511 - PreCommit-HIVE-Build
> [ Interface changes ] Keep HMS interfaces backward compatible with changes
> for HIVE-21637
> -
>
> Key: HIVE-22017
> URL: https://issues.apache.org/jira/browse/HIVE-22017
> Project: Hive
> Issue Type: Sub-task
>Affects Versions: 2.3.7
>Reporter: Daniel Dai
>Assignee: Kishen Das
>Priority: Major
> Attachments: HIVE-22017.1.patch, HIVE-22017.2.patch
>
>
> As part of HIVE-21637 we would have to introduce ValidWriteIdList in several
> methods. Also, in the long term, we should deprecate and remove all the
> methods that take direct arguments, as the service definition keeps changing
> whenever we add/remove arguments, making it hard to maintain backward
> compatibility. So, instead of adding writeId in bunch of get_xxx calls that
> take direct arguments, we will create new set of methods that take Request
> object and return Response object. We shall mark those deprecated and remove
> in future version.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23498) Disable HTTP Trace method on ThriftHttpCliService
[ https://issues.apache.org/jira/browse/HIVE-23498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17121240#comment-17121240 ] Naveen Gangam commented on HIVE-23498: -- [~Rajkumar Singh] sorry, I must have looked at the old results from the prior tests in the email. Tests are clean. So +1 > Disable HTTP Trace method on ThriftHttpCliService > - > > Key: HIVE-23498 > URL: https://issues.apache.org/jira/browse/HIVE-23498 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Affects Versions: 3.1.2 >Reporter: Rajkumar Singh >Assignee: Rajkumar Singh >Priority: Major > Attachments: HIVE-23498.01.patch, HIVE-23498.01.patch, > HIVE-23498.02.patch, HIVE-23498.03.patch, HIVE-23498.04.patch, > HIVE-23498.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23365) Put RS deduplication optimization under cost based decision
[ https://issues.apache.org/jira/browse/HIVE-23365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17121151#comment-17121151 ] Jesus Camacho Rodriguez commented on HIVE-23365: Left a minor comment in the review. Other than that, patch LGTM. +1 (pending tests) > Put RS deduplication optimization under cost based decision > --- > > Key: HIVE-23365 > URL: https://issues.apache.org/jira/browse/HIVE-23365 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Reporter: Jesus Camacho Rodriguez >Assignee: Stamatis Zampetakis >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23365.01.patch, HIVE-23365.02.patch, > HIVE-23365.03.patch, HIVE-23365.04.patch, HIVE-23365.05.patch > > Time Spent: 40m > Remaining Estimate: 0h > > Currently, RS deduplication is always executed whenever it is semantically > correct. However, it could be beneficial to leave both RS operators in the > plan, e.g., if the NDV of the second RS is very low. Thus, we would like this > decision to be cost-based. We could use a simple heuristic that would work > fine for most of the cases without introducing regressions for existing > cases, e.g., if NDV for partition column is less than estimated parallelism > in the second RS, do not execute deduplication. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23435) Full outer join result is missing rows
[
https://issues.apache.org/jira/browse/HIVE-23435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17121172#comment-17121172
]
Ashutosh Chauhan commented on HIVE-23435:
-
I don't see any code changes for vectorized join operator? Is this because they
end up calling genAllOneUniqueJoinObject() which is now fixed? Or this bug
didnt exist on vectorized operator?
> Full outer join result is missing rows
> ---
>
> Key: HIVE-23435
> URL: https://issues.apache.org/jira/browse/HIVE-23435
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 3.1.0
>Reporter: Naveen Gangam
>Assignee: Mustafa Iman
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-23435.1.patch, HIVE-23435.1.patch,
> HIVE-23435.patch, HIVE-23435.patch, HIVE-23435.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Full Outer join result has missing rows. Appears to be a bug with the full
> outer join logic. Expected output is receiving when we do a left and right
> outer join.
> Reproducible steps are mentioned below.
> ~~
> SUPPORT ANALYSIS
> Steps to Reproduce:
> 1. Create a table and insert data:
> create table x (z char(5), x int, y int);
> insert into x values ('one', 1, 50),
> ('two', 2, 30),
> ('three', 3, 30),
> ('four', 4, 60),
> ('five', 5, 70),
> ('six', 6, 80);
> 2. Try full outer with the below command. The result is incomplete, it is
> missing the row:
> NULL NULL NULL three 3 30.0
> Full Outer Join:
> select x1.`z`, x1.`x`, x1.`y`, x2.`z`,
> x2.`x`, x2.`y`
> from `x` x1 full outer join
> `x` x2 on (x1.`x` > 3) and (x2.`x` < 4) and (x1.`x` =
> x2.`x`);
> Result:
> --+
> x1.z x1.x x1.y x2.z x2.x x2.y
> --+
> one 1 50 NULL NULL NULL
> NULL NULL NULL one 1 50
> two 2 30 NULL NULL NULL
> NULL NULL NULL two 2 30
> three 3 30 NULL NULL NULL
> four 4 60 NULL NULL NULL
> NULL NULL NULL four 4 60
> five 5 70 NULL NULL NULL
> NULL NULL NULL five 5 70
> six 6 80 NULL NULL NULL
> NULL NULL NULL six 6 80
> --+
> 3. Expected output is coming when we use left/right join + union:
> select x1.`z`, x1.`x`, x1.`y`, x2.`z`,
> x2.`x`, x2.`y`
> from `x` x1 left outer join
> `x` x2 on (x1.`x` > 3) and (x2.`x` < 4) and (x1.`x` =
> x2.`x`)
> union
> select x1.`z`, x1.`x`, x1.`y`, x2.`z`,
> x2.`x`, x2.`y`
> from `x` x1 right outer join
> `x` x2 on (x1.`x` > 3) and (x2.`x` < 4) and (x1.`x` =
> x2.`x`);
> Result:
> +
> z x y _col3 _col4 _col5
> +
> NULL NULL NULL five 5 70
> NULL NULL NULL four 4 60
> NULL NULL NULL one 1 50
> four 4 60 NULL NULL NULL
> one 1 50 NULL NULL NULL
> six 6 80 NULL NULL NULL
> three 3 30 NULL NULL NULL
> two 2 30 NULL NULL NULL
> NULL NULL NULL six 6 80
> NULL NULL NULL three 3 30
> NULL NULL NULL two 2 30
> five 5 70 NULL NULL NULL
> +
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23435) Full outer join result is missing rows
[
https://issues.apache.org/jira/browse/HIVE-23435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17121186#comment-17121186
]
Ashutosh Chauhan commented on HIVE-23435:
-
+1
> Full outer join result is missing rows
> ---
>
> Key: HIVE-23435
> URL: https://issues.apache.org/jira/browse/HIVE-23435
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 3.1.0
>Reporter: Naveen Gangam
>Assignee: Mustafa Iman
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-23435.1.patch, HIVE-23435.1.patch,
> HIVE-23435.patch, HIVE-23435.patch, HIVE-23435.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Full Outer join result has missing rows. Appears to be a bug with the full
> outer join logic. Expected output is receiving when we do a left and right
> outer join.
> Reproducible steps are mentioned below.
> ~~
> SUPPORT ANALYSIS
> Steps to Reproduce:
> 1. Create a table and insert data:
> create table x (z char(5), x int, y int);
> insert into x values ('one', 1, 50),
> ('two', 2, 30),
> ('three', 3, 30),
> ('four', 4, 60),
> ('five', 5, 70),
> ('six', 6, 80);
> 2. Try full outer with the below command. The result is incomplete, it is
> missing the row:
> NULL NULL NULL three 3 30.0
> Full Outer Join:
> select x1.`z`, x1.`x`, x1.`y`, x2.`z`,
> x2.`x`, x2.`y`
> from `x` x1 full outer join
> `x` x2 on (x1.`x` > 3) and (x2.`x` < 4) and (x1.`x` =
> x2.`x`);
> Result:
> --+
> x1.z x1.x x1.y x2.z x2.x x2.y
> --+
> one 1 50 NULL NULL NULL
> NULL NULL NULL one 1 50
> two 2 30 NULL NULL NULL
> NULL NULL NULL two 2 30
> three 3 30 NULL NULL NULL
> four 4 60 NULL NULL NULL
> NULL NULL NULL four 4 60
> five 5 70 NULL NULL NULL
> NULL NULL NULL five 5 70
> six 6 80 NULL NULL NULL
> NULL NULL NULL six 6 80
> --+
> 3. Expected output is coming when we use left/right join + union:
> select x1.`z`, x1.`x`, x1.`y`, x2.`z`,
> x2.`x`, x2.`y`
> from `x` x1 left outer join
> `x` x2 on (x1.`x` > 3) and (x2.`x` < 4) and (x1.`x` =
> x2.`x`)
> union
> select x1.`z`, x1.`x`, x1.`y`, x2.`z`,
> x2.`x`, x2.`y`
> from `x` x1 right outer join
> `x` x2 on (x1.`x` > 3) and (x2.`x` < 4) and (x1.`x` =
> x2.`x`);
> Result:
> +
> z x y _col3 _col4 _col5
> +
> NULL NULL NULL five 5 70
> NULL NULL NULL four 4 60
> NULL NULL NULL one 1 50
> four 4 60 NULL NULL NULL
> one 1 50 NULL NULL NULL
> six 6 80 NULL NULL NULL
> three 3 30 NULL NULL NULL
> two 2 30 NULL NULL NULL
> NULL NULL NULL six 6 80
> NULL NULL NULL three 3 30
> NULL NULL NULL two 2 30
> five 5 70 NULL NULL NULL
> +
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22017) [ Interface changes ] Keep HMS interfaces backward compatible with changes for HIVE-21637
[
https://issues.apache.org/jira/browse/HIVE-22017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17121199#comment-17121199
]
Hive QA commented on HIVE-22017:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13004508/HIVE-22017.2.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22703/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22703/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22703/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2020-06-01 17:34:35.160
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-22703/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2020-06-01 17:34:35.163
+ cd apache-github-source-source
+ git fetch origin
>From https://github.com/apache/hive
60ff1fc..779c42f master -> origin/master
+ git reset --hard HEAD
HEAD is now at 60ff1fc HIVE-23068: Error when submitting fragment to LLAP via
external client: IllegalStateException: Only a single registration allowed per
entity (Jason Dere, reviewed by Prasanth Jayachandran)
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is behind 'origin/master' by 1 commit, and can be fast-forwarded.
(use "git pull" to update your local branch)
+ git reset --hard origin/master
HEAD is now at 779c42f HIVE-23435 : Full outer join result is missing rows
(Mustafa Iman via Ashutosh Chauhan)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2020-06-01 17:34:36.490
+ rm -rf ../yetus_PreCommit-HIVE-Build-22703
+ mkdir ../yetus_PreCommit-HIVE-Build-22703
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-22703
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-22703/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
Trying to apply the patch with -p0
Going to apply patch with: git apply -p0
/data/hiveptest/working/scratch/build.patch:142: trailing whitespace.
tmpMap.put(_Fields.DB_NAME, new
org.apache.thrift.meta_data.FieldMetaData("dbName",
org.apache.thrift.TFieldRequirementType.REQUIRED,
/data/hiveptest/working/scratch/build.patch:144: trailing whitespace.
tmpMap.put(_Fields.TBL_NAME, new
org.apache.thrift.meta_data.FieldMetaData("tblName",
org.apache.thrift.TFieldRequirementType.REQUIRED,
/data/hiveptest/working/scratch/build.patch:146: trailing whitespace.
tmpMap.put(_Fields.ENV_CONTEXT, new
org.apache.thrift.meta_data.FieldMetaData("envContext",
org.apache.thrift.TFieldRequirementType.OPTIONAL,
/data/hiveptest/working/scratch/build.patch:148: trailing whitespace.
tmpMap.put(_Fields.VALID_WRITE_ID_LIST, new
org.apache.thrift.meta_data.FieldMetaData("validWriteIdList",
org.apache.thrift.TFieldRequirementType.OPTIONAL,
/data/hiveptest/working/scratch/build.patch:150: trailing whitespace.
tmpMap.put(_Fields.TABLE_ID, new
org.apache.thrift.meta_data.FieldMetaData("tableId",
org.apache.thrift.TFieldRequirementType.OPTIONAL,
warning: squelched 463 whitespace errors
warning: 468 lines add whitespace errors.
+ [[ maven == \m\a\v\e\n ]]
+ rm -rf /data/hiveptest/working/maven/org/apache/hive
+ mvn -B clean install -DskipTests -T 4 -q
-Dmaven.repo.local=/data/hiveptest/working/maven
protoc-jar: executing: [/tmp/protoc4846693055781834601.exe, --version]
libprotoc 2.6.1
protoc-jar: executing: [/tmp/protoc4846693055781834601.exe,
[jira] [Updated] (HIVE-22017) [ Interface changes ] Keep HMS interfaces backward compatible with changes for HIVE-21637
[ https://issues.apache.org/jira/browse/HIVE-22017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kishen Das updated HIVE-22017: -- Attachment: HIVE-22017.2.patch > [ Interface changes ] Keep HMS interfaces backward compatible with changes > for HIVE-21637 > - > > Key: HIVE-22017 > URL: https://issues.apache.org/jira/browse/HIVE-22017 > Project: Hive > Issue Type: Sub-task >Affects Versions: 2.3.7 >Reporter: Daniel Dai >Assignee: Kishen Das >Priority: Major > Attachments: HIVE-216371.1.patch, HIVE-22017.1.patch, > HIVE-22017.2.patch, HIVE-22017.2.patch > > > As part of HIVE-21637 we would have to introduce ValidWriteIdList in several > methods. Also, in the long term, we should deprecate and remove all the > methods that take direct arguments, as the service definition keeps changing > whenever we add/remove arguments, making it hard to maintain backward > compatibility. So, instead of adding writeId in bunch of get_xxx calls that > take direct arguments, we will create new set of methods that take Request > object and return Response object. We shall mark those deprecated and remove > in future version. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23587) Remove JODA Time From LlapServiceDriver
[ https://issues.apache.org/jira/browse/HIVE-23587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17121148#comment-17121148 ] Ashutosh Chauhan commented on HIVE-23587: - +1 > Remove JODA Time From LlapServiceDriver > --- > > Key: HIVE-23587 > URL: https://issues.apache.org/jira/browse/HIVE-23587 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Use JDK facilities instead. > To keep it as simple as possible, and using only Java standard stuff, > changing the existing format: > From: > "ddMMM" (25DEC2020) > To "Basic ISO date": > "mmdd" (25122020) > https://docs.oracle.com/javase/8/docs/api/java/time/format/DateTimeFormatter.html#BASIC_ISO_DATE -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23365) Put RS deduplication optimization under cost based decision
[
https://issues.apache.org/jira/browse/HIVE-23365?focusedWorklogId=439593=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-439593
]
ASF GitHub Bot logged work on HIVE-23365:
-
Author: ASF GitHub Bot
Created on: 01/Jun/20 16:34
Start Date: 01/Jun/20 16:34
Worklog Time Spent: 10m
Work Description: jcamachor commented on a change in pull request #1035:
URL: https://github.com/apache/hive/pull/1035#discussion_r433349779
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/optimizer/correlation/ReduceSinkDeDuplicationUtils.java
##
@@ -197,6 +186,27 @@ public static boolean merge(ReduceSinkOperator cRS,
ReduceSinkOperator pRS, int
return true;
}
+ private static long estimateReducers(HiveConf conf, ReduceSinkOperator rs) {
+// TODO: Check if we can somehow exploit the logic in SetReducerParallelism
+if (rs.getConf().getNumReducers() > 0) {
+ return rs.getConf().getNumReducers();
+}
+int constantReducers = conf.getIntVar(HiveConf.ConfVars.HADOOPNUMREDUCERS);
+if (constantReducers > 0) {
+ return constantReducers;
+}
+long inputTotalBytes = 0;
+List> rsSiblings =
rs.getChildOperators().get(0).getParentOperators();
+for (Operator sibling : rsSiblings) {
+ if (sibling.getStatistics() != null) {
Review comment:
I am wondering, if you do not have stats available for any of the
siblings, we should probably skip the parallelism check, and thus, fallback to
previous behavior.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 439593)
Time Spent: 40m (was: 0.5h)
> Put RS deduplication optimization under cost based decision
> ---
>
> Key: HIVE-23365
> URL: https://issues.apache.org/jira/browse/HIVE-23365
> Project: Hive
> Issue Type: Improvement
> Components: Physical Optimizer
>Reporter: Jesus Camacho Rodriguez
>Assignee: Stamatis Zampetakis
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-23365.01.patch, HIVE-23365.02.patch,
> HIVE-23365.03.patch, HIVE-23365.04.patch, HIVE-23365.05.patch
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Currently, RS deduplication is always executed whenever it is semantically
> correct. However, it could be beneficial to leave both RS operators in the
> plan, e.g., if the NDV of the second RS is very low. Thus, we would like this
> decision to be cost-based. We could use a simple heuristic that would work
> fine for most of the cases without introducing regressions for existing
> cases, e.g., if NDV for partition column is less than estimated parallelism
> in the second RS, do not execute deduplication.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22675) Replace Base64 in hive-standalone-metastore Package
[ https://issues.apache.org/jira/browse/HIVE-22675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17121154#comment-17121154 ] Ashutosh Chauhan commented on HIVE-22675: - +1 > Replace Base64 in hive-standalone-metastore Package > --- > > Key: HIVE-22675 > URL: https://issues.apache.org/jira/browse/HIVE-22675 > Project: Hive > Issue Type: Sub-task >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Attachments: HIVE-22675.1.patch, HIVE-22675.2.patch, > HIVE-22675.3.patch, HIVE-22675.3.patch, HIVE-22675.3.patch, > HIVE-22675.3.patch, HIVE-22675.3.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22017) [ Interface changes ] Keep HMS interfaces backward compatible with changes for HIVE-21637
[ https://issues.apache.org/jira/browse/HIVE-22017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kishen Das updated HIVE-22017: -- Attachment: (was: HIVE-216371.2.patch) > [ Interface changes ] Keep HMS interfaces backward compatible with changes > for HIVE-21637 > - > > Key: HIVE-22017 > URL: https://issues.apache.org/jira/browse/HIVE-22017 > Project: Hive > Issue Type: Sub-task >Affects Versions: 2.3.7 >Reporter: Daniel Dai >Assignee: Kishen Das >Priority: Major > Attachments: HIVE-216371.1.patch, HIVE-22017.1.patch, > HIVE-22017.2.patch > > > As part of HIVE-21637 we would have to introduce ValidWriteIdList in several > methods. Also, in the long term, we should deprecate and remove all the > methods that take direct arguments, as the service definition keeps changing > whenever we add/remove arguments, making it hard to maintain backward > compatibility. So, instead of adding writeId in bunch of get_xxx calls that > take direct arguments, we will create new set of methods that take Request > object and return Response object. We shall mark those deprecated and remove > in future version. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23514) Add Atlas metadata replication metrics
[ https://issues.apache.org/jira/browse/HIVE-23514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17120929#comment-17120929 ] Aasha Medhi commented on HIVE-23514: +1 > Add Atlas metadata replication metrics > -- > > Key: HIVE-23514 > URL: https://issues.apache.org/jira/browse/HIVE-23514 > Project: Hive > Issue Type: Task >Reporter: Pravin Sinha >Assignee: Pravin Sinha >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23514.01.patch > > Time Spent: 1h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22675) Replace Base64 in hive-standalone-metastore Package
[ https://issues.apache.org/jira/browse/HIVE-22675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-22675: -- Attachment: HIVE-22675.3.patch > Replace Base64 in hive-standalone-metastore Package > --- > > Key: HIVE-22675 > URL: https://issues.apache.org/jira/browse/HIVE-22675 > Project: Hive > Issue Type: Sub-task >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Attachments: HIVE-22675.1.patch, HIVE-22675.2.patch, > HIVE-22675.3.patch, HIVE-22675.3.patch, HIVE-22675.3.patch, > HIVE-22675.3.patch, HIVE-22675.3.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23587) Remove JODA Time From LlapServiceDriver
[ https://issues.apache.org/jira/browse/HIVE-23587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-23587: -- Labels: pull-request-available (was: ) > Remove JODA Time From LlapServiceDriver > --- > > Key: HIVE-23587 > URL: https://issues.apache.org/jira/browse/HIVE-23587 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Use JDK facilities instead. > To keep it as simple as possible, and using only Java standard stuff, > changing the existing format: > From: > "ddMMM" (25DEC2020) > To "Basic ISO date": > "mmdd" (25122020) > https://docs.oracle.com/javase/8/docs/api/java/time/format/DateTimeFormatter.html#BASIC_ISO_DATE -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22017) [ Interface changes ] Keep HMS interfaces backward compatible with changes for HIVE-21637
[ https://issues.apache.org/jira/browse/HIVE-22017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kishen Das updated HIVE-22017: -- Attachment: HIVE-22017.2.patch > [ Interface changes ] Keep HMS interfaces backward compatible with changes > for HIVE-21637 > - > > Key: HIVE-22017 > URL: https://issues.apache.org/jira/browse/HIVE-22017 > Project: Hive > Issue Type: Sub-task >Affects Versions: 2.3.7 >Reporter: Daniel Dai >Assignee: Kishen Das >Priority: Major > Attachments: HIVE-216371.1.patch, HIVE-216371.2.patch, > HIVE-22017.1.patch, HIVE-22017.2.patch > > > As part of HIVE-21637 we would have to introduce ValidWriteIdList in several > methods. Also, in the long term, we should deprecate and remove all the > methods that take direct arguments, as the service definition keeps changing > whenever we add/remove arguments, making it hard to maintain backward > compatibility. So, instead of adding writeId in bunch of get_xxx calls that > take direct arguments, we will create new set of methods that take Request > object and return Response object. We shall mark those deprecated and remove > in future version. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23435) Full outer join result is missing rows
[
https://issues.apache.org/jira/browse/HIVE-23435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17121180#comment-17121180
]
Mustafa Iman commented on HIVE-23435:
-
[~ashutoshc] yes, vectorized operator inherits the fixes in here
> Full outer join result is missing rows
> ---
>
> Key: HIVE-23435
> URL: https://issues.apache.org/jira/browse/HIVE-23435
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 3.1.0
>Reporter: Naveen Gangam
>Assignee: Mustafa Iman
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-23435.1.patch, HIVE-23435.1.patch,
> HIVE-23435.patch, HIVE-23435.patch, HIVE-23435.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Full Outer join result has missing rows. Appears to be a bug with the full
> outer join logic. Expected output is receiving when we do a left and right
> outer join.
> Reproducible steps are mentioned below.
> ~~
> SUPPORT ANALYSIS
> Steps to Reproduce:
> 1. Create a table and insert data:
> create table x (z char(5), x int, y int);
> insert into x values ('one', 1, 50),
> ('two', 2, 30),
> ('three', 3, 30),
> ('four', 4, 60),
> ('five', 5, 70),
> ('six', 6, 80);
> 2. Try full outer with the below command. The result is incomplete, it is
> missing the row:
> NULL NULL NULL three 3 30.0
> Full Outer Join:
> select x1.`z`, x1.`x`, x1.`y`, x2.`z`,
> x2.`x`, x2.`y`
> from `x` x1 full outer join
> `x` x2 on (x1.`x` > 3) and (x2.`x` < 4) and (x1.`x` =
> x2.`x`);
> Result:
> --+
> x1.z x1.x x1.y x2.z x2.x x2.y
> --+
> one 1 50 NULL NULL NULL
> NULL NULL NULL one 1 50
> two 2 30 NULL NULL NULL
> NULL NULL NULL two 2 30
> three 3 30 NULL NULL NULL
> four 4 60 NULL NULL NULL
> NULL NULL NULL four 4 60
> five 5 70 NULL NULL NULL
> NULL NULL NULL five 5 70
> six 6 80 NULL NULL NULL
> NULL NULL NULL six 6 80
> --+
> 3. Expected output is coming when we use left/right join + union:
> select x1.`z`, x1.`x`, x1.`y`, x2.`z`,
> x2.`x`, x2.`y`
> from `x` x1 left outer join
> `x` x2 on (x1.`x` > 3) and (x2.`x` < 4) and (x1.`x` =
> x2.`x`)
> union
> select x1.`z`, x1.`x`, x1.`y`, x2.`z`,
> x2.`x`, x2.`y`
> from `x` x1 right outer join
> `x` x2 on (x1.`x` > 3) and (x2.`x` < 4) and (x1.`x` =
> x2.`x`);
> Result:
> +
> z x y _col3 _col4 _col5
> +
> NULL NULL NULL five 5 70
> NULL NULL NULL four 4 60
> NULL NULL NULL one 1 50
> four 4 60 NULL NULL NULL
> one 1 50 NULL NULL NULL
> six 6 80 NULL NULL NULL
> three 3 30 NULL NULL NULL
> two 2 30 NULL NULL NULL
> NULL NULL NULL six 6 80
> NULL NULL NULL three 3 30
> NULL NULL NULL two 2 30
> five 5 70 NULL NULL NULL
> +
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23435) Full outer join result is missing rows
[
https://issues.apache.org/jira/browse/HIVE-23435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated HIVE-23435:
Fix Version/s: 4.0.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
Pushed to master. Thanks, Mustafa!
> Full outer join result is missing rows
> ---
>
> Key: HIVE-23435
> URL: https://issues.apache.org/jira/browse/HIVE-23435
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 3.1.0
>Reporter: Naveen Gangam
>Assignee: Mustafa Iman
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-23435.1.patch, HIVE-23435.1.patch,
> HIVE-23435.patch, HIVE-23435.patch, HIVE-23435.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Full Outer join result has missing rows. Appears to be a bug with the full
> outer join logic. Expected output is receiving when we do a left and right
> outer join.
> Reproducible steps are mentioned below.
> ~~
> SUPPORT ANALYSIS
> Steps to Reproduce:
> 1. Create a table and insert data:
> create table x (z char(5), x int, y int);
> insert into x values ('one', 1, 50),
> ('two', 2, 30),
> ('three', 3, 30),
> ('four', 4, 60),
> ('five', 5, 70),
> ('six', 6, 80);
> 2. Try full outer with the below command. The result is incomplete, it is
> missing the row:
> NULL NULL NULL three 3 30.0
> Full Outer Join:
> select x1.`z`, x1.`x`, x1.`y`, x2.`z`,
> x2.`x`, x2.`y`
> from `x` x1 full outer join
> `x` x2 on (x1.`x` > 3) and (x2.`x` < 4) and (x1.`x` =
> x2.`x`);
> Result:
> --+
> x1.z x1.x x1.y x2.z x2.x x2.y
> --+
> one 1 50 NULL NULL NULL
> NULL NULL NULL one 1 50
> two 2 30 NULL NULL NULL
> NULL NULL NULL two 2 30
> three 3 30 NULL NULL NULL
> four 4 60 NULL NULL NULL
> NULL NULL NULL four 4 60
> five 5 70 NULL NULL NULL
> NULL NULL NULL five 5 70
> six 6 80 NULL NULL NULL
> NULL NULL NULL six 6 80
> --+
> 3. Expected output is coming when we use left/right join + union:
> select x1.`z`, x1.`x`, x1.`y`, x2.`z`,
> x2.`x`, x2.`y`
> from `x` x1 left outer join
> `x` x2 on (x1.`x` > 3) and (x2.`x` < 4) and (x1.`x` =
> x2.`x`)
> union
> select x1.`z`, x1.`x`, x1.`y`, x2.`z`,
> x2.`x`, x2.`y`
> from `x` x1 right outer join
> `x` x2 on (x1.`x` > 3) and (x2.`x` < 4) and (x1.`x` =
> x2.`x`);
> Result:
> +
> z x y _col3 _col4 _col5
> +
> NULL NULL NULL five 5 70
> NULL NULL NULL four 4 60
> NULL NULL NULL one 1 50
> four 4 60 NULL NULL NULL
> one 1 50 NULL NULL NULL
> six 6 80 NULL NULL NULL
> three 3 30 NULL NULL NULL
> two 2 30 NULL NULL NULL
> NULL NULL NULL six 6 80
> NULL NULL NULL three 3 30
> NULL NULL NULL two 2 30
> five 5 70 NULL NULL NULL
> +
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23514) Add Atlas metadata replication metrics
[
https://issues.apache.org/jira/browse/HIVE-23514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17121123#comment-17121123
]
Hive QA commented on HIVE-23514:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13004502/HIVE-23514.02.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17221 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22702/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22702/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22702/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13004502 - PreCommit-HIVE-Build
> Add Atlas metadata replication metrics
> --
>
> Key: HIVE-23514
> URL: https://issues.apache.org/jira/browse/HIVE-23514
> Project: Hive
> Issue Type: Task
>Reporter: Pravin Sinha
>Assignee: Pravin Sinha
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-23514.01.patch, HIVE-23514.02.patch,
> HIVE-23514.02.patch
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23498) Disable HTTP Trace method on ThriftHttpCliService
[ https://issues.apache.org/jira/browse/HIVE-23498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17120949#comment-17120949 ] Naveen Gangam commented on HIVE-23498: -- Latest patch looks good to me. +1 pending clean run. [~Rajkumar Singh] Please re-submit to re-run the tests. Thanks > Disable HTTP Trace method on ThriftHttpCliService > - > > Key: HIVE-23498 > URL: https://issues.apache.org/jira/browse/HIVE-23498 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Affects Versions: 3.1.2 >Reporter: Rajkumar Singh >Assignee: Rajkumar Singh >Priority: Major > Attachments: HIVE-23498.01.patch, HIVE-23498.01.patch, > HIVE-23498.02.patch, HIVE-23498.03.patch, HIVE-23498.04.patch, > HIVE-23498.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23586) load data overwrite into bucket table failed
[ https://issues.apache.org/jira/browse/HIVE-23586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhaolong updated HIVE-23586: Attachment: image-2020-06-01-21-40-21-726.png > load data overwrite into bucket table failed > > > Key: HIVE-23586 > URL: https://issues.apache.org/jira/browse/HIVE-23586 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.0, 3.1.2 >Reporter: zhaolong >Priority: Critical > Attachments: image-2020-06-01-21-40-21-726.png > > > load data overwrite into bucket table is failed if filename is not like > 00_0, but insert new data in the table. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23586) load data overwrite into bucket table failed
[ https://issues.apache.org/jira/browse/HIVE-23586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhaolong updated HIVE-23586: Description: load data overwrite into bucket table is failed if filename is not like 00_0, but insert new data in the table. for example: CREATE EXTERNAL TABLE IF NOT EXISTS test_hive2 (name string,account string) PARTITIONED BY (logdate string) CLUSTERED BY (account) INTO 4 BUCKETS row format delimited fields terminated by '|' STORED AS textfile; load data inpath 'hdfs://hacluster/tmp/zltest' overwrite into table default.test_hive2 partition (logdate='20200508'); !image-2020-06-01-21-40-21-726.png! load data inpath 'hdfs://hacluster/tmp/zltest' overwrite into table default.test_hive2 partition (logdate='20200508');// should overwrite buf insert new data !image-2020-06-01-21-41-28-732.png! was:load data overwrite into bucket table is failed if filename is not like 00_0, but insert new data in the table. > load data overwrite into bucket table failed > > > Key: HIVE-23586 > URL: https://issues.apache.org/jira/browse/HIVE-23586 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.0, 3.1.2 >Reporter: zhaolong >Priority: Critical > Attachments: image-2020-06-01-21-40-21-726.png, > image-2020-06-01-21-41-28-732.png > > > load data overwrite into bucket table is failed if filename is not like > 00_0, but insert new data in the table. > > for example: > CREATE EXTERNAL TABLE IF NOT EXISTS test_hive2 (name string,account string) > PARTITIONED BY (logdate string) CLUSTERED BY (account) INTO 4 BUCKETS row > format delimited fields terminated by '|' STORED AS textfile; > load data inpath 'hdfs://hacluster/tmp/zltest' overwrite into table > default.test_hive2 partition (logdate='20200508'); > !image-2020-06-01-21-40-21-726.png! > load data inpath 'hdfs://hacluster/tmp/zltest' overwrite into table > default.test_hive2 partition (logdate='20200508');// should overwrite buf > insert new data > !image-2020-06-01-21-41-28-732.png! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-13663) Create UDFs using existing functions in Hive instead from JAR files
[
https://issues.apache.org/jira/browse/HIVE-13663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nuthan Reddy Gantla resolved HIVE-13663.
Resolution: Abandoned
> Create UDFs using existing functions in Hive instead from JAR files
> ---
>
> Key: HIVE-13663
> URL: https://issues.apache.org/jira/browse/HIVE-13663
> Project: Hive
> Issue Type: Improvement
> Components: UDF
>Reporter: Nuthan Reddy Gantla
>Assignee: Jason Dere
>Priority: Minor
> Labels: features
>
> Recently i came across
> FIRST_DAY_OF_YEAR UDF in SQL which returns Jan 1st of that year.
> This could be made simply using existing functions in Hive as
> suppose our input date is '2016-03-04'
> concat(Year('2016-03-04'),'-01-01')
> instead of loading a JAR file. And moreover i guess using JAR has less
> performance when compared to the above implementation.
> We can also use UDFs for hiding complex functions from our query which makes
> it more readable.
> Kindly let me know if i'm wrong about performance.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22675) Replace Base64 in hive-standalone-metastore Package
[
https://issues.apache.org/jira/browse/HIVE-22675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17121057#comment-17121057
]
Hive QA commented on HIVE-22675:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13004497/HIVE-22675.3.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17219 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22701/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22701/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22701/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13004497 - PreCommit-HIVE-Build
> Replace Base64 in hive-standalone-metastore Package
> ---
>
> Key: HIVE-22675
> URL: https://issues.apache.org/jira/browse/HIVE-22675
> Project: Hive
> Issue Type: Sub-task
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HIVE-22675.1.patch, HIVE-22675.2.patch,
> HIVE-22675.3.patch, HIVE-22675.3.patch, HIVE-22675.3.patch,
> HIVE-22675.3.patch, HIVE-22675.3.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23587) Remove JODA Time From LlapServiceDriver
[ https://issues.apache.org/jira/browse/HIVE-23587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-23587: -- Description: Use JDK facilities instead. To keep it as simple as possible, and using only Java standard stuff, changing the existing format: From: "ddMMM" (25DEC2020) To "Basic ISO date": "mmdd" (25122020) https://docs.oracle.com/javase/8/docs/api/java/time/format/DateTimeFormatter.html#BASIC_ISO_DATE was:Use JDK facilities instead. > Remove JODA Time From LlapServiceDriver > --- > > Key: HIVE-23587 > URL: https://issues.apache.org/jira/browse/HIVE-23587 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > > Use JDK facilities instead. > To keep it as simple as possible, and using only Java standard stuff, > changing the existing format: > From: > "ddMMM" (25DEC2020) > To "Basic ISO date": > "mmdd" (25122020) > https://docs.oracle.com/javase/8/docs/api/java/time/format/DateTimeFormatter.html#BASIC_ISO_DATE -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23514) Add Atlas metadata replication metrics
[ https://issues.apache.org/jira/browse/HIVE-23514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pravin Sinha updated HIVE-23514: Attachment: HIVE-23514.02.patch > Add Atlas metadata replication metrics > -- > > Key: HIVE-23514 > URL: https://issues.apache.org/jira/browse/HIVE-23514 > Project: Hive > Issue Type: Task >Reporter: Pravin Sinha >Assignee: Pravin Sinha >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23514.01.patch, HIVE-23514.02.patch, > HIVE-23514.02.patch > > Time Spent: 1h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23514) Add Atlas metadata replication metrics
[ https://issues.apache.org/jira/browse/HIVE-23514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pravin Sinha updated HIVE-23514: Attachment: HIVE-23514.02.patch > Add Atlas metadata replication metrics > -- > > Key: HIVE-23514 > URL: https://issues.apache.org/jira/browse/HIVE-23514 > Project: Hive > Issue Type: Task >Reporter: Pravin Sinha >Assignee: Pravin Sinha >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23514.01.patch, HIVE-23514.02.patch > > Time Spent: 1h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23516) Store hive replication policy execution metrics in the relational DB
[ https://issues.apache.org/jira/browse/HIVE-23516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-23516: -- Labels: pull-request-available (was: ) > Store hive replication policy execution metrics in the relational DB > > > Key: HIVE-23516 > URL: https://issues.apache.org/jira/browse/HIVE-23516 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: Replication Metrics.pdf > > Time Spent: 10m > Remaining Estimate: 0h > > Details documented in the attached doc -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23516) Store hive replication policy execution metrics in the relational DB
[ https://issues.apache.org/jira/browse/HIVE-23516?focusedWorklogId=439494=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-439494 ] ASF GitHub Bot logged work on HIVE-23516: - Author: ASF GitHub Bot Created on: 01/Jun/20 12:00 Start Date: 01/Jun/20 12:00 Worklog Time Spent: 10m Work Description: aasha opened a new pull request #1044: URL: https://github.com/apache/hive/pull/1044 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 439494) Remaining Estimate: 0h Time Spent: 10m > Store hive replication policy execution metrics in the relational DB > > > Key: HIVE-23516 > URL: https://issues.apache.org/jira/browse/HIVE-23516 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Attachments: Replication Metrics.pdf > > Time Spent: 10m > Remaining Estimate: 0h > > Details documented in the attached doc -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work started] (HIVE-23516) Store hive replication policy execution metrics in the relational DB
[ https://issues.apache.org/jira/browse/HIVE-23516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-23516 started by Aasha Medhi. -- > Store hive replication policy execution metrics in the relational DB > > > Key: HIVE-23516 > URL: https://issues.apache.org/jira/browse/HIVE-23516 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Attachments: Replication Metrics.pdf > > > Details documented in the attached doc -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23514) Add Atlas metadata replication metrics
[
https://issues.apache.org/jira/browse/HIVE-23514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17121019#comment-17121019
]
Hive QA commented on HIVE-23514:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13004487/HIVE-23514.02.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17218 tests
executed
*Failed tests:*
{noformat}
org.apache.hive.service.cli.thrift.TestThriftCLIServiceWithHttp.org.apache.hive.service.cli.thrift.TestThriftCLIServiceWithHttp
(batchId=213)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22700/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22700/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22700/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13004487 - PreCommit-HIVE-Build
> Add Atlas metadata replication metrics
> --
>
> Key: HIVE-23514
> URL: https://issues.apache.org/jira/browse/HIVE-23514
> Project: Hive
> Issue Type: Task
>Reporter: Pravin Sinha
>Assignee: Pravin Sinha
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-23514.01.patch, HIVE-23514.02.patch
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23586) load data overwrite into bucket table failed
[ https://issues.apache.org/jira/browse/HIVE-23586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhaolong updated HIVE-23586: Attachment: image-2020-06-01-21-41-28-732.png > load data overwrite into bucket table failed > > > Key: HIVE-23586 > URL: https://issues.apache.org/jira/browse/HIVE-23586 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.0, 3.1.2 >Reporter: zhaolong >Priority: Critical > Attachments: image-2020-06-01-21-40-21-726.png, > image-2020-06-01-21-41-28-732.png > > > load data overwrite into bucket table is failed if filename is not like > 00_0, but insert new data in the table. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23587) Remove JODA Time From LlapServiceDriver
[ https://issues.apache.org/jira/browse/HIVE-23587?focusedWorklogId=439540=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-439540 ] ASF GitHub Bot logged work on HIVE-23587: - Author: ASF GitHub Bot Created on: 01/Jun/20 13:52 Start Date: 01/Jun/20 13:52 Worklog Time Spent: 10m Work Description: belugabehr opened a new pull request #1045: URL: https://github.com/apache/hive/pull/1045 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 439540) Remaining Estimate: 0h Time Spent: 10m > Remove JODA Time From LlapServiceDriver > --- > > Key: HIVE-23587 > URL: https://issues.apache.org/jira/browse/HIVE-23587 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > Use JDK facilities instead. > To keep it as simple as possible, and using only Java standard stuff, > changing the existing format: > From: > "ddMMM" (25DEC2020) > To "Basic ISO date": > "mmdd" (25122020) > https://docs.oracle.com/javase/8/docs/api/java/time/format/DateTimeFormatter.html#BASIC_ISO_DATE -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-22503) Harmonize JODA Time Version in Module hive-hcatalog-it-unit
[ https://issues.apache.org/jira/browse/HIVE-22503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor resolved HIVE-22503. --- Resolution: Won't Fix > Harmonize JODA Time Version in Module hive-hcatalog-it-unit > --- > > Key: HIVE-22503 > URL: https://issues.apache.org/jira/browse/HIVE-22503 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > > https://github.com/apache/hive/blob/078182ade4b76e810ca945354f4897dbe36ad5c2/itests/hcatalog-unit/pom.xml#L296 > Currently hard-coded as version 2.2 > See if we can get away with using the same version of Joda as the rest of the > Hive project. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-13189) Consider using Joda DateTimeFormatter instead of SimpleDateFormat in GenericUDFDateAdd
[
https://issues.apache.org/jira/browse/HIVE-13189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor resolved HIVE-13189.
---
Resolution: Won't Fix
Moving away from JODA time since it's no longer supported and has been rolled
into the JDK itself.
> Consider using Joda DateTimeFormatter instead of SimpleDateFormat in
> GenericUDFDateAdd
> --
>
> Key: HIVE-13189
> URL: https://issues.apache.org/jira/browse/HIVE-13189
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Reporter: Rajesh Balamohan
>Assignee: varun a kumar
>Priority: Major
> Attachments: HIVE-13189.1.patch
>
>
> Quite an amount was spent by tasks in trying to parse date string in
> GenericUDFDateAdd.
> {noformat}
> java.lang.Thread.State: RUNNABLE
> at java.text.DecimalFormat.subparse(DecimalFormat.java:1467)
> at java.text.DecimalFormat.parse(DecimalFormat.java:1268)
> at java.text.SimpleDateFormat.subParse(SimpleDateFormat.java:2088)
> at java.text.SimpleDateFormat.parse(SimpleDateFormat.java:1455)
> at java.text.DateFormat.parse(DateFormat.java:355)
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDFDateAdd.evaluate(GenericUDFDateAdd.java:172)
> at
> org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator._evaluate(ExprNodeGenericFuncEvaluator.java:186)
> at
> org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:77)
> at
> org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator$DeferredExprObject.get(ExprNodeGenericFuncEvaluator.java:87)
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPGreaterThan.evaluate(GenericUDFOPGreaterThan.java:80)
> at
> org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator._evaluate(ExprNodeGenericFuncEvaluator.java:186)
> at
> org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:77)
> at
> org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:65)
> at
> org.apache.hadoop.hive.ql.exec.FilterOperator.process(FilterOperator.java:108)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:838)
> at
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator.internalForward(CommonJoinOperator.java:644)
> {noformat}
> Joda DateTimeFormatter can be considered for better performance.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-21241) Migrate TimeStamp Parser From Joda Time
[
https://issues.apache.org/jira/browse/HIVE-21241?focusedWorklogId=439529=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-439529
]
ASF GitHub Bot logged work on HIVE-21241:
-
Author: ASF GitHub Bot
Created on: 01/Jun/20 13:32
Start Date: 01/Jun/20 13:32
Worklog Time Spent: 10m
Work Description: belugabehr closed pull request #531:
URL: https://github.com/apache/hive/pull/531
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 439529)
Time Spent: 0.5h (was: 20m)
> Migrate TimeStamp Parser From Joda Time
> ---
>
> Key: HIVE-21241
> URL: https://issues.apache.org/jira/browse/HIVE-21241
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 3.2.0
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-21241.1.patch, HIVE-21241.2.patch,
> HIVE-21241.3.patch, HIVE-21241.4.patch, HIVE-21241.5.patch, HIVE-21241.6.patch
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Hive uses Joda time for its TimeStampParser.
> {quote}
> Joda-Time is the de facto standard date and time library for Java prior to
> Java SE 8. Users are now asked to migrate to java.time (JSR-310).
> https://www.joda.org/joda-time/
> {quote}
> Migrate TimeStampParser to {{java.time}}
> I also added a couple new pre-canned timestamp parsers for convenience:
> * ISO 8601
> * RFC 1123
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-21241) Migrate TimeStamp Parser From Joda Time
[
https://issues.apache.org/jira/browse/HIVE-21241?focusedWorklogId=439528=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-439528
]
ASF GitHub Bot logged work on HIVE-21241:
-
Author: ASF GitHub Bot
Created on: 01/Jun/20 13:32
Start Date: 01/Jun/20 13:32
Worklog Time Spent: 10m
Work Description: belugabehr commented on pull request #531:
URL: https://github.com/apache/hive/pull/531#issuecomment-636862784
Merged to master (4.0).
https://issues.apache.org/jira/browse/HIVE-21241
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 439528)
Time Spent: 20m (was: 10m)
> Migrate TimeStamp Parser From Joda Time
> ---
>
> Key: HIVE-21241
> URL: https://issues.apache.org/jira/browse/HIVE-21241
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 3.2.0
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-21241.1.patch, HIVE-21241.2.patch,
> HIVE-21241.3.patch, HIVE-21241.4.patch, HIVE-21241.5.patch, HIVE-21241.6.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Hive uses Joda time for its TimeStampParser.
> {quote}
> Joda-Time is the de facto standard date and time library for Java prior to
> Java SE 8. Users are now asked to migrate to java.time (JSR-310).
> https://www.joda.org/joda-time/
> {quote}
> Migrate TimeStampParser to {{java.time}}
> I also added a couple new pre-canned timestamp parsers for convenience:
> * ISO 8601
> * RFC 1123
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-23587) Remove JODA Time From LlapServiceDriver
[ https://issues.apache.org/jira/browse/HIVE-23587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor reassigned HIVE-23587: - > Remove JODA Time From LlapServiceDriver > --- > > Key: HIVE-23587 > URL: https://issues.apache.org/jira/browse/HIVE-23587 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > > Use JDK facilities instead. -- This message was sent by Atlassian Jira (v8.3.4#803005)
