[jira] [Commented] (HIVE-23605) 'Wrong FS' error during _external_tables_info creation when staging location is remote
[
https://issues.apache.org/jira/browse/HIVE-23605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17126456#comment-17126456
]
Hive QA commented on HIVE-23605:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
36s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
8s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
52s{color} | {color:blue} ql in master has 1522 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
33s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
13s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
17s{color} | {color:red} itests/hive-unit: The patch generated 9 new + 70
unchanged - 0 fixed = 79 total (was 70) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 30m 33s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22753/dev-support/hive-personality.sh
|
| git revision | master / b5e3bcd |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22753/yetus/diff-checkstyle-itests_hive-unit.txt
|
| modules | C: ql itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22753/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> 'Wrong FS' error during _external_tables_info creation when staging location
> is remote
> --
>
> Key: HIVE-23605
> URL: https://issues.apache.org/jira/browse/HIVE-23605
> Project: Hive
> Issue Type: Bug
>Reporter: Pravin Sinha
>Assignee: Pravin Sinha
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-23605.01.patch, HIVE-23605.02.patch,
> HIVE-23605.02.patch, HIVE-23605.02.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When staging location is on target cluster, Repl Dump fails to create
> _external_tables_info file.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23516) Store hive replication policy execution metrics in the relational DB
[ https://issues.apache.org/jira/browse/HIVE-23516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-23516: --- Attachment: HIVE-23516.09.patch Status: Patch Available (was: In Progress) > Store hive replication policy execution metrics in the relational DB > > > Key: HIVE-23516 > URL: https://issues.apache.org/jira/browse/HIVE-23516 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23516.01.patch, HIVE-23516.02.patch, > HIVE-23516.03.patch, HIVE-23516.04.patch, HIVE-23516.05.patch, > HIVE-23516.06.patch, HIVE-23516.07.patch, HIVE-23516.08.patch, > HIVE-23516.09.patch, Replication Metrics.pdf > > Time Spent: 4.5h > Remaining Estimate: 0h > > Details documented in the attached doc -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23516) Store hive replication policy execution metrics in the relational DB
[ https://issues.apache.org/jira/browse/HIVE-23516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-23516: --- Status: In Progress (was: Patch Available) > Store hive replication policy execution metrics in the relational DB > > > Key: HIVE-23516 > URL: https://issues.apache.org/jira/browse/HIVE-23516 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23516.01.patch, HIVE-23516.02.patch, > HIVE-23516.03.patch, HIVE-23516.04.patch, HIVE-23516.05.patch, > HIVE-23516.06.patch, HIVE-23516.07.patch, HIVE-23516.08.patch, > HIVE-23516.09.patch, Replication Metrics.pdf > > Time Spent: 4.5h > Remaining Estimate: 0h > > Details documented in the attached doc -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23516) Store hive replication policy execution metrics in the relational DB
[ https://issues.apache.org/jira/browse/HIVE-23516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-23516: --- Status: In Progress (was: Patch Available) > Store hive replication policy execution metrics in the relational DB > > > Key: HIVE-23516 > URL: https://issues.apache.org/jira/browse/HIVE-23516 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23516.01.patch, HIVE-23516.02.patch, > HIVE-23516.03.patch, HIVE-23516.04.patch, HIVE-23516.05.patch, > HIVE-23516.06.patch, HIVE-23516.07.patch, HIVE-23516.08.patch, Replication > Metrics.pdf > > Time Spent: 4.5h > Remaining Estimate: 0h > > Details documented in the attached doc -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23516) Store hive replication policy execution metrics in the relational DB
[ https://issues.apache.org/jira/browse/HIVE-23516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-23516: --- Attachment: HIVE-23516.08.patch Status: Patch Available (was: In Progress) > Store hive replication policy execution metrics in the relational DB > > > Key: HIVE-23516 > URL: https://issues.apache.org/jira/browse/HIVE-23516 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23516.01.patch, HIVE-23516.02.patch, > HIVE-23516.03.patch, HIVE-23516.04.patch, HIVE-23516.05.patch, > HIVE-23516.06.patch, HIVE-23516.07.patch, HIVE-23516.08.patch, Replication > Metrics.pdf > > Time Spent: 4.5h > Remaining Estimate: 0h > > Details documented in the attached doc -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23614) Always pass HiveConfig to removeTempOrDuplicateFiles
[ https://issues.apache.org/jira/browse/HIVE-23614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17126432#comment-17126432 ] Zoltan Haindrich commented on HIVE-23614: - the timeframe aligns up when I was executing the kubernetes version upgrade...I think for around ~5 minutes the github events were discarded... sorry for the inconvinience > Always pass HiveConfig to removeTempOrDuplicateFiles > > > Key: HIVE-23614 > URL: https://issues.apache.org/jira/browse/HIVE-23614 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: John Sherman >Assignee: John Sherman >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > As part of HIVE-23354, we check the provided HiveConf for speculative > execution and throw an error if it is enabled. There is one path that did not > previously provide a HiveConf value which shows up in test failures in > runtime_skewjoin_mapjoin_spark.q, skewjoin.q and skewjoin_onesideskew.q > (which we do not run as part of pre-commit tests) or at least not through > TestCliDriver. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23614) Always pass HiveConfig to removeTempOrDuplicateFiles
[ https://issues.apache.org/jira/browse/HIVE-23614?focusedWorklogId=441690&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441690 ] ASF GitHub Bot logged work on HIVE-23614: - Author: ASF GitHub Bot Created on: 05/Jun/20 06:17 Start Date: 05/Jun/20 06:17 Worklog Time Spent: 10m Work Description: kgyrtkirk closed pull request #1059: URL: https://github.com/apache/hive/pull/1059 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 441690) Time Spent: 0.5h (was: 20m) > Always pass HiveConfig to removeTempOrDuplicateFiles > > > Key: HIVE-23614 > URL: https://issues.apache.org/jira/browse/HIVE-23614 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: John Sherman >Assignee: John Sherman >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > As part of HIVE-23354, we check the provided HiveConf for speculative > execution and throw an error if it is enabled. There is one path that did not > previously provide a HiveConf value which shows up in test failures in > runtime_skewjoin_mapjoin_spark.q, skewjoin.q and skewjoin_onesideskew.q > (which we do not run as part of pre-commit tests) or at least not through > TestCliDriver. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23614) Always pass HiveConfig to removeTempOrDuplicateFiles
[ https://issues.apache.org/jira/browse/HIVE-23614?focusedWorklogId=441691&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441691 ] ASF GitHub Bot logged work on HIVE-23614: - Author: ASF GitHub Bot Created on: 05/Jun/20 06:17 Start Date: 05/Jun/20 06:17 Worklog Time Spent: 10m Work Description: jfsii opened a new pull request #1059: URL: https://github.com/apache/hive/pull/1059 Change-Id: I3b734e0ab90e69e9a90452001b9171022d2e89eb This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 441691) Time Spent: 40m (was: 0.5h) > Always pass HiveConfig to removeTempOrDuplicateFiles > > > Key: HIVE-23614 > URL: https://issues.apache.org/jira/browse/HIVE-23614 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: John Sherman >Assignee: John Sherman >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > As part of HIVE-23354, we check the provided HiveConf for speculative > execution and throw an error if it is enabled. There is one path that did not > previously provide a HiveConf value which shows up in test failures in > runtime_skewjoin_mapjoin_spark.q, skewjoin.q and skewjoin_onesideskew.q > (which we do not run as part of pre-commit tests) or at least not through > TestCliDriver. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23614) Always pass HiveConfig to removeTempOrDuplicateFiles
[ https://issues.apache.org/jira/browse/HIVE-23614?focusedWorklogId=441688&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441688 ] ASF GitHub Bot logged work on HIVE-23614: - Author: ASF GitHub Bot Created on: 05/Jun/20 06:14 Start Date: 05/Jun/20 06:14 Worklog Time Spent: 10m Work Description: kgyrtkirk commented on pull request #1059: URL: https://github.com/apache/hive/pull/1059#issuecomment-639281098 . This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 441688) Time Spent: 20m (was: 10m) > Always pass HiveConfig to removeTempOrDuplicateFiles > > > Key: HIVE-23614 > URL: https://issues.apache.org/jira/browse/HIVE-23614 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: John Sherman >Assignee: John Sherman >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > As part of HIVE-23354, we check the provided HiveConf for speculative > execution and throw an error if it is enabled. There is one path that did not > previously provide a HiveConf value which shows up in test failures in > runtime_skewjoin_mapjoin_spark.q, skewjoin.q and skewjoin_onesideskew.q > (which we do not run as part of pre-commit tests) or at least not through > TestCliDriver. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23516) Store hive replication policy execution metrics in the relational DB
[ https://issues.apache.org/jira/browse/HIVE-23516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-23516: --- Status: In Progress (was: Patch Available) > Store hive replication policy execution metrics in the relational DB > > > Key: HIVE-23516 > URL: https://issues.apache.org/jira/browse/HIVE-23516 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23516.01.patch, HIVE-23516.02.patch, > HIVE-23516.03.patch, HIVE-23516.04.patch, HIVE-23516.05.patch, > HIVE-23516.06.patch, HIVE-23516.07.patch, Replication Metrics.pdf > > Time Spent: 4.5h > Remaining Estimate: 0h > > Details documented in the attached doc -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23516) Store hive replication policy execution metrics in the relational DB
[ https://issues.apache.org/jira/browse/HIVE-23516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-23516: --- Attachment: HIVE-23516.07.patch Status: Patch Available (was: In Progress) > Store hive replication policy execution metrics in the relational DB > > > Key: HIVE-23516 > URL: https://issues.apache.org/jira/browse/HIVE-23516 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23516.01.patch, HIVE-23516.02.patch, > HIVE-23516.03.patch, HIVE-23516.04.patch, HIVE-23516.05.patch, > HIVE-23516.06.patch, HIVE-23516.07.patch, Replication Metrics.pdf > > Time Spent: 4.5h > Remaining Estimate: 0h > > Details documented in the attached doc -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23605) 'Wrong FS' error during _external_tables_info creation when staging location is remote
[ https://issues.apache.org/jira/browse/HIVE-23605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pravin Sinha updated HIVE-23605: Attachment: HIVE-23605.02.patch > 'Wrong FS' error during _external_tables_info creation when staging location > is remote > -- > > Key: HIVE-23605 > URL: https://issues.apache.org/jira/browse/HIVE-23605 > Project: Hive > Issue Type: Bug >Reporter: Pravin Sinha >Assignee: Pravin Sinha >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23605.01.patch, HIVE-23605.02.patch, > HIVE-23605.02.patch, HIVE-23605.02.patch > > Time Spent: 10m > Remaining Estimate: 0h > > When staging location is on target cluster, Repl Dump fails to create > _external_tables_info file. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23605) 'Wrong FS' error during _external_tables_info creation when staging location is remote
[
https://issues.apache.org/jira/browse/HIVE-23605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17126399#comment-17126399
]
Hive QA commented on HIVE-23605:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13004873/HIVE-23605.02.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 17070 tests
executed
*Failed tests:*
{noformat}
TestStatsReplicationScenariosACID - did not produce a TEST-*.xml file (likely
timed out) (batchId=185)
org.apache.hive.jdbc.TestJdbcWithMiniHS2.org.apache.hive.jdbc.TestJdbcWithMiniHS2
(batchId=219)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22752/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22752/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22752/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13004873 - PreCommit-HIVE-Build
> 'Wrong FS' error during _external_tables_info creation when staging location
> is remote
> --
>
> Key: HIVE-23605
> URL: https://issues.apache.org/jira/browse/HIVE-23605
> Project: Hive
> Issue Type: Bug
>Reporter: Pravin Sinha
>Assignee: Pravin Sinha
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-23605.01.patch, HIVE-23605.02.patch,
> HIVE-23605.02.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When staging location is on target cluster, Repl Dump fails to create
> _external_tables_info file.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23605) 'Wrong FS' error during _external_tables_info creation when staging location is remote
[ https://issues.apache.org/jira/browse/HIVE-23605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pravin Sinha updated HIVE-23605: Attachment: HIVE-23605.02.patch > 'Wrong FS' error during _external_tables_info creation when staging location > is remote > -- > > Key: HIVE-23605 > URL: https://issues.apache.org/jira/browse/HIVE-23605 > Project: Hive > Issue Type: Bug >Reporter: Pravin Sinha >Assignee: Pravin Sinha >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23605.01.patch, HIVE-23605.02.patch, > HIVE-23605.02.patch > > Time Spent: 10m > Remaining Estimate: 0h > > When staging location is on target cluster, Repl Dump fails to create > _external_tables_info file. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23520) REPL: repl dump could add support for immutable dataset
[ https://issues.apache.org/jira/browse/HIVE-23520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-23520: -- Labels: pull-request-available (was: ) > REPL: repl dump could add support for immutable dataset > --- > > Key: HIVE-23520 > URL: https://issues.apache.org/jira/browse/HIVE-23520 > Project: Hive > Issue Type: Improvement >Reporter: Rajesh Balamohan >Assignee: Rajesh Balamohan >Priority: Minor > Labels: pull-request-available > Attachments: HIVE-23520.1.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Currently, "REPL DUMP" ends up copying entire dataset along with partition > information, stats etc in its dump folder. However, there are cases (e.g > large reference datasets), where we need a way to just retain metadata along > with partition information & stats. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23520) REPL: repl dump could add support for immutable dataset
[ https://issues.apache.org/jira/browse/HIVE-23520?focusedWorklogId=441662&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441662 ] ASF GitHub Bot logged work on HIVE-23520: - Author: ASF GitHub Bot Created on: 05/Jun/20 03:38 Start Date: 05/Jun/20 03:38 Worklog Time Spent: 10m Work Description: rbalamohan opened a new pull request #1060: URL: https://github.com/apache/hive/pull/1060 HIVE-23520: REPL: repl dump could add support for immutable dataset. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 441662) Remaining Estimate: 0h Time Spent: 10m > REPL: repl dump could add support for immutable dataset > --- > > Key: HIVE-23520 > URL: https://issues.apache.org/jira/browse/HIVE-23520 > Project: Hive > Issue Type: Improvement >Reporter: Rajesh Balamohan >Assignee: Rajesh Balamohan >Priority: Minor > Attachments: HIVE-23520.1.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Currently, "REPL DUMP" ends up copying entire dataset along with partition > information, stats etc in its dump folder. However, there are cases (e.g > large reference datasets), where we need a way to just retain metadata along > with partition information & stats. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23615) Null pointers should not be dereferenced
[ https://issues.apache.org/jira/browse/HIVE-23615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kvlasov updated HIVE-23615: --- Description: [This pull request|https://github.com/apache/hive/pull/62] is focused on resolving occurrences of Sonar rule squid:S2259 (was: [This pull request|https://github.com/apache/hive/pull/62] is focused on resolving occurrences of Sonar rule squid:S2259 - Null pointers should not be dereferenced) > Null pointers should not be dereferenced > > > Key: HIVE-23615 > URL: https://issues.apache.org/jira/browse/HIVE-23615 > Project: Hive > Issue Type: Bug >Reporter: kvlasov >Priority: Major > > [This pull request|https://github.com/apache/hive/pull/62] is focused on > resolving occurrences of Sonar rule squid:S2259 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23596) LLAP: Encode initial guaranteed task information in containerId
[ https://issues.apache.org/jira/browse/HIVE-23596?focusedWorklogId=441608&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441608 ] ASF GitHub Bot logged work on HIVE-23596: - Author: ASF GitHub Bot Created on: 05/Jun/20 00:30 Start Date: 05/Jun/20 00:30 Worklog Time Spent: 10m Work Description: mustafaiman closed pull request #1053: URL: https://github.com/apache/hive/pull/1053 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 441608) Time Spent: 20m (was: 10m) > LLAP: Encode initial guaranteed task information in containerId > --- > > Key: HIVE-23596 > URL: https://issues.apache.org/jira/browse/HIVE-23596 > Project: Hive > Issue Type: Improvement > Components: llap >Reporter: Mustafa Iman >Assignee: Mustafa Iman >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Time Spent: 20m > Remaining Estimate: 0h > > We should avoid calling LlapTaskScheduler to get initial isguaranteed flag > for all the tasks. It causes arbitrary delays in sending tasks out. Since > communicator is a single thread, any blocking there delays all the tasks. > There are [https://jira.apache.org/jira/browse/TEZ-4192] and > [https://jira.apache.org/jira/browse/HIVE-23589] for a proper solution to > this. However, that requires a Tez release which seems far right now. We can > replace the current hack with another hack that does not require locking. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23614) Always pass HiveConfig to removeTempOrDuplicateFiles
[ https://issues.apache.org/jira/browse/HIVE-23614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17126284#comment-17126284 ] Jesus Camacho Rodriguez commented on HIVE-23614: [~kgyrtkirk], any idea of what may be going on? > Always pass HiveConfig to removeTempOrDuplicateFiles > > > Key: HIVE-23614 > URL: https://issues.apache.org/jira/browse/HIVE-23614 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: John Sherman >Assignee: John Sherman >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > As part of HIVE-23354, we check the provided HiveConf for speculative > execution and throw an error if it is enabled. There is one path that did not > previously provide a HiveConf value which shows up in test failures in > runtime_skewjoin_mapjoin_spark.q, skewjoin.q and skewjoin_onesideskew.q > (which we do not run as part of pre-commit tests) or at least not through > TestCliDriver. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23614) Always pass HiveConfig to removeTempOrDuplicateFiles
[ https://issues.apache.org/jira/browse/HIVE-23614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17126283#comment-17126283 ] John Sherman commented on HIVE-23614: - [~jcamachorodriguez] This is my first time using the new PR based tests. I'm not sure what to expect. It did do this: [https://travis-ci.org/github/apache/hive/builds/694812403] but I don't see any test results and it seemed kind of quick. > Always pass HiveConfig to removeTempOrDuplicateFiles > > > Key: HIVE-23614 > URL: https://issues.apache.org/jira/browse/HIVE-23614 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: John Sherman >Assignee: John Sherman >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > As part of HIVE-23354, we check the provided HiveConf for speculative > execution and throw an error if it is enabled. There is one path that did not > previously provide a HiveConf value which shows up in test failures in > runtime_skewjoin_mapjoin_spark.q, skewjoin.q and skewjoin_onesideskew.q > (which we do not run as part of pre-commit tests) or at least not through > TestCliDriver. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23614) Always pass HiveConfig to removeTempOrDuplicateFiles
[ https://issues.apache.org/jira/browse/HIVE-23614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17126278#comment-17126278 ] Jesus Camacho Rodriguez commented on HIVE-23614: [~jfs], it seems the tests for the PR were not queued? (or at least I do not see the label 'tests pending') > Always pass HiveConfig to removeTempOrDuplicateFiles > > > Key: HIVE-23614 > URL: https://issues.apache.org/jira/browse/HIVE-23614 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: John Sherman >Assignee: John Sherman >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > As part of HIVE-23354, we check the provided HiveConf for speculative > execution and throw an error if it is enabled. There is one path that did not > previously provide a HiveConf value which shows up in test failures in > runtime_skewjoin_mapjoin_spark.q, skewjoin.q and skewjoin_onesideskew.q > (which we do not run as part of pre-commit tests) or at least not through > TestCliDriver. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23614) Always pass HiveConfig to removeTempOrDuplicateFiles
[ https://issues.apache.org/jira/browse/HIVE-23614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17126277#comment-17126277 ] Jesus Camacho Rodriguez commented on HIVE-23614: +1 > Always pass HiveConfig to removeTempOrDuplicateFiles > > > Key: HIVE-23614 > URL: https://issues.apache.org/jira/browse/HIVE-23614 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: John Sherman >Assignee: John Sherman >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > As part of HIVE-23354, we check the provided HiveConf for speculative > execution and throw an error if it is enabled. There is one path that did not > previously provide a HiveConf value which shows up in test failures in > runtime_skewjoin_mapjoin_spark.q, skewjoin.q and skewjoin_onesideskew.q > (which we do not run as part of pre-commit tests) or at least not through > TestCliDriver. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22634) Improperly SemanticException when filter is optimized to False on a partition table
[
https://issues.apache.org/jira/browse/HIVE-22634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17126275#comment-17126275
]
Jesus Camacho Rodriguez commented on HIVE-22634:
Patch LGTM. Can we add the test to the PR?
> Improperly SemanticException when filter is optimized to False on a partition
> table
> ---
>
> Key: HIVE-22634
> URL: https://issues.apache.org/jira/browse/HIVE-22634
> Project: Hive
> Issue Type: Improvement
>Reporter: EdisonWang
>Assignee: EdisonWang
>Priority: Minor
> Labels: pull-request-available
> Attachments: HIVE-22634.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When filter is optimized to False on a partition table, it will throw
> improperly SemanticException reporting that there is no partition predicate
> found.
> The step to reproduce is
> {code:java}
> set hive.strict.checks.no.partition.filter=true;
> CREATE TABLE test(id int, name string)PARTITIONED BY (`date` string);
> select * from test where `date` = '20191201' and 1<>1;
> {code}
>
> The above sql will throw "Queries against partitioned tables without a
> partition filter" exception.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-19653) Incorrect predicate pushdown for groupby with grouping sets
[
https://issues.apache.org/jira/browse/HIVE-19653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17126272#comment-17126272
]
Hive QA commented on HIVE-19653:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12927286/HIVE-19653.1.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22751/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22751/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22751/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2020-06-04 23:47:40.164
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-22751/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2020-06-04 23:47:40.167
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at b5e3bcd HIVE-23596: LLAP: Encode initial guaranteed task
information in containerId (Mustafa Iman, reviewed by Gopal V)
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at b5e3bcd HIVE-23596: LLAP: Encode initial guaranteed task
information in containerId (Mustafa Iman, reviewed by Gopal V)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2020-06-04 23:47:41.070
+ rm -rf ../yetus_PreCommit-HIVE-Build-22751
+ mkdir ../yetus_PreCommit-HIVE-Build-22751
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-22751
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-22751/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
Trying to apply the patch with -p0
error: patch failed: itests/src/test/resources/testconfiguration.properties:855
Falling back to three-way merge...
Applied patch to 'itests/src/test/resources/testconfiguration.properties' with
conflicts.
Going to apply patch with: git apply -p0
error: patch failed: itests/src/test/resources/testconfiguration.properties:855
Falling back to three-way merge...
Applied patch to 'itests/src/test/resources/testconfiguration.properties' with
conflicts.
U itests/src/test/resources/testconfiguration.properties
+ result=1
+ '[' 1 -ne 0 ']'
+ rm -rf yetus_PreCommit-HIVE-Build-22751
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12927286 - PreCommit-HIVE-Build
> Incorrect predicate pushdown for groupby with grouping sets
> ---
>
> Key: HIVE-19653
> URL: https://issues.apache.org/jira/browse/HIVE-19653
> Project: Hive
> Issue Type: Bug
> Components: Logical Optimizer
>Reporter: Zhang Li
>Assignee: Zhang Li
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-19653.1.patch, HIVE-19653.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Consider the following query:
> {code:java}
> CREATE TABLE T1(a STRING, b STRING, s BIGINT);
> INSERT OVERWRITE TABLE T1 VALUES ('', '', 123456);
> SELECT * FROM (
> SELECT a, b, sum(s)
> FROM T1
> GROUP BY a, b GROUPING SETS ((), (a), (b), (a, b))
> ) t WHERE a IS NOT NULL;
> {code}
> When hive.optimize.ppd is enabled (and hive.cbo.enable=false), the query will
> output:
> {code:java}
> NULL NULL123456
> NULL 123456
> NULL123456
> 123456
> {code}
> We can see the predicate "a IS NOT NULL" takes no effect, w
[jira] [Commented] (HIVE-23605) 'Wrong FS' error during _external_tables_info creation when staging location is remote
[
https://issues.apache.org/jira/browse/HIVE-23605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17126271#comment-17126271
]
Hive QA commented on HIVE-23605:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13004861/HIVE-23605.02.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17100 tests
executed
*Failed tests:*
{noformat}
TestStatsReplicationScenariosACID - did not produce a TEST-*.xml file (likely
timed out) (batchId=185)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22750/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22750/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22750/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13004861 - PreCommit-HIVE-Build
> 'Wrong FS' error during _external_tables_info creation when staging location
> is remote
> --
>
> Key: HIVE-23605
> URL: https://issues.apache.org/jira/browse/HIVE-23605
> Project: Hive
> Issue Type: Bug
>Reporter: Pravin Sinha
>Assignee: Pravin Sinha
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-23605.01.patch, HIVE-23605.02.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When staging location is on target cluster, Repl Dump fails to create
> _external_tables_info file.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (HIVE-19653) Incorrect predicate pushdown for groupby with grouping sets
[
https://issues.apache.org/jira/browse/HIVE-19653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17126258#comment-17126258
]
Zhihua Deng edited comment on HIVE-19653 at 6/4/20, 11:21 PM:
--
the issue has been idle for some time and still be there. If you are not
working on it, can I take over. Thanks [~richox]
was (Author: dengzh):
the issue has been idle for some time and still be there. If you are not
working on it, I will take over. Thanks [~richox]
> Incorrect predicate pushdown for groupby with grouping sets
> ---
>
> Key: HIVE-19653
> URL: https://issues.apache.org/jira/browse/HIVE-19653
> Project: Hive
> Issue Type: Bug
> Components: Logical Optimizer
>Reporter: Zhang Li
>Assignee: Zhang Li
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-19653.1.patch, HIVE-19653.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Consider the following query:
> {code:java}
> CREATE TABLE T1(a STRING, b STRING, s BIGINT);
> INSERT OVERWRITE TABLE T1 VALUES ('', '', 123456);
> SELECT * FROM (
> SELECT a, b, sum(s)
> FROM T1
> GROUP BY a, b GROUPING SETS ((), (a), (b), (a, b))
> ) t WHERE a IS NOT NULL;
> {code}
> When hive.optimize.ppd is enabled (and hive.cbo.enable=false), the query will
> output:
> {code:java}
> NULL NULL123456
> NULL 123456
> NULL123456
> 123456
> {code}
> We can see the predicate "a IS NOT NULL" takes no effect, which is incorrect.
> When performing PPD optimization for a GBY operator, we should make sure all
> grouping sets contains the processing expr before pushdown. otherwise the
> expr value after GBY is changed and the result is wrong.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (HIVE-19653) Incorrect predicate pushdown for groupby with grouping sets
[
https://issues.apache.org/jira/browse/HIVE-19653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17126258#comment-17126258
]
Zhihua Deng edited comment on HIVE-19653 at 6/4/20, 11:21 PM:
--
the issue has been idle for some time and still be there. If you are not
working on it, can I take over? Thanks [~richox]
was (Author: dengzh):
the issue has been idle for some time and still be there. If you are not
working on it, can I take over. Thanks [~richox]
> Incorrect predicate pushdown for groupby with grouping sets
> ---
>
> Key: HIVE-19653
> URL: https://issues.apache.org/jira/browse/HIVE-19653
> Project: Hive
> Issue Type: Bug
> Components: Logical Optimizer
>Reporter: Zhang Li
>Assignee: Zhang Li
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-19653.1.patch, HIVE-19653.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Consider the following query:
> {code:java}
> CREATE TABLE T1(a STRING, b STRING, s BIGINT);
> INSERT OVERWRITE TABLE T1 VALUES ('', '', 123456);
> SELECT * FROM (
> SELECT a, b, sum(s)
> FROM T1
> GROUP BY a, b GROUPING SETS ((), (a), (b), (a, b))
> ) t WHERE a IS NOT NULL;
> {code}
> When hive.optimize.ppd is enabled (and hive.cbo.enable=false), the query will
> output:
> {code:java}
> NULL NULL123456
> NULL 123456
> NULL123456
> 123456
> {code}
> We can see the predicate "a IS NOT NULL" takes no effect, which is incorrect.
> When performing PPD optimization for a GBY operator, we should make sure all
> grouping sets contains the processing expr before pushdown. otherwise the
> expr value after GBY is changed and the result is wrong.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-19653) Incorrect predicate pushdown for groupby with grouping sets
[
https://issues.apache.org/jira/browse/HIVE-19653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17126258#comment-17126258
]
Zhihua Deng commented on HIVE-19653:
the issue has been idle for some time and still be there. If you are not
working on it, I will take over. Thanks [~richox]
> Incorrect predicate pushdown for groupby with grouping sets
> ---
>
> Key: HIVE-19653
> URL: https://issues.apache.org/jira/browse/HIVE-19653
> Project: Hive
> Issue Type: Bug
> Components: Logical Optimizer
>Reporter: Zhang Li
>Assignee: Zhang Li
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-19653.1.patch, HIVE-19653.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Consider the following query:
> {code:java}
> CREATE TABLE T1(a STRING, b STRING, s BIGINT);
> INSERT OVERWRITE TABLE T1 VALUES ('', '', 123456);
> SELECT * FROM (
> SELECT a, b, sum(s)
> FROM T1
> GROUP BY a, b GROUPING SETS ((), (a), (b), (a, b))
> ) t WHERE a IS NOT NULL;
> {code}
> When hive.optimize.ppd is enabled (and hive.cbo.enable=false), the query will
> output:
> {code:java}
> NULL NULL123456
> NULL 123456
> NULL123456
> 123456
> {code}
> We can see the predicate "a IS NOT NULL" takes no effect, which is incorrect.
> When performing PPD optimization for a GBY operator, we should make sure all
> grouping sets contains the processing expr before pushdown. otherwise the
> expr value after GBY is changed and the result is wrong.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23605) 'Wrong FS' error during _external_tables_info creation when staging location is remote
[
https://issues.apache.org/jira/browse/HIVE-23605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17126256#comment-17126256
]
Hive QA commented on HIVE-23605:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
59s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
50s{color} | {color:blue} ql in master has 1522 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
31s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
11s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
36s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
16s{color} | {color:red} itests/hive-unit: The patch generated 9 new + 70
unchanged - 0 fixed = 79 total (was 70) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 29m 46s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22750/dev-support/hive-personality.sh
|
| git revision | master / b5e3bcd |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22750/yetus/diff-checkstyle-itests_hive-unit.txt
|
| modules | C: ql itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22750/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> 'Wrong FS' error during _external_tables_info creation when staging location
> is remote
> --
>
> Key: HIVE-23605
> URL: https://issues.apache.org/jira/browse/HIVE-23605
> Project: Hive
> Issue Type: Bug
>Reporter: Pravin Sinha
>Assignee: Pravin Sinha
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-23605.01.patch, HIVE-23605.02.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When staging location is on target cluster, Repl Dump fails to create
> _external_tables_info file.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23605) 'Wrong FS' error during _external_tables_info creation when staging location is remote
[ https://issues.apache.org/jira/browse/HIVE-23605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pravin Sinha updated HIVE-23605: Attachment: HIVE-23605.02.patch > 'Wrong FS' error during _external_tables_info creation when staging location > is remote > -- > > Key: HIVE-23605 > URL: https://issues.apache.org/jira/browse/HIVE-23605 > Project: Hive > Issue Type: Bug >Reporter: Pravin Sinha >Assignee: Pravin Sinha >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23605.01.patch, HIVE-23605.02.patch > > Time Spent: 10m > Remaining Estimate: 0h > > When staging location is on target cluster, Repl Dump fails to create > _external_tables_info file. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22634) Improperly SemanticException when filter is optimized to False on a partition table
[
https://issues.apache.org/jira/browse/HIVE-22634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17126220#comment-17126220
]
David Mollitor commented on HIVE-22634:
---
[~jcamachorodriguez] Are you able to take a look at this one?
> Improperly SemanticException when filter is optimized to False on a partition
> table
> ---
>
> Key: HIVE-22634
> URL: https://issues.apache.org/jira/browse/HIVE-22634
> Project: Hive
> Issue Type: Improvement
>Reporter: EdisonWang
>Assignee: EdisonWang
>Priority: Minor
> Labels: pull-request-available
> Attachments: HIVE-22634.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When filter is optimized to False on a partition table, it will throw
> improperly SemanticException reporting that there is no partition predicate
> found.
> The step to reproduce is
> {code:java}
> set hive.strict.checks.no.partition.filter=true;
> CREATE TABLE test(id int, name string)PARTITIONED BY (`date` string);
> select * from test where `date` = '20191201' and 1<>1;
> {code}
>
> The above sql will throw "Queries against partitioned tables without a
> partition filter" exception.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-1360) Allow UDFs to access constant parameter values at compile time
[ https://issues.apache.org/jira/browse/HIVE-1360?focusedWorklogId=441555&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441555 ] ASF GitHub Bot logged work on HIVE-1360: Author: ASF GitHub Bot Created on: 04/Jun/20 21:55 Start Date: 04/Jun/20 21:55 Worklog Time Spent: 10m Work Description: belugabehr commented on pull request #2: URL: https://github.com/apache/hive/pull/2#issuecomment-639139765 Issue fixed through manual patch process: https://issues.apache.org/jira/browse/HIVE-1360 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 441555) Time Spent: 20m (was: 10m) > Allow UDFs to access constant parameter values at compile time > -- > > Key: HIVE-1360 > URL: https://issues.apache.org/jira/browse/HIVE-1360 > Project: Hive > Issue Type: Improvement > Components: Query Processor, UDF >Affects Versions: 0.5.0 >Reporter: Carl Steinbach >Assignee: Jonathan Chang >Priority: Major > Labels: pull-request-available > Fix For: 0.8.0 > > Attachments: HIVE-1360.2.patch.txt, HIVE-1360.2.test-failures.txt, > HIVE-1360.3.patch, HIVE-1360.part2.patch, HIVE-1360.patch, HIVE-1360.patch > > Time Spent: 20m > Remaining Estimate: 0h > > UDFs should be able to access constant parameter values at compile time. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-1360) Allow UDFs to access constant parameter values at compile time
[ https://issues.apache.org/jira/browse/HIVE-1360?focusedWorklogId=441554&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441554 ] ASF GitHub Bot logged work on HIVE-1360: Author: ASF GitHub Bot Created on: 04/Jun/20 21:55 Start Date: 04/Jun/20 21:55 Worklog Time Spent: 10m Work Description: belugabehr closed pull request #2: URL: https://github.com/apache/hive/pull/2 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 441554) Remaining Estimate: 0h Time Spent: 10m > Allow UDFs to access constant parameter values at compile time > -- > > Key: HIVE-1360 > URL: https://issues.apache.org/jira/browse/HIVE-1360 > Project: Hive > Issue Type: Improvement > Components: Query Processor, UDF >Affects Versions: 0.5.0 >Reporter: Carl Steinbach >Assignee: Jonathan Chang >Priority: Major > Fix For: 0.8.0 > > Attachments: HIVE-1360.2.patch.txt, HIVE-1360.2.test-failures.txt, > HIVE-1360.3.patch, HIVE-1360.part2.patch, HIVE-1360.patch, HIVE-1360.patch > > Time Spent: 10m > Remaining Estimate: 0h > > UDFs should be able to access constant parameter values at compile time. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-1360) Allow UDFs to access constant parameter values at compile time
[ https://issues.apache.org/jira/browse/HIVE-1360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-1360: - Labels: pull-request-available (was: ) > Allow UDFs to access constant parameter values at compile time > -- > > Key: HIVE-1360 > URL: https://issues.apache.org/jira/browse/HIVE-1360 > Project: Hive > Issue Type: Improvement > Components: Query Processor, UDF >Affects Versions: 0.5.0 >Reporter: Carl Steinbach >Assignee: Jonathan Chang >Priority: Major > Labels: pull-request-available > Fix For: 0.8.0 > > Attachments: HIVE-1360.2.patch.txt, HIVE-1360.2.test-failures.txt, > HIVE-1360.3.patch, HIVE-1360.part2.patch, HIVE-1360.patch, HIVE-1360.patch > > Time Spent: 10m > Remaining Estimate: 0h > > UDFs should be able to access constant parameter values at compile time. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-10296) Cast exception observed when hive runs a multi join query on metastore (postgres), since postgres pushes the filter into the join, and ignores the condition before applyi
[
https://issues.apache.org/jira/browse/HIVE-10296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-10296:
--
Labels: pull-request-available (was: )
> Cast exception observed when hive runs a multi join query on metastore
> (postgres), since postgres pushes the filter into the join, and ignores the
> condition before applying cast
> -
>
> Key: HIVE-10296
> URL: https://issues.apache.org/jira/browse/HIVE-10296
> Project: Hive
> Issue Type: Bug
>Affects Versions: 0.13.1
>Reporter: Yash Datta
>Assignee: Karthik Manamcheri
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0, 3.2.0
>
> Attachments: HIVE-10296.1.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Try to drop a partition from hive:
> ALTER TABLE f___edr_bin_source___900_sub_id DROP IF EXISTS PARTITION (
> exporttimestamp=1427824800, timestamp=1427824800)
> This triggers a query on the metastore like this :
> "select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on
> "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join
> "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? inner join
> "PARTITION_KEY_VALS" "FILTER0" on "FILTER0"."PART_ID" =
> "PARTITIONS"."PART_ID" and "FILTER0"."INTEGER_IDX" = 0 inner join
> "PARTITION_KEY_VALS" "FILTER1" on "FILTER1"."PART_ID" =
> "PARTITIONS"."PART_ID" and "FILTER1"."INTEGER_IDX" = 1 where ( (((case when
> "TBLS"."TBL_NAME" = ? and "DBS"."NAME" = ? then cast("FILTER0"."PART_KEY_VAL"
> as decimal(21,0)) else null end) = ?) and ((case when "TBLS"."TBL_NAME" = ?
> and "DBS"."NAME" = ? then cast("FILTER1"."PART_KEY_VAL" as decimal(21,0))
> else null end) = ?)) )"
> In some cases, when the internal tables in postgres (metastore) have some
> amount of data, the query plan pushes the condition down into the join.
> Now because of DERBY-6358 , case when clause is used before the cast, but in
> this case , cast is evaluated before condition being evaluated. So in case we
> have different tables partitioned on string and integer columns, cast
> exception is observed!
> 15/04/06 08:41:20 ERROR metastore.ObjectStore: Direct SQL failed, falling
> back to ORM
> javax.jdo.JDODataStoreException: Error executing SQL query "select
> "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on
> "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join
> "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? inner join
> "PARTITION_KEY_VALS" "FILTER0" on "FILTER0"."PART_ID" =
> "PARTITIONS"."PART_ID" and "FILTER0"."INTEGER_IDX" = 0 inner join
> "PARTITION_KEY_VALS" "FILTER1" on "FILTER1"."PART_ID" =
> "PARTITIONS"."PART_ID" and "FILTER1"."INTEGER_IDX" = 1 where ( (((case when
> "TBLS"."TBL_NAME" = ? and "DBS"."NAME" = ? then cast("FILTER0"."PART_KEY_VAL"
> as decimal(21,0)) else null end) = ?) and ((case when "TBLS"."TBL_NAME" = ?
> and "DBS"."NAME" = ? then cast("FILTER1"."PART_KEY_VAL" as decimal(21,0))
> else null end) = ?)) )".
> at
> org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:451)
>
> at
> org.datanucleus.api.jdo.JDOQuery.executeWithArray(JDOQuery.java:321)
> at
> org.apache.hadoop.hive.metastore.MetaStoreDirectSql.getPartitionsViaSqlFilterInternal(MetaStoreDirectSql.java:300)
>
> at
> org.apache.hadoop.hive.metastore.MetaStoreDirectSql.getPartitionsViaSqlFilter(MetaStoreDirectSql.java:211)
>
> at
> org.apache.hadoop.hive.metastore.ObjectStore$3.getSqlResult(ObjectStore.java:1915)
>
> at
> org.apache.hadoop.hive.metastore.ObjectStore$3.getSqlResult(ObjectStore.java:1909)
>
> at
> org.apache.hadoop.hive.metastore.ObjectStore$GetHelper.run(ObjectStore.java:2208)
>
> at
> org.apache.hadoop.hive.metastore.ObjectStore.getPartitionsByExprInternal(ObjectStore.java:1909)
>
> at
> org.apache.hadoop.hive.metastore.ObjectStore.getPartitionsByExpr(ObjectStore.java:1882)
>
> org.postgresql.util.PSQLException: ERROR: invalid input syntax for type
> numeric: "__DEFAULT_BINSRC__"
> 15/04/06 08:41:20 INFO metastore.ObjectStore: JDO filter pushdown cannot be
> used: Filtering is supported only on partition keys of type string
> 15/04/06 08:41:20 ERROR metastore.ObjectStore:
> javax.jdo.JDOException: Exception thrown when executing query
> at
> org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:596)
>
> at org.datanucleus.api.jdo.JDOQuery.execute(JDOQuery.java:275)
> at
> org.apache.hadoop.hive.
[jira] [Work logged] (HIVE-10296) Cast exception observed when hive runs a multi join query on metastore (postgres), since postgres pushes the filter into the join, and ignores the condition before ap
[
https://issues.apache.org/jira/browse/HIVE-10296?focusedWorklogId=441553&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441553
]
ASF GitHub Bot logged work on HIVE-10296:
-
Author: ASF GitHub Bot
Created on: 04/Jun/20 21:54
Start Date: 04/Jun/20 21:54
Worklog Time Spent: 10m
Work Description: belugabehr commented on pull request #32:
URL: https://github.com/apache/hive/pull/32#issuecomment-639139359
Issue resolve through manual patch process:
https://issues.apache.org/jira/browse/HIVE-10296
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 441553)
Time Spent: 20m (was: 10m)
> Cast exception observed when hive runs a multi join query on metastore
> (postgres), since postgres pushes the filter into the join, and ignores the
> condition before applying cast
> -
>
> Key: HIVE-10296
> URL: https://issues.apache.org/jira/browse/HIVE-10296
> Project: Hive
> Issue Type: Bug
>Affects Versions: 0.13.1
>Reporter: Yash Datta
>Assignee: Karthik Manamcheri
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0, 3.2.0
>
> Attachments: HIVE-10296.1.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Try to drop a partition from hive:
> ALTER TABLE f___edr_bin_source___900_sub_id DROP IF EXISTS PARTITION (
> exporttimestamp=1427824800, timestamp=1427824800)
> This triggers a query on the metastore like this :
> "select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on
> "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join
> "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? inner join
> "PARTITION_KEY_VALS" "FILTER0" on "FILTER0"."PART_ID" =
> "PARTITIONS"."PART_ID" and "FILTER0"."INTEGER_IDX" = 0 inner join
> "PARTITION_KEY_VALS" "FILTER1" on "FILTER1"."PART_ID" =
> "PARTITIONS"."PART_ID" and "FILTER1"."INTEGER_IDX" = 1 where ( (((case when
> "TBLS"."TBL_NAME" = ? and "DBS"."NAME" = ? then cast("FILTER0"."PART_KEY_VAL"
> as decimal(21,0)) else null end) = ?) and ((case when "TBLS"."TBL_NAME" = ?
> and "DBS"."NAME" = ? then cast("FILTER1"."PART_KEY_VAL" as decimal(21,0))
> else null end) = ?)) )"
> In some cases, when the internal tables in postgres (metastore) have some
> amount of data, the query plan pushes the condition down into the join.
> Now because of DERBY-6358 , case when clause is used before the cast, but in
> this case , cast is evaluated before condition being evaluated. So in case we
> have different tables partitioned on string and integer columns, cast
> exception is observed!
> 15/04/06 08:41:20 ERROR metastore.ObjectStore: Direct SQL failed, falling
> back to ORM
> javax.jdo.JDODataStoreException: Error executing SQL query "select
> "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on
> "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join
> "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? inner join
> "PARTITION_KEY_VALS" "FILTER0" on "FILTER0"."PART_ID" =
> "PARTITIONS"."PART_ID" and "FILTER0"."INTEGER_IDX" = 0 inner join
> "PARTITION_KEY_VALS" "FILTER1" on "FILTER1"."PART_ID" =
> "PARTITIONS"."PART_ID" and "FILTER1"."INTEGER_IDX" = 1 where ( (((case when
> "TBLS"."TBL_NAME" = ? and "DBS"."NAME" = ? then cast("FILTER0"."PART_KEY_VAL"
> as decimal(21,0)) else null end) = ?) and ((case when "TBLS"."TBL_NAME" = ?
> and "DBS"."NAME" = ? then cast("FILTER1"."PART_KEY_VAL" as decimal(21,0))
> else null end) = ?)) )".
> at
> org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:451)
>
> at
> org.datanucleus.api.jdo.JDOQuery.executeWithArray(JDOQuery.java:321)
> at
> org.apache.hadoop.hive.metastore.MetaStoreDirectSql.getPartitionsViaSqlFilterInternal(MetaStoreDirectSql.java:300)
>
> at
> org.apache.hadoop.hive.metastore.MetaStoreDirectSql.getPartitionsViaSqlFilter(MetaStoreDirectSql.java:211)
>
> at
> org.apache.hadoop.hive.metastore.ObjectStore$3.getSqlResult(ObjectStore.java:1915)
>
> at
> org.apache.hadoop.hive.metastore.ObjectStore$3.getSqlResult(ObjectStore.java:1909)
>
> at
> org.apache.hadoop.hive.metastore.ObjectStore$GetHelper.run(ObjectStore.java:2208)
>
>
[jira] [Work logged] (HIVE-11134) HS2 should log open session failure
[ https://issues.apache.org/jira/browse/HIVE-11134?focusedWorklogId=441551&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441551 ] ASF GitHub Bot logged work on HIVE-11134: - Author: ASF GitHub Bot Created on: 04/Jun/20 21:54 Start Date: 04/Jun/20 21:54 Worklog Time Spent: 10m Work Description: belugabehr closed pull request #43: URL: https://github.com/apache/hive/pull/43 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 441551) Time Spent: 20m (was: 10m) > HS2 should log open session failure > --- > > Key: HIVE-11134 > URL: https://issues.apache.org/jira/browse/HIVE-11134 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Thejas Nair >Assignee: Thejas Nair >Priority: Major > Labels: pull-request-available > Fix For: 2.0.0 > > Attachments: HIVE-11134.1.patch > > Time Spent: 20m > Remaining Estimate: 0h > > HiveServer2 should log OpenSession failure. If beeline is not running with > "--verbose=true" all stack trace information is not available for later > debugging, as it is not currently logged in server side. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-10296) Cast exception observed when hive runs a multi join query on metastore (postgres), since postgres pushes the filter into the join, and ignores the condition before ap
[
https://issues.apache.org/jira/browse/HIVE-10296?focusedWorklogId=441552&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441552
]
ASF GitHub Bot logged work on HIVE-10296:
-
Author: ASF GitHub Bot
Created on: 04/Jun/20 21:54
Start Date: 04/Jun/20 21:54
Worklog Time Spent: 10m
Work Description: belugabehr closed pull request #32:
URL: https://github.com/apache/hive/pull/32
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 441552)
Remaining Estimate: 0h
Time Spent: 10m
> Cast exception observed when hive runs a multi join query on metastore
> (postgres), since postgres pushes the filter into the join, and ignores the
> condition before applying cast
> -
>
> Key: HIVE-10296
> URL: https://issues.apache.org/jira/browse/HIVE-10296
> Project: Hive
> Issue Type: Bug
>Affects Versions: 0.13.1
>Reporter: Yash Datta
>Assignee: Karthik Manamcheri
>Priority: Major
> Fix For: 4.0.0, 3.2.0
>
> Attachments: HIVE-10296.1.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Try to drop a partition from hive:
> ALTER TABLE f___edr_bin_source___900_sub_id DROP IF EXISTS PARTITION (
> exporttimestamp=1427824800, timestamp=1427824800)
> This triggers a query on the metastore like this :
> "select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on
> "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join
> "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? inner join
> "PARTITION_KEY_VALS" "FILTER0" on "FILTER0"."PART_ID" =
> "PARTITIONS"."PART_ID" and "FILTER0"."INTEGER_IDX" = 0 inner join
> "PARTITION_KEY_VALS" "FILTER1" on "FILTER1"."PART_ID" =
> "PARTITIONS"."PART_ID" and "FILTER1"."INTEGER_IDX" = 1 where ( (((case when
> "TBLS"."TBL_NAME" = ? and "DBS"."NAME" = ? then cast("FILTER0"."PART_KEY_VAL"
> as decimal(21,0)) else null end) = ?) and ((case when "TBLS"."TBL_NAME" = ?
> and "DBS"."NAME" = ? then cast("FILTER1"."PART_KEY_VAL" as decimal(21,0))
> else null end) = ?)) )"
> In some cases, when the internal tables in postgres (metastore) have some
> amount of data, the query plan pushes the condition down into the join.
> Now because of DERBY-6358 , case when clause is used before the cast, but in
> this case , cast is evaluated before condition being evaluated. So in case we
> have different tables partitioned on string and integer columns, cast
> exception is observed!
> 15/04/06 08:41:20 ERROR metastore.ObjectStore: Direct SQL failed, falling
> back to ORM
> javax.jdo.JDODataStoreException: Error executing SQL query "select
> "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on
> "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join
> "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? inner join
> "PARTITION_KEY_VALS" "FILTER0" on "FILTER0"."PART_ID" =
> "PARTITIONS"."PART_ID" and "FILTER0"."INTEGER_IDX" = 0 inner join
> "PARTITION_KEY_VALS" "FILTER1" on "FILTER1"."PART_ID" =
> "PARTITIONS"."PART_ID" and "FILTER1"."INTEGER_IDX" = 1 where ( (((case when
> "TBLS"."TBL_NAME" = ? and "DBS"."NAME" = ? then cast("FILTER0"."PART_KEY_VAL"
> as decimal(21,0)) else null end) = ?) and ((case when "TBLS"."TBL_NAME" = ?
> and "DBS"."NAME" = ? then cast("FILTER1"."PART_KEY_VAL" as decimal(21,0))
> else null end) = ?)) )".
> at
> org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:451)
>
> at
> org.datanucleus.api.jdo.JDOQuery.executeWithArray(JDOQuery.java:321)
> at
> org.apache.hadoop.hive.metastore.MetaStoreDirectSql.getPartitionsViaSqlFilterInternal(MetaStoreDirectSql.java:300)
>
> at
> org.apache.hadoop.hive.metastore.MetaStoreDirectSql.getPartitionsViaSqlFilter(MetaStoreDirectSql.java:211)
>
> at
> org.apache.hadoop.hive.metastore.ObjectStore$3.getSqlResult(ObjectStore.java:1915)
>
> at
> org.apache.hadoop.hive.metastore.ObjectStore$3.getSqlResult(ObjectStore.java:1909)
>
> at
> org.apache.hadoop.hive.metastore.ObjectStore$GetHelper.run(ObjectStore.java:2208)
>
> at
> org.apache.hadoop.hive.metastore.ObjectStore.getPartitionsByExprInternal(ObjectStore.java:1909)
>
> at
> org.apache.hado
[jira] [Work logged] (HIVE-11134) HS2 should log open session failure
[ https://issues.apache.org/jira/browse/HIVE-11134?focusedWorklogId=441550&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441550 ] ASF GitHub Bot logged work on HIVE-11134: - Author: ASF GitHub Bot Created on: 04/Jun/20 21:54 Start Date: 04/Jun/20 21:54 Worklog Time Spent: 10m Work Description: belugabehr commented on pull request #43: URL: https://github.com/apache/hive/pull/43#issuecomment-639139100 Issue resolved through manual patch process: https://issues.apache.org/jira/browse/HIVE-11134 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 441550) Remaining Estimate: 0h Time Spent: 10m > HS2 should log open session failure > --- > > Key: HIVE-11134 > URL: https://issues.apache.org/jira/browse/HIVE-11134 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Thejas Nair >Assignee: Thejas Nair >Priority: Major > Fix For: 2.0.0 > > Attachments: HIVE-11134.1.patch > > Time Spent: 10m > Remaining Estimate: 0h > > HiveServer2 should log OpenSession failure. If beeline is not running with > "--verbose=true" all stack trace information is not available for later > debugging, as it is not currently logged in server side. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-11134) HS2 should log open session failure
[ https://issues.apache.org/jira/browse/HIVE-11134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-11134: -- Labels: pull-request-available (was: ) > HS2 should log open session failure > --- > > Key: HIVE-11134 > URL: https://issues.apache.org/jira/browse/HIVE-11134 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Thejas Nair >Assignee: Thejas Nair >Priority: Major > Labels: pull-request-available > Fix For: 2.0.0 > > Attachments: HIVE-11134.1.patch > > Time Spent: 10m > Remaining Estimate: 0h > > HiveServer2 should log OpenSession failure. If beeline is not running with > "--verbose=true" all stack trace information is not available for later > debugging, as it is not currently logged in server side. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-12261) schematool version info exit status should depend on compatibility, not equality
[ https://issues.apache.org/jira/browse/HIVE-12261?focusedWorklogId=441549&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441549 ] ASF GitHub Bot logged work on HIVE-12261: - Author: ASF GitHub Bot Created on: 04/Jun/20 21:52 Start Date: 04/Jun/20 21:52 Worklog Time Spent: 10m Work Description: belugabehr closed pull request #51: URL: https://github.com/apache/hive/pull/51 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 441549) Time Spent: 20m (was: 10m) > schematool version info exit status should depend on compatibility, not > equality > > > Key: HIVE-12261 > URL: https://issues.apache.org/jira/browse/HIVE-12261 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 1.3.0, 2.0.0 >Reporter: Thejas Nair >Assignee: Thejas Nair >Priority: Major > Labels: pull-request-available > Fix For: 1.3.0, 2.0.0 > > Attachments: HIVE-12261-branch-1.0.0.patch, > HIVE-12261-branch-1.patch, HIVE-12261.1.patch > > Time Spent: 20m > Remaining Estimate: 0h > > Newer versions of metastore schema are compatible with older versions of > hive, as only new tables or columns are added with additional information. > HIVE-11613 added a check in hive schematool -info command to see if schema > version is equal. > However, the state where db schema version is ahead of hive software version > is often seen when a 'rolling upgrade' or 'rolling downgrade' is happening. > This is a state where hive is functional and returning non zero status for it > is misleading. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-12261) schematool version info exit status should depend on compatibility, not equality
[ https://issues.apache.org/jira/browse/HIVE-12261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-12261: -- Labels: pull-request-available (was: ) > schematool version info exit status should depend on compatibility, not > equality > > > Key: HIVE-12261 > URL: https://issues.apache.org/jira/browse/HIVE-12261 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 1.3.0, 2.0.0 >Reporter: Thejas Nair >Assignee: Thejas Nair >Priority: Major > Labels: pull-request-available > Fix For: 1.3.0, 2.0.0 > > Attachments: HIVE-12261-branch-1.0.0.patch, > HIVE-12261-branch-1.patch, HIVE-12261.1.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Newer versions of metastore schema are compatible with older versions of > hive, as only new tables or columns are added with additional information. > HIVE-11613 added a check in hive schematool -info command to see if schema > version is equal. > However, the state where db schema version is ahead of hive software version > is often seen when a 'rolling upgrade' or 'rolling downgrade' is happening. > This is a state where hive is functional and returning non zero status for it > is misleading. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-12261) schematool version info exit status should depend on compatibility, not equality
[ https://issues.apache.org/jira/browse/HIVE-12261?focusedWorklogId=441548&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441548 ] ASF GitHub Bot logged work on HIVE-12261: - Author: ASF GitHub Bot Created on: 04/Jun/20 21:52 Start Date: 04/Jun/20 21:52 Worklog Time Spent: 10m Work Description: belugabehr commented on pull request #51: URL: https://github.com/apache/hive/pull/51#issuecomment-639138483 Resolved through manual patch process. https://issues.apache.org/jira/browse/HIVE-12261 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 441548) Remaining Estimate: 0h Time Spent: 10m > schematool version info exit status should depend on compatibility, not > equality > > > Key: HIVE-12261 > URL: https://issues.apache.org/jira/browse/HIVE-12261 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 1.3.0, 2.0.0 >Reporter: Thejas Nair >Assignee: Thejas Nair >Priority: Major > Fix For: 1.3.0, 2.0.0 > > Attachments: HIVE-12261-branch-1.0.0.patch, > HIVE-12261-branch-1.patch, HIVE-12261.1.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Newer versions of metastore schema are compatible with older versions of > hive, as only new tables or columns are added with additional information. > HIVE-11613 added a check in hive schematool -info command to see if schema > version is equal. > However, the state where db schema version is ahead of hive software version > is often seen when a 'rolling upgrade' or 'rolling downgrade' is happening. > This is a state where hive is functional and returning non zero status for it > is misleading. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-12943) Use default doesnot work in Hive 1.2.1
[ https://issues.apache.org/jira/browse/HIVE-12943?focusedWorklogId=441547&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441547 ] ASF GitHub Bot logged work on HIVE-12943: - Author: ASF GitHub Bot Created on: 04/Jun/20 21:48 Start Date: 04/Jun/20 21:48 Worklog Time Spent: 10m Work Description: belugabehr closed pull request #65: URL: https://github.com/apache/hive/pull/65 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 441547) Time Spent: 20m (was: 10m) > Use default doesnot work in Hive 1.2.1 > -- > > Key: HIVE-12943 > URL: https://issues.apache.org/jira/browse/HIVE-12943 > Project: Hive > Issue Type: Bug > Components: Database/Schema >Affects Versions: 1.2.1 >Reporter: Kashish Jain >Priority: Minor > Labels: pull-request-available > Attachments: HIVE-12943.patch > > Time Spent: 20m > Remaining Estimate: 0h > > "USE Default" does not work with the latest hive 1.2.1 > The message is > " >Cannot recognize input near 'default' '' '' in switch database > statement; line 1 pos 4 >NoViableAltException(81@[]) >at > org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.identifier(HiveParser_IdentifiersParser.java:11577) >at > org.apache.hadoop.hive.ql.parse.HiveParser.identifier(HiveParser.java:46055) > " -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-12943) Use default doesnot work in Hive 1.2.1
[ https://issues.apache.org/jira/browse/HIVE-12943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-12943: -- Labels: pull-request-available (was: ) > Use default doesnot work in Hive 1.2.1 > -- > > Key: HIVE-12943 > URL: https://issues.apache.org/jira/browse/HIVE-12943 > Project: Hive > Issue Type: Bug > Components: Database/Schema >Affects Versions: 1.2.1 >Reporter: Kashish Jain >Priority: Minor > Labels: pull-request-available > Attachments: HIVE-12943.patch > > Time Spent: 10m > Remaining Estimate: 0h > > "USE Default" does not work with the latest hive 1.2.1 > The message is > " >Cannot recognize input near 'default' '' '' in switch database > statement; line 1 pos 4 >NoViableAltException(81@[]) >at > org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.identifier(HiveParser_IdentifiersParser.java:11577) >at > org.apache.hadoop.hive.ql.parse.HiveParser.identifier(HiveParser.java:46055) > " -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-12943) Use default doesnot work in Hive 1.2.1
[ https://issues.apache.org/jira/browse/HIVE-12943?focusedWorklogId=441546&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441546 ] ASF GitHub Bot logged work on HIVE-12943: - Author: ASF GitHub Bot Created on: 04/Jun/20 21:48 Start Date: 04/Jun/20 21:48 Worklog Time Spent: 10m Work Description: belugabehr commented on pull request #65: URL: https://github.com/apache/hive/pull/65#issuecomment-639136821 If you would like to verify and continue working on this, please let us know. Feel free to reopen. https://issues.apache.org/jira/browse/HIVE-12943 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 441546) Remaining Estimate: 0h Time Spent: 10m > Use default doesnot work in Hive 1.2.1 > -- > > Key: HIVE-12943 > URL: https://issues.apache.org/jira/browse/HIVE-12943 > Project: Hive > Issue Type: Bug > Components: Database/Schema >Affects Versions: 1.2.1 >Reporter: Kashish Jain >Priority: Minor > Attachments: HIVE-12943.patch > > Time Spent: 10m > Remaining Estimate: 0h > > "USE Default" does not work with the latest hive 1.2.1 > The message is > " >Cannot recognize input near 'default' '' '' in switch database > statement; line 1 pos 4 >NoViableAltException(81@[]) >at > org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.identifier(HiveParser_IdentifiersParser.java:11577) >at > org.apache.hadoop.hive.ql.parse.HiveParser.identifier(HiveParser.java:46055) > " -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-12943) Use default doesnot work in Hive 1.2.1
[ https://issues.apache.org/jira/browse/HIVE-12943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-12943: -- Priority: Minor (was: Blocker) > Use default doesnot work in Hive 1.2.1 > -- > > Key: HIVE-12943 > URL: https://issues.apache.org/jira/browse/HIVE-12943 > Project: Hive > Issue Type: Bug > Components: Database/Schema >Affects Versions: 1.2.1 >Reporter: Kashish Jain >Priority: Minor > Attachments: HIVE-12943.patch > > > "USE Default" does not work with the latest hive 1.2.1 > The message is > " >Cannot recognize input near 'default' '' '' in switch database > statement; line 1 pos 4 >NoViableAltException(81@[]) >at > org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.identifier(HiveParser_IdentifiersParser.java:11577) >at > org.apache.hadoop.hive.ql.parse.HiveParser.identifier(HiveParser.java:46055) > " -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-12943) Use default doesnot work in Hive 1.2.1
[ https://issues.apache.org/jira/browse/HIVE-12943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-12943: -- Resolution: Won't Fix Status: Resolved (was: Patch Available) Please let us know if this is still an issue. Feel free to reopen. > Use default doesnot work in Hive 1.2.1 > -- > > Key: HIVE-12943 > URL: https://issues.apache.org/jira/browse/HIVE-12943 > Project: Hive > Issue Type: Bug > Components: Database/Schema >Affects Versions: 1.2.1 >Reporter: Kashish Jain >Priority: Minor > Attachments: HIVE-12943.patch > > > "USE Default" does not work with the latest hive 1.2.1 > The message is > " >Cannot recognize input near 'default' '' '' in switch database > statement; line 1 pos 4 >NoViableAltException(81@[]) >at > org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.identifier(HiveParser_IdentifiersParser.java:11577) >at > org.apache.hadoop.hive.ql.parse.HiveParser.identifier(HiveParser.java:46055) > " -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-12002) correct implementation typo
[ https://issues.apache.org/jira/browse/HIVE-12002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-12002: -- Labels: newbie pull-request-available typo (was: newbie typo) > correct implementation typo > --- > > Key: HIVE-12002 > URL: https://issues.apache.org/jira/browse/HIVE-12002 > Project: Hive > Issue Type: Improvement > Components: Beeline, HCatalog, Metastore >Affects Versions: 1.2.1 >Reporter: Alex Moundalexis >Assignee: Alex Moundalexis >Priority: Trivial > Labels: newbie, pull-request-available, typo > Fix For: 2.0.0 > > Attachments: HIVE-12002.patch > > Time Spent: 10m > Remaining Estimate: 0h > > The term "implemenation" is seen in HiveMetaScore INFO logs. Correcting. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-12002) correct implementation typo
[ https://issues.apache.org/jira/browse/HIVE-12002?focusedWorklogId=441545&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441545 ] ASF GitHub Bot logged work on HIVE-12002: - Author: ASF GitHub Bot Created on: 04/Jun/20 21:44 Start Date: 04/Jun/20 21:44 Worklog Time Spent: 10m Work Description: belugabehr commented on pull request #50: URL: https://github.com/apache/hive/pull/50#issuecomment-639135390 Issue fixed through manual patch process: https://issues.apache.org/jira/browse/HIVE-12002 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 441545) Time Spent: 20m (was: 10m) > correct implementation typo > --- > > Key: HIVE-12002 > URL: https://issues.apache.org/jira/browse/HIVE-12002 > Project: Hive > Issue Type: Improvement > Components: Beeline, HCatalog, Metastore >Affects Versions: 1.2.1 >Reporter: Alex Moundalexis >Assignee: Alex Moundalexis >Priority: Trivial > Labels: newbie, pull-request-available, typo > Fix For: 2.0.0 > > Attachments: HIVE-12002.patch > > Time Spent: 20m > Remaining Estimate: 0h > > The term "implemenation" is seen in HiveMetaScore INFO logs. Correcting. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-12002) correct implementation typo
[ https://issues.apache.org/jira/browse/HIVE-12002?focusedWorklogId=441544&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441544 ] ASF GitHub Bot logged work on HIVE-12002: - Author: ASF GitHub Bot Created on: 04/Jun/20 21:44 Start Date: 04/Jun/20 21:44 Worklog Time Spent: 10m Work Description: belugabehr closed pull request #50: URL: https://github.com/apache/hive/pull/50 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 441544) Remaining Estimate: 0h Time Spent: 10m > correct implementation typo > --- > > Key: HIVE-12002 > URL: https://issues.apache.org/jira/browse/HIVE-12002 > Project: Hive > Issue Type: Improvement > Components: Beeline, HCatalog, Metastore >Affects Versions: 1.2.1 >Reporter: Alex Moundalexis >Assignee: Alex Moundalexis >Priority: Trivial > Labels: newbie, typo > Fix For: 2.0.0 > > Attachments: HIVE-12002.patch > > Time Spent: 10m > Remaining Estimate: 0h > > The term "implemenation" is seen in HiveMetaScore INFO logs. Correcting. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-4331) Integrated StorageHandler for Hive and HCat using the HiveStorageHandler
[ https://issues.apache.org/jira/browse/HIVE-4331?focusedWorklogId=441542&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441542 ] ASF GitHub Bot logged work on HIVE-4331: Author: ASF GitHub Bot Created on: 04/Jun/20 21:37 Start Date: 04/Jun/20 21:37 Worklog Time Spent: 10m Work Description: belugabehr closed pull request #8: URL: https://github.com/apache/hive/pull/8 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 441542) Remaining Estimate: 0h Time Spent: 10m > Integrated StorageHandler for Hive and HCat using the HiveStorageHandler > > > Key: HIVE-4331 > URL: https://issues.apache.org/jira/browse/HIVE-4331 > Project: Hive > Issue Type: Task > Components: HBase Handler, HCatalog >Affects Versions: 0.12.0 >Reporter: Ashutosh Chauhan >Assignee: Viraj Bhat >Priority: Major > Fix For: 0.12.0 > > Attachments: HIVE-4331.1.patch, HIVE-4331.2.patch, HIVE-4331.patch, > HIVE4331_07-17.patch, StorageHandlerDesign_HIVE4331.pdf, > hive4331hcatrebase.patch > > Time Spent: 10m > Remaining Estimate: 0h > > 1) Deprecate the HCatHBaseStorageHandler and "RevisionManager" from HCatalog. > These will now continue to function but internally they will use the > "DefaultStorageHandler" from Hive. They will be removed in future release of > Hive. > 2) Design a HivePassThroughFormat so that any new StorageHandler in Hive will > bypass the HiveOutputFormat. We will use this class in Hive's > "HBaseStorageHandler" instead of the "HiveHBaseTableOutputFormat". > 3) Write new unit tests in the HCat's "storagehandler" so that systems such > as Pig and Map Reduce can use the Hive's "HBaseStorageHandler" instead of the > "HCatHBaseStorageHandler". > 4) Make sure all the old and new unit tests pass without backward > compatibility (except known issues as described in the Design Document). > 5) Replace all instances of the HCat source code, which point to > "HCatStorageHandler" to use the"HiveStorageHandler" including the > "FosterStorageHandler". > I have attached the design document for the same and will attach a patch to > this Jira. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-4331) Integrated StorageHandler for Hive and HCat using the HiveStorageHandler
[ https://issues.apache.org/jira/browse/HIVE-4331?focusedWorklogId=441543&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441543 ] ASF GitHub Bot logged work on HIVE-4331: Author: ASF GitHub Bot Created on: 04/Jun/20 21:37 Start Date: 04/Jun/20 21:37 Worklog Time Spent: 10m Work Description: belugabehr commented on pull request #8: URL: https://github.com/apache/hive/pull/8#issuecomment-639132694 Issues resolve through manual patch process: https://issues.apache.org/jira/browse/HIVE-4331 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 441543) Time Spent: 20m (was: 10m) > Integrated StorageHandler for Hive and HCat using the HiveStorageHandler > > > Key: HIVE-4331 > URL: https://issues.apache.org/jira/browse/HIVE-4331 > Project: Hive > Issue Type: Task > Components: HBase Handler, HCatalog >Affects Versions: 0.12.0 >Reporter: Ashutosh Chauhan >Assignee: Viraj Bhat >Priority: Major > Labels: pull-request-available > Fix For: 0.12.0 > > Attachments: HIVE-4331.1.patch, HIVE-4331.2.patch, HIVE-4331.patch, > HIVE4331_07-17.patch, StorageHandlerDesign_HIVE4331.pdf, > hive4331hcatrebase.patch > > Time Spent: 20m > Remaining Estimate: 0h > > 1) Deprecate the HCatHBaseStorageHandler and "RevisionManager" from HCatalog. > These will now continue to function but internally they will use the > "DefaultStorageHandler" from Hive. They will be removed in future release of > Hive. > 2) Design a HivePassThroughFormat so that any new StorageHandler in Hive will > bypass the HiveOutputFormat. We will use this class in Hive's > "HBaseStorageHandler" instead of the "HiveHBaseTableOutputFormat". > 3) Write new unit tests in the HCat's "storagehandler" so that systems such > as Pig and Map Reduce can use the Hive's "HBaseStorageHandler" instead of the > "HCatHBaseStorageHandler". > 4) Make sure all the old and new unit tests pass without backward > compatibility (except known issues as described in the Design Document). > 5) Replace all instances of the HCat source code, which point to > "HCatStorageHandler" to use the"HiveStorageHandler" including the > "FosterStorageHandler". > I have attached the design document for the same and will attach a patch to > this Jira. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-4331) Integrated StorageHandler for Hive and HCat using the HiveStorageHandler
[ https://issues.apache.org/jira/browse/HIVE-4331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-4331: - Labels: pull-request-available (was: ) > Integrated StorageHandler for Hive and HCat using the HiveStorageHandler > > > Key: HIVE-4331 > URL: https://issues.apache.org/jira/browse/HIVE-4331 > Project: Hive > Issue Type: Task > Components: HBase Handler, HCatalog >Affects Versions: 0.12.0 >Reporter: Ashutosh Chauhan >Assignee: Viraj Bhat >Priority: Major > Labels: pull-request-available > Fix For: 0.12.0 > > Attachments: HIVE-4331.1.patch, HIVE-4331.2.patch, HIVE-4331.patch, > HIVE4331_07-17.patch, StorageHandlerDesign_HIVE4331.pdf, > hive4331hcatrebase.patch > > Time Spent: 10m > Remaining Estimate: 0h > > 1) Deprecate the HCatHBaseStorageHandler and "RevisionManager" from HCatalog. > These will now continue to function but internally they will use the > "DefaultStorageHandler" from Hive. They will be removed in future release of > Hive. > 2) Design a HivePassThroughFormat so that any new StorageHandler in Hive will > bypass the HiveOutputFormat. We will use this class in Hive's > "HBaseStorageHandler" instead of the "HiveHBaseTableOutputFormat". > 3) Write new unit tests in the HCat's "storagehandler" so that systems such > as Pig and Map Reduce can use the Hive's "HBaseStorageHandler" instead of the > "HCatHBaseStorageHandler". > 4) Make sure all the old and new unit tests pass without backward > compatibility (except known issues as described in the Design Document). > 5) Replace all instances of the HCat source code, which point to > "HCatStorageHandler" to use the"HiveStorageHandler" including the > "FosterStorageHandler". > I have attached the design document for the same and will attach a patch to > this Jira. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-13491) Testing : log thread stacks when metastore fails to start
[
https://issues.apache.org/jira/browse/HIVE-13491?focusedWorklogId=441537&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441537
]
ASF GitHub Bot logged work on HIVE-13491:
-
Author: ASF GitHub Bot
Created on: 04/Jun/20 21:36
Start Date: 04/Jun/20 21:36
Worklog Time Spent: 10m
Work Description: belugabehr commented on pull request #71:
URL: https://github.com/apache/hive/pull/71#issuecomment-639132170
Issues resolved through manual patch process:
https://issues.apache.org/jira/browse/HIVE-13491
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 441537)
Remaining Estimate: 0h
Time Spent: 10m
> Testing : log thread stacks when metastore fails to start
> --
>
> Key: HIVE-13491
> URL: https://issues.apache.org/jira/browse/HIVE-13491
> Project: Hive
> Issue Type: Bug
> Components: Test, Testing Infrastructure
>Reporter: Thejas Nair
>Assignee: Thejas Nair
>Priority: Major
> Fix For: 1.3.0, 2.1.0
>
> Attachments: HIVE-13491.1.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Many tests are failing in ptest2 because metastore fails to startup in the
> expected time.
> There is not enough information to figure out why the metastore startup
> failed/got hung in the hive.log file. Printing the thread dumps when that
> happens would be useful for finding the root cause.
> The stack in test failure looks like this -
> {code}
> java.net.ConnectException: Connection refused
> at java.net.PlainSocketImpl.socketConnect(Native Method)
> at
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
> at
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:198)
> at
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
> at java.net.Socket.connect(Socket.java:579)
> at
> org.apache.hadoop.hive.metastore.MetaStoreUtils.loopUntilHMSReady(MetaStoreUtils.java:1208)
> at
> org.apache.hadoop.hive.metastore.MetaStoreUtils.startMetaStore(MetaStoreUtils.java:1195)
> at
> org.apache.hadoop.hive.metastore.MetaStoreUtils.startMetaStore(MetaStoreUtils.java:1177)
> at
> org.apache.hadoop.hive.thrift.TestHadoopAuthBridge23.setup(TestHadoopAuthBridge23.java:153)
> at
> org.apache.hadoop.hive.thrift.TestHadoopAuthBridge23.testMetastoreProxyUser(TestHadoopAuthBridge23.java:241)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-13491) Testing : log thread stacks when metastore fails to start
[
https://issues.apache.org/jira/browse/HIVE-13491?focusedWorklogId=441538&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441538
]
ASF GitHub Bot logged work on HIVE-13491:
-
Author: ASF GitHub Bot
Created on: 04/Jun/20 21:36
Start Date: 04/Jun/20 21:36
Worklog Time Spent: 10m
Work Description: belugabehr closed pull request #71:
URL: https://github.com/apache/hive/pull/71
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 441538)
Time Spent: 20m (was: 10m)
> Testing : log thread stacks when metastore fails to start
> --
>
> Key: HIVE-13491
> URL: https://issues.apache.org/jira/browse/HIVE-13491
> Project: Hive
> Issue Type: Bug
> Components: Test, Testing Infrastructure
>Reporter: Thejas Nair
>Assignee: Thejas Nair
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.3.0, 2.1.0
>
> Attachments: HIVE-13491.1.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Many tests are failing in ptest2 because metastore fails to startup in the
> expected time.
> There is not enough information to figure out why the metastore startup
> failed/got hung in the hive.log file. Printing the thread dumps when that
> happens would be useful for finding the root cause.
> The stack in test failure looks like this -
> {code}
> java.net.ConnectException: Connection refused
> at java.net.PlainSocketImpl.socketConnect(Native Method)
> at
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
> at
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:198)
> at
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
> at java.net.Socket.connect(Socket.java:579)
> at
> org.apache.hadoop.hive.metastore.MetaStoreUtils.loopUntilHMSReady(MetaStoreUtils.java:1208)
> at
> org.apache.hadoop.hive.metastore.MetaStoreUtils.startMetaStore(MetaStoreUtils.java:1195)
> at
> org.apache.hadoop.hive.metastore.MetaStoreUtils.startMetaStore(MetaStoreUtils.java:1177)
> at
> org.apache.hadoop.hive.thrift.TestHadoopAuthBridge23.setup(TestHadoopAuthBridge23.java:153)
> at
> org.apache.hadoop.hive.thrift.TestHadoopAuthBridge23.testMetastoreProxyUser(TestHadoopAuthBridge23.java:241)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-13491) Testing : log thread stacks when metastore fails to start
[
https://issues.apache.org/jira/browse/HIVE-13491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-13491:
--
Labels: pull-request-available (was: )
> Testing : log thread stacks when metastore fails to start
> --
>
> Key: HIVE-13491
> URL: https://issues.apache.org/jira/browse/HIVE-13491
> Project: Hive
> Issue Type: Bug
> Components: Test, Testing Infrastructure
>Reporter: Thejas Nair
>Assignee: Thejas Nair
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.3.0, 2.1.0
>
> Attachments: HIVE-13491.1.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Many tests are failing in ptest2 because metastore fails to startup in the
> expected time.
> There is not enough information to figure out why the metastore startup
> failed/got hung in the hive.log file. Printing the thread dumps when that
> happens would be useful for finding the root cause.
> The stack in test failure looks like this -
> {code}
> java.net.ConnectException: Connection refused
> at java.net.PlainSocketImpl.socketConnect(Native Method)
> at
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
> at
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:198)
> at
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
> at java.net.Socket.connect(Socket.java:579)
> at
> org.apache.hadoop.hive.metastore.MetaStoreUtils.loopUntilHMSReady(MetaStoreUtils.java:1208)
> at
> org.apache.hadoop.hive.metastore.MetaStoreUtils.startMetaStore(MetaStoreUtils.java:1195)
> at
> org.apache.hadoop.hive.metastore.MetaStoreUtils.startMetaStore(MetaStoreUtils.java:1177)
> at
> org.apache.hadoop.hive.thrift.TestHadoopAuthBridge23.setup(TestHadoopAuthBridge23.java:153)
> at
> org.apache.hadoop.hive.thrift.TestHadoopAuthBridge23.testMetastoreProxyUser(TestHadoopAuthBridge23.java:241)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-13418) HiveServer2 HTTP mode should support X-Forwarded-Host header for authorization/audits
[ https://issues.apache.org/jira/browse/HIVE-13418?focusedWorklogId=441536&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441536 ] ASF GitHub Bot logged work on HIVE-13418: - Author: ASF GitHub Bot Created on: 04/Jun/20 21:35 Start Date: 04/Jun/20 21:35 Worklog Time Spent: 10m Work Description: belugabehr closed pull request #69: URL: https://github.com/apache/hive/pull/69 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 441536) Time Spent: 20m (was: 10m) > HiveServer2 HTTP mode should support X-Forwarded-Host header for > authorization/audits > - > > Key: HIVE-13418 > URL: https://issues.apache.org/jira/browse/HIVE-13418 > Project: Hive > Issue Type: New Feature > Components: Authorization, HiveServer2 >Reporter: Thejas Nair >Assignee: Thejas Nair >Priority: Major > Labels: pull-request-available > Fix For: 2.1.0 > > Attachments: HIVE-13418.1.patch > > Time Spent: 20m > Remaining Estimate: 0h > > Apache Knox acts as a proxy for requests coming from the end users. In these > cases, the IP address that HiveServer2 passes to the authorization/audit > plugins via the HiveAuthzContext object only the IP address of the proxy, and > not the end user. > For auditing purposes, the IP address of the end user and any proxies in > between are useful. > HiveServer2 should pass the information from 'X-Forwarded-Host' header to > the HiveAuthorizer plugins. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-13418) HiveServer2 HTTP mode should support X-Forwarded-Host header for authorization/audits
[ https://issues.apache.org/jira/browse/HIVE-13418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-13418: -- Labels: pull-request-available (was: ) > HiveServer2 HTTP mode should support X-Forwarded-Host header for > authorization/audits > - > > Key: HIVE-13418 > URL: https://issues.apache.org/jira/browse/HIVE-13418 > Project: Hive > Issue Type: New Feature > Components: Authorization, HiveServer2 >Reporter: Thejas Nair >Assignee: Thejas Nair >Priority: Major > Labels: pull-request-available > Fix For: 2.1.0 > > Attachments: HIVE-13418.1.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Apache Knox acts as a proxy for requests coming from the end users. In these > cases, the IP address that HiveServer2 passes to the authorization/audit > plugins via the HiveAuthzContext object only the IP address of the proxy, and > not the end user. > For auditing purposes, the IP address of the end user and any proxies in > between are useful. > HiveServer2 should pass the information from 'X-Forwarded-Host' header to > the HiveAuthorizer plugins. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-12660) HS2 memory leak with .hiverc file use
[ https://issues.apache.org/jira/browse/HIVE-12660?focusedWorklogId=441533&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441533 ] ASF GitHub Bot logged work on HIVE-12660: - Author: ASF GitHub Bot Created on: 04/Jun/20 21:34 Start Date: 04/Jun/20 21:34 Worklog Time Spent: 10m Work Description: belugabehr commented on pull request #64: URL: https://github.com/apache/hive/pull/64#issuecomment-639131405 This has already been merged through the manual patch process. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 441533) Remaining Estimate: 0h Time Spent: 10m > HS2 memory leak with .hiverc file use > - > > Key: HIVE-12660 > URL: https://issues.apache.org/jira/browse/HIVE-12660 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Affects Versions: 0.14.0, 1.0.0, 1.2.0, 1.1.0, 1.2.1 >Reporter: Thejas Nair >Assignee: Thejas Nair >Priority: Major > Fix For: 2.0.0 > > Attachments: HIVE-12660-branch-1.addendum.patch, HIVE-12660.1.patch > > Time Spent: 10m > Remaining Estimate: 0h > > The Operation objects created to process .hiverc file in HS2 are not closed. > In HiveSessionImpl, GlobalHivercFileProcessor calls executeStatementInternal > but ignores the OperationHandle it returns. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-13418) HiveServer2 HTTP mode should support X-Forwarded-Host header for authorization/audits
[ https://issues.apache.org/jira/browse/HIVE-13418?focusedWorklogId=441535&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441535 ] ASF GitHub Bot logged work on HIVE-13418: - Author: ASF GitHub Bot Created on: 04/Jun/20 21:34 Start Date: 04/Jun/20 21:34 Worklog Time Spent: 10m Work Description: belugabehr commented on pull request #69: URL: https://github.com/apache/hive/pull/69#issuecomment-639131702 This has already been resolved through the manual patch process. https://issues.apache.org/jira/browse/HIVE-13418 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 441535) Remaining Estimate: 0h Time Spent: 10m > HiveServer2 HTTP mode should support X-Forwarded-Host header for > authorization/audits > - > > Key: HIVE-13418 > URL: https://issues.apache.org/jira/browse/HIVE-13418 > Project: Hive > Issue Type: New Feature > Components: Authorization, HiveServer2 >Reporter: Thejas Nair >Assignee: Thejas Nair >Priority: Major > Fix For: 2.1.0 > > Attachments: HIVE-13418.1.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Apache Knox acts as a proxy for requests coming from the end users. In these > cases, the IP address that HiveServer2 passes to the authorization/audit > plugins via the HiveAuthzContext object only the IP address of the proxy, and > not the end user. > For auditing purposes, the IP address of the end user and any proxies in > between are useful. > HiveServer2 should pass the information from 'X-Forwarded-Host' header to > the HiveAuthorizer plugins. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-12660) HS2 memory leak with .hiverc file use
[ https://issues.apache.org/jira/browse/HIVE-12660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-12660: -- Labels: pull-request-available (was: ) > HS2 memory leak with .hiverc file use > - > > Key: HIVE-12660 > URL: https://issues.apache.org/jira/browse/HIVE-12660 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Affects Versions: 0.14.0, 1.0.0, 1.2.0, 1.1.0, 1.2.1 >Reporter: Thejas Nair >Assignee: Thejas Nair >Priority: Major > Labels: pull-request-available > Fix For: 2.0.0 > > Attachments: HIVE-12660-branch-1.addendum.patch, HIVE-12660.1.patch > > Time Spent: 10m > Remaining Estimate: 0h > > The Operation objects created to process .hiverc file in HS2 are not closed. > In HiveSessionImpl, GlobalHivercFileProcessor calls executeStatementInternal > but ignores the OperationHandle it returns. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-12660) HS2 memory leak with .hiverc file use
[ https://issues.apache.org/jira/browse/HIVE-12660?focusedWorklogId=441534&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441534 ] ASF GitHub Bot logged work on HIVE-12660: - Author: ASF GitHub Bot Created on: 04/Jun/20 21:34 Start Date: 04/Jun/20 21:34 Worklog Time Spent: 10m Work Description: belugabehr closed pull request #64: URL: https://github.com/apache/hive/pull/64 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 441534) Time Spent: 20m (was: 10m) > HS2 memory leak with .hiverc file use > - > > Key: HIVE-12660 > URL: https://issues.apache.org/jira/browse/HIVE-12660 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Affects Versions: 0.14.0, 1.0.0, 1.2.0, 1.1.0, 1.2.1 >Reporter: Thejas Nair >Assignee: Thejas Nair >Priority: Major > Labels: pull-request-available > Fix For: 2.0.0 > > Attachments: HIVE-12660-branch-1.addendum.patch, HIVE-12660.1.patch > > Time Spent: 20m > Remaining Estimate: 0h > > The Operation objects created to process .hiverc file in HS2 are not closed. > In HiveSessionImpl, GlobalHivercFileProcessor calls executeStatementInternal > but ignores the OperationHandle it returns. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-20447) Add JSON Outputformat support
[
https://issues.apache.org/jira/browse/HIVE-20447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17126196#comment-17126196
]
Hive QA commented on HIVE-20447:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12943658/HIVE-20447.01.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 17104 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.metastore.txn.TestTxnHandler.testReplTimeouts
(batchId=245)
org.apache.hive.jdbc.TestServiceDiscoveryWithMiniHS2.testGetAllUrlsDirect
(batchId=220)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22749/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22749/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22749/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12943658 - PreCommit-HIVE-Build
> Add JSON Outputformat support
> -
>
> Key: HIVE-20447
> URL: https://issues.apache.org/jira/browse/HIVE-20447
> Project: Hive
> Issue Type: Task
> Components: Beeline
>Reporter: Max Efremov
>Assignee: Max Efremov
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-20447.01.patch
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> This function is present in SQLLine. We need add it to beeline too.
> https://github.com/julianhyde/sqlline/pull/84
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23614) Always pass HiveConfig to removeTempOrDuplicateFiles
[ https://issues.apache.org/jira/browse/HIVE-23614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Sherman updated HIVE-23614: Description: As part of HIVE-23354, we check the provided HiveConf for speculative execution and throw an error if it is enabled. There is one path that did not previously provide a HiveConf value which shows up in test failures in runtime_skewjoin_mapjoin_spark.q, skewjoin.q and skewjoin_onesideskew.q (which we do not run as part of pre-commit tests) or at least not through TestCliDriver. (was: As part of HIVE-23354, we check the provided HiveConf for speculative execution and throw an error if it is enabled. There is one path that did not previously provide a HiveConf value which shows up in test failures in runtime_skewjoin_mapjoin_spark.q, skewjoin.q and skewjoin_onesideskew.q (which we do not run as part of pre-commit tests).) > Always pass HiveConfig to removeTempOrDuplicateFiles > > > Key: HIVE-23614 > URL: https://issues.apache.org/jira/browse/HIVE-23614 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: John Sherman >Assignee: John Sherman >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > As part of HIVE-23354, we check the provided HiveConf for speculative > execution and throw an error if it is enabled. There is one path that did not > previously provide a HiveConf value which shows up in test failures in > runtime_skewjoin_mapjoin_spark.q, skewjoin.q and skewjoin_onesideskew.q > (which we do not run as part of pre-commit tests) or at least not through > TestCliDriver. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23614) Always pass HiveConfig to removeTempOrDuplicateFiles
[ https://issues.apache.org/jira/browse/HIVE-23614?focusedWorklogId=441526&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441526 ] ASF GitHub Bot logged work on HIVE-23614: - Author: ASF GitHub Bot Created on: 04/Jun/20 21:17 Start Date: 04/Jun/20 21:17 Worklog Time Spent: 10m Work Description: jfsii opened a new pull request #1059: URL: https://github.com/apache/hive/pull/1059 Change-Id: I3b734e0ab90e69e9a90452001b9171022d2e89eb This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 441526) Remaining Estimate: 0h Time Spent: 10m > Always pass HiveConfig to removeTempOrDuplicateFiles > > > Key: HIVE-23614 > URL: https://issues.apache.org/jira/browse/HIVE-23614 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: John Sherman >Assignee: John Sherman >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > As part of HIVE-23354, we check the provided HiveConf for speculative > execution and throw an error if it is enabled. There is one path that did not > previously provide a HiveConf value which shows up in test failures in > runtime_skewjoin_mapjoin_spark.q, skewjoin.q and skewjoin_onesideskew.q > (which we do not run as part of pre-commit tests). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23614) Always pass HiveConfig to removeTempOrDuplicateFiles
[ https://issues.apache.org/jira/browse/HIVE-23614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-23614: -- Labels: pull-request-available (was: ) > Always pass HiveConfig to removeTempOrDuplicateFiles > > > Key: HIVE-23614 > URL: https://issues.apache.org/jira/browse/HIVE-23614 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: John Sherman >Assignee: John Sherman >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > As part of HIVE-23354, we check the provided HiveConf for speculative > execution and throw an error if it is enabled. There is one path that did not > previously provide a HiveConf value which shows up in test failures in > runtime_skewjoin_mapjoin_spark.q, skewjoin.q and skewjoin_onesideskew.q > (which we do not run as part of pre-commit tests). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-23614) Always pass HiveConfig to removeTempOrDuplicateFiles
[ https://issues.apache.org/jira/browse/HIVE-23614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Sherman reassigned HIVE-23614: --- > Always pass HiveConfig to removeTempOrDuplicateFiles > > > Key: HIVE-23614 > URL: https://issues.apache.org/jira/browse/HIVE-23614 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: John Sherman >Assignee: John Sherman >Priority: Major > > As part of HIVE-23354, we check the provided HiveConf for speculative > execution and throw an error if it is enabled. There is one path that did not > previously provide a HiveConf value which shows up in test failures in > runtime_skewjoin_mapjoin_spark.q, skewjoin.q and skewjoin_onesideskew.q > (which we do not run as part of pre-commit tests). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-20447) Add JSON Outputformat support
[
https://issues.apache.org/jira/browse/HIVE-20447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17126173#comment-17126173
]
Hive QA commented on HIVE-20447:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
29s{color} | {color:blue} beeline in master has 48 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
13s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
11s{color} | {color:red} beeline: The patch generated 8 new + 51 unchanged - 0
fixed = 59 total (was 51) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 3 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
38s{color} | {color:red} beeline generated 1 new + 48 unchanged - 0 fixed = 49
total (was 48) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 11m 36s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:beeline |
| | Load of known null value in
org.apache.hive.beeline.JsonOutputFormat.setJsonValue(StringBuilder, String,
int) At JsonOutputFormat.java:in
org.apache.hive.beeline.JsonOutputFormat.setJsonValue(StringBuilder, String,
int) At JsonOutputFormat.java:[line 119] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22749/dev-support/hive-personality.sh
|
| git revision | master / b5e3bcd |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22749/yetus/diff-checkstyle-beeline.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22749/yetus/whitespace-eol.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22749/yetus/new-findbugs-beeline.html
|
| modules | C: beeline U: beeline |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22749/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Add JSON Outputformat support
> -
>
> Key: HIVE-20447
> URL: https://issues.apache.org/jira/browse/HIVE-20447
> Project: Hive
> Issue Type: Task
> Components: Beeline
>Reporter: Max Efremov
>Assignee: Max Efremov
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-20447.01.patch
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> This function is present in SQLLine. We need add it to beeline too.
> https://github.com/julianhyde/sqlline/pull/84
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-20447) Add JSON Outputformat support
[ https://issues.apache.org/jira/browse/HIVE-20447?focusedWorklogId=441497&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441497 ] ASF GitHub Bot logged work on HIVE-20447: - Author: ASF GitHub Bot Created on: 04/Jun/20 19:55 Start Date: 04/Jun/20 19:55 Worklog Time Spent: 10m Work Description: belugabehr commented on pull request #421: URL: https://github.com/apache/hive/pull/421#issuecomment-639081834 Perhaps we just need 2 versions of this output format, one with a no-op `printHeader()` and `printFooter()` method and one that overrides that with the relevant text (and also overrides to add a comma after each element) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 441497) Time Spent: 0.5h (was: 20m) > Add JSON Outputformat support > - > > Key: HIVE-20447 > URL: https://issues.apache.org/jira/browse/HIVE-20447 > Project: Hive > Issue Type: Task > Components: Beeline >Reporter: Max Efremov >Assignee: Max Efremov >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20447.01.patch > > Time Spent: 0.5h > Remaining Estimate: 0h > > This function is present in SQLLine. We need add it to beeline too. > https://github.com/julianhyde/sqlline/pull/84 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-20447) Add JSON Outputformat support
[
https://issues.apache.org/jira/browse/HIVE-20447?focusedWorklogId=441496&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441496
]
ASF GitHub Bot logged work on HIVE-20447:
-
Author: ASF GitHub Bot
Created on: 04/Jun/20 19:52
Start Date: 04/Jun/20 19:52
Worklog Time Spent: 10m
Work Description: belugabehr commented on pull request #421:
URL: https://github.com/apache/hive/pull/421#issuecomment-639080487
Thanks for the contribution.
By default, the JSON SerDe in Hive does not handle "Standard" JSON. Each
JSON object is on it's own line:
```
{"name":"Joe","age":24}
{"name":"Mary","age":28}
...
```
It would be great to have some symmetry here. Any ideas on how to enable
such a behavior? It can be added in a follow-up JIRA, just curious if you have
any thoughts.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 441496)
Time Spent: 20m (was: 10m)
> Add JSON Outputformat support
> -
>
> Key: HIVE-20447
> URL: https://issues.apache.org/jira/browse/HIVE-20447
> Project: Hive
> Issue Type: Task
> Components: Beeline
>Reporter: Max Efremov
>Assignee: Max Efremov
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-20447.01.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> This function is present in SQLLine. We need add it to beeline too.
> https://github.com/julianhyde/sqlline/pull/84
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-20447) Add JSON Outputformat support
[
https://issues.apache.org/jira/browse/HIVE-20447?focusedWorklogId=441495&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441495
]
ASF GitHub Bot logged work on HIVE-20447:
-
Author: ASF GitHub Bot
Created on: 04/Jun/20 19:50
Start Date: 04/Jun/20 19:50
Worklog Time Spent: 10m
Work Description: belugabehr commented on a change in pull request #421:
URL: https://github.com/apache/hive/pull/421#discussion_r435505118
##
File path: beeline/src/java/org/apache/hive/beeline/JsonOutputFormat.java
##
@@ -0,0 +1,163 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+/*
+ * This source file is based on code taken from SQLLine 1.4.0
+ * See SQLLine notice in LICENSE
+ */
+package org.apache.hive.beeline;
+
+import java.sql.SQLException;
+import java.sql.Types;
+import java.util.HashMap;
+import java.util.Map;
+
+/**
+ * Implementation of {@link OutputFormat} that formats rows as JSON.
+ */
+public class JsonOutputFormat extends AbstractOutputFormat {
+ private static final Map ESCAPING_MAP = new
HashMap();
+
+ static {
+ESCAPING_MAP.put('\\', "");
+ESCAPING_MAP.put('\"', "\\\"");
+ESCAPING_MAP.put('\b', "\\b");
+ESCAPING_MAP.put('\f', "\\f");
+ESCAPING_MAP.put('\n', "\\n");
+ESCAPING_MAP.put('\r', "\\r");
+ESCAPING_MAP.put('\t', "\\t");
+ESCAPING_MAP.put('/', "\\/");
+ESCAPING_MAP.put('\u', "\\u");
+ESCAPING_MAP.put('\u0001', "\\u0001");
+ESCAPING_MAP.put('\u0002', "\\u0002");
+ESCAPING_MAP.put('\u0003', "\\u0003");
+ESCAPING_MAP.put('\u0004', "\\u0004");
+ESCAPING_MAP.put('\u0005', "\\u0005");
+ESCAPING_MAP.put('\u0006', "\\u0006");
+ESCAPING_MAP.put('\u0007', "\\u0007");
+// ESCAPING_MAP.put('\u0008', "\\u0008");
+// covered by ESCAPING_MAP.put('\b', "\\b");
+// ESCAPING_MAP.put('\u0009', "\\u0009");
+// covered by ESCAPING_MAP.put('\t', "\\t");
+// ESCAPING_MAP.put((char) 10, "\\u000A");
+// covered by ESCAPING_MAP.put('\n', "\\n");
+ESCAPING_MAP.put('\u000B', "\\u000B");
+// ESCAPING_MAP.put('\u000C', "\\u000C");
+// covered by ESCAPING_MAP.put('\f', "\\f");
+// ESCAPING_MAP.put((char) 13, "\\u000D");
+// covered by ESCAPING_MAP.put('\r', "\\r");
+ESCAPING_MAP.put('\u000E', "\\u000E");
+ESCAPING_MAP.put('\u000F', "\\u000F");
+ESCAPING_MAP.put('\u0010', "\\u0010");
+ESCAPING_MAP.put('\u0011', "\\u0011");
+ESCAPING_MAP.put('\u0012', "\\u0012");
+ESCAPING_MAP.put('\u0013', "\\u0013");
+ESCAPING_MAP.put('\u0014', "\\u0014");
+ESCAPING_MAP.put('\u0015', "\\u0015");
+ESCAPING_MAP.put('\u0016', "\\u0016");
+ESCAPING_MAP.put('\u0017', "\\u0017");
+ESCAPING_MAP.put('\u0018', "\\u0018");
+ESCAPING_MAP.put('\u0019', "\\u0019");
+ESCAPING_MAP.put('\u001A', "\\u001A");
+ESCAPING_MAP.put('\u001B', "\\u001B");
+ESCAPING_MAP.put('\u001C', "\\u001C");
+ESCAPING_MAP.put('\u001D', "\\u001D");
+ESCAPING_MAP.put('\u001E', "\\u001E");
+ESCAPING_MAP.put('\u001F', "\\u001F");
+ }
+
+ private int[] columnTypes;
+
+ public JsonOutputFormat(BeeLine beeLine) {
+super(beeLine);
+ }
+
+ @Override
+ void printHeader(Rows.Row header) {
+beeLine.output("{\"resultset\":[");
+ }
+
+ @Override
+ void printFooter(Rows.Row header) {
+beeLine.output("]}");
+ }
+
+ @Override
+ void printRow(Rows rows, Rows.Row header, Rows.Row row) {
+String[] head = header.values;
+String[] vals = row.values;
+StringBuilder sb = new StringBuilder("{");
+for (int i = 0; (i < head.length) && (i < vals.length); i++) {
Review comment:
Can this be something like...
```
final int columnCount = Math.min(head.length, vals.length);
for (int i = 0; i < columnCount; i++) {
...
```
##
File path: beeline/src/java/org/apache/hive/beeline/JsonOutputFormat.java
##
@@ -0,0 +1,163 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright owne
[jira] [Commented] (HIVE-23590) Close stale PRs automatically
[ https://issues.apache.org/jira/browse/HIVE-23590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17126159#comment-17126159 ] Jesus Camacho Rodriguez commented on HIVE-23590: +1 > Close stale PRs automatically > - > > Key: HIVE-23590 > URL: https://issues.apache.org/jira/browse/HIVE-23590 > Project: Hive > Issue Type: Sub-task >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Labels: pull-request-available > Time Spent: 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23590) Close stale PRs automatically
[
https://issues.apache.org/jira/browse/HIVE-23590?focusedWorklogId=441494&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441494
]
ASF GitHub Bot logged work on HIVE-23590:
-
Author: ASF GitHub Bot
Created on: 04/Jun/20 19:44
Start Date: 04/Jun/20 19:44
Worklog Time Spent: 10m
Work Description: jcamachor commented on a change in pull request #1049:
URL: https://github.com/apache/hive/pull/1049#discussion_r435507340
##
File path: .github/workflows/stale.yml
##
@@ -0,0 +1,15 @@
+name: "Close stale pull requests"
+on:
+ schedule:
+ - cron: "0 0 * * *"
+
+jobs:
+ stale:
+runs-on: ubuntu-latest
+steps:
+- uses: actions/stale@v3
+ with:
+repo-token: ${{ secrets.GITHUB_TOKEN }}
+stale-pr-message: 'This pull request has been automatically marked as
stale because it has not had recent activity. It will be closed if no further
activity occurs.'
+exempt-pr-labels: 'awaiting-approval,work-in-progress'
+stale-pr-label: stale
Review comment:
Oh, I did not realize that. Yes, I think making them explicit may be
better.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 441494)
Time Spent: 50m (was: 40m)
> Close stale PRs automatically
> -
>
> Key: HIVE-23590
> URL: https://issues.apache.org/jira/browse/HIVE-23590
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23556) Support hive.metastore.limit.partition.request for get_partitions_ps
[
https://issues.apache.org/jira/browse/HIVE-23556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17126146#comment-17126146
]
Hive QA commented on HIVE-23556:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13004849/HIVE-23556.2.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 17103 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.metastore.client.TestListPartitions.testListPartitionsWithAuthByValuesHighMaxParts[Embedded]
(batchId=153)
org.apache.hadoop.hive.metastore.client.TestListPartitions.testListPartitionsWithAuthByValuesHighMaxParts[Remote]
(batchId=153)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22748/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22748/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22748/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13004849 - PreCommit-HIVE-Build
> Support hive.metastore.limit.partition.request for get_partitions_ps
>
>
> Key: HIVE-23556
> URL: https://issues.apache.org/jira/browse/HIVE-23556
> Project: Hive
> Issue Type: Improvement
>Reporter: Toshihiko Uchida
>Assignee: Toshihiko Uchida
>Priority: Minor
> Attachments: HIVE-23556.2.patch, HIVE-23556.patch
>
>
> HIVE-13884 added the configuration hive.metastore.limit.partition.request to
> limit the number of partitions that can be requested.
> Currently, it takes in effect for the following MetaStore APIs
> * get_partitions,
> * get_partitions_with_auth,
> * get_partitions_by_filter,
> * get_partitions_spec_by_filter,
> * get_partitions_by_expr,
> but not for
> * get_partitions_ps,
> * get_partitions_ps_with_auth.
> This issue proposes to apply the configuration also to get_partitions_ps and
> get_partitions_ps_with_auth.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-23590) Close stale PRs automatically
[
https://issues.apache.org/jira/browse/HIVE-23590?focusedWorklogId=441469&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441469
]
ASF GitHub Bot logged work on HIVE-23590:
-
Author: ASF GitHub Bot
Created on: 04/Jun/20 18:33
Start Date: 04/Jun/20 18:33
Worklog Time Spent: 10m
Work Description: kgyrtkirk commented on a change in pull request #1049:
URL: https://github.com/apache/hive/pull/1049#discussion_r435467034
##
File path: .github/workflows/stale.yml
##
@@ -0,0 +1,15 @@
+name: "Close stale pull requests"
+on:
+ schedule:
+ - cron: "0 0 * * *"
+
+jobs:
+ stale:
+runs-on: ubuntu-latest
+steps:
+- uses: actions/stale@v3
+ with:
+repo-token: ${{ secrets.GITHUB_TOKEN }}
+stale-pr-message: 'This pull request has been automatically marked as
stale because it has not had recent activity. It will be closed if no further
activity occurs.'
+exempt-pr-labels: 'awaiting-approval,work-in-progress'
+stale-pr-label: stale
Review comment:
I thinked that we could use the defaults which are 60/7 respectively
https://github.com/actions/stale/blob/master/action.yml
should I add them to make it clearer?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 441469)
Time Spent: 40m (was: 0.5h)
> Close stale PRs automatically
> -
>
> Key: HIVE-23590
> URL: https://issues.apache.org/jira/browse/HIVE-23590
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23556) Support hive.metastore.limit.partition.request for get_partitions_ps
[
https://issues.apache.org/jira/browse/HIVE-23556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17126127#comment-17126127
]
Hive QA commented on HIVE-23556:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
48s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
17s{color} | {color:red} metastore-common in master failed. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
15s{color} | {color:red} metastore-server in master failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
30s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
32s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 9 new + 1189 unchanged - 0 fixed = 1198 total (was 1189) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
10s{color} | {color:red} itests/hcatalog-unit: The patch generated 1 new + 28
unchanged - 0 fixed = 29 total (was 28) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
14s{color} | {color:red} metastore-common in the patch failed. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
14s{color} | {color:red} metastore-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
32s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
11s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 20m 53s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22748/dev-support/hive-personality.sh
|
| git revision | master / b5e3bcd |
| Default Java | 1.8.0_111 |
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22748/yetus/branch-findbugs-standalone-metastore_metastore-common.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22748/yetus/branch-findbugs-standalone-metastore_metastore-server.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22748/yetus/diff-checkstyle-standalone-metastore_metastore-server.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22748/yetus/diff-checkstyle-itests_hcatalog-unit.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22748/yetus/patch-findbugs-standalone-metastore_metastore-common.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22748/yetus/patch-findbugs-standalone-metastore_metastore-server.txt
|
| modules | C: standalone-metastore/metastore-common
standalone-metastore/metastore-server itests/hcatalog-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22748/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This
[jira] [Updated] (HIVE-23556) Support hive.metastore.limit.partition.request for get_partitions_ps
[ https://issues.apache.org/jira/browse/HIVE-23556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiko Uchida updated HIVE-23556: Attachment: HIVE-23556.2.patch Status: Patch Available (was: Open) > Support hive.metastore.limit.partition.request for get_partitions_ps > > > Key: HIVE-23556 > URL: https://issues.apache.org/jira/browse/HIVE-23556 > Project: Hive > Issue Type: Improvement >Reporter: Toshihiko Uchida >Assignee: Toshihiko Uchida >Priority: Minor > Attachments: HIVE-23556.2.patch, HIVE-23556.patch > > > HIVE-13884 added the configuration hive.metastore.limit.partition.request to > limit the number of partitions that can be requested. > Currently, it takes in effect for the following MetaStore APIs > * get_partitions, > * get_partitions_with_auth, > * get_partitions_by_filter, > * get_partitions_spec_by_filter, > * get_partitions_by_expr, > but not for > * get_partitions_ps, > * get_partitions_ps_with_auth. > This issue proposes to apply the configuration also to get_partitions_ps and > get_partitions_ps_with_auth. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23556) Support hive.metastore.limit.partition.request for get_partitions_ps
[ https://issues.apache.org/jira/browse/HIVE-23556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiko Uchida updated HIVE-23556: Status: Open (was: Patch Available) > Support hive.metastore.limit.partition.request for get_partitions_ps > > > Key: HIVE-23556 > URL: https://issues.apache.org/jira/browse/HIVE-23556 > Project: Hive > Issue Type: Improvement >Reporter: Toshihiko Uchida >Assignee: Toshihiko Uchida >Priority: Minor > Attachments: HIVE-23556.patch > > > HIVE-13884 added the configuration hive.metastore.limit.partition.request to > limit the number of partitions that can be requested. > Currently, it takes in effect for the following MetaStore APIs > * get_partitions, > * get_partitions_with_auth, > * get_partitions_by_filter, > * get_partitions_spec_by_filter, > * get_partitions_by_expr, > but not for > * get_partitions_ps, > * get_partitions_ps_with_auth. > This issue proposes to apply the configuration also to get_partitions_ps and > get_partitions_ps_with_auth. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23596) LLAP: Encode initial guaranteed task information in containerId
[ https://issues.apache.org/jira/browse/HIVE-23596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal Vijayaraghavan updated HIVE-23596: Fix Version/s: 4.0.0 > LLAP: Encode initial guaranteed task information in containerId > --- > > Key: HIVE-23596 > URL: https://issues.apache.org/jira/browse/HIVE-23596 > Project: Hive > Issue Type: Improvement > Components: llap >Reporter: Mustafa Iman >Assignee: Mustafa Iman >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Time Spent: 10m > Remaining Estimate: 0h > > We should avoid calling LlapTaskScheduler to get initial isguaranteed flag > for all the tasks. It causes arbitrary delays in sending tasks out. Since > communicator is a single thread, any blocking there delays all the tasks. > There are [https://jira.apache.org/jira/browse/TEZ-4192] and > [https://jira.apache.org/jira/browse/HIVE-23589] for a proper solution to > this. However, that requires a Tez release which seems far right now. We can > replace the current hack with another hack that does not require locking. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-23596) LLAP: Encode initial guaranteed task information in containerId
[ https://issues.apache.org/jira/browse/HIVE-23596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal Vijayaraghavan resolved HIVE-23596. - Resolution: Fixed Pushed to master, thanks [~mustafaiman] > LLAP: Encode initial guaranteed task information in containerId > --- > > Key: HIVE-23596 > URL: https://issues.apache.org/jira/browse/HIVE-23596 > Project: Hive > Issue Type: Improvement > Components: llap >Reporter: Mustafa Iman >Assignee: Mustafa Iman >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Time Spent: 10m > Remaining Estimate: 0h > > We should avoid calling LlapTaskScheduler to get initial isguaranteed flag > for all the tasks. It causes arbitrary delays in sending tasks out. Since > communicator is a single thread, any blocking there delays all the tasks. > There are [https://jira.apache.org/jira/browse/TEZ-4192] and > [https://jira.apache.org/jira/browse/HIVE-23589] for a proper solution to > this. However, that requires a Tez release which seems far right now. We can > replace the current hack with another hack that does not require locking. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23596) LLAP: Encode initial guaranteed task information in containerId
[ https://issues.apache.org/jira/browse/HIVE-23596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal Vijayaraghavan updated HIVE-23596: Component/s: llap > LLAP: Encode initial guaranteed task information in containerId > --- > > Key: HIVE-23596 > URL: https://issues.apache.org/jira/browse/HIVE-23596 > Project: Hive > Issue Type: Improvement > Components: llap >Reporter: Mustafa Iman >Assignee: Mustafa Iman >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > We should avoid calling LlapTaskScheduler to get initial isguaranteed flag > for all the tasks. It causes arbitrary delays in sending tasks out. Since > communicator is a single thread, any blocking there delays all the tasks. > There are [https://jira.apache.org/jira/browse/TEZ-4192] and > [https://jira.apache.org/jira/browse/HIVE-23589] for a proper solution to > this. However, that requires a Tez release which seems far right now. We can > replace the current hack with another hack that does not require locking. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23590) Close stale PRs automatically
[
https://issues.apache.org/jira/browse/HIVE-23590?focusedWorklogId=441425&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441425
]
ASF GitHub Bot logged work on HIVE-23590:
-
Author: ASF GitHub Bot
Created on: 04/Jun/20 17:19
Start Date: 04/Jun/20 17:19
Worklog Time Spent: 10m
Work Description: jcamachor commented on a change in pull request #1049:
URL: https://github.com/apache/hive/pull/1049#discussion_r435421458
##
File path: .github/workflows/stale.yml
##
@@ -0,0 +1,15 @@
+name: "Close stale pull requests"
+on:
+ schedule:
+ - cron: "0 0 * * *"
+
+jobs:
+ stale:
+runs-on: ubuntu-latest
+steps:
+- uses: actions/stale@v3
+ with:
+repo-token: ${{ secrets.GITHUB_TOKEN }}
+stale-pr-message: 'This pull request has been automatically marked as
stale because it has not had recent activity. It will be closed if no further
activity occurs.'
+exempt-pr-labels: 'awaiting-approval,work-in-progress'
+stale-pr-label: stale
Review comment:
@kgyrtkirk , should we use the `days-before-stale` and
`days-before-close` configs? Maybe closing them automatically after 7 days?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 441425)
Time Spent: 0.5h (was: 20m)
> Close stale PRs automatically
> -
>
> Key: HIVE-23590
> URL: https://issues.apache.org/jira/browse/HIVE-23590
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23596) LLAP: Encode initial guaranteed task information in containerId
[ https://issues.apache.org/jira/browse/HIVE-23596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal Vijayaraghavan updated HIVE-23596: Summary: LLAP: Encode initial guaranteed task information in containerId (was: LLAP: Encode guaranteed task information in containerId) > LLAP: Encode initial guaranteed task information in containerId > --- > > Key: HIVE-23596 > URL: https://issues.apache.org/jira/browse/HIVE-23596 > Project: Hive > Issue Type: Improvement >Reporter: Mustafa Iman >Assignee: Mustafa Iman >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > We should avoid calling LlapTaskScheduler to get initial isguaranteed flag > for all the tasks. It causes arbitrary delays in sending tasks out. Since > communicator is a single thread, any blocking there delays all the tasks. > There are [https://jira.apache.org/jira/browse/TEZ-4192] and > [https://jira.apache.org/jira/browse/HIVE-23589] for a proper solution to > this. However, that requires a Tez release which seems far right now. We can > replace the current hack with another hack that does not require locking. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-20890) ACID: Allow whole table ReadLocks to skip all partition locks
[
https://issues.apache.org/jira/browse/HIVE-20890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17126086#comment-17126086
]
Hive QA commented on HIVE-20890:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13004832/HIVE-20890.4.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17100 tests
executed
*Failed tests:*
{noformat}
TestStatsReplicationScenariosACID - did not produce a TEST-*.xml file (likely
timed out) (batchId=185)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22747/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22747/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22747/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13004832 - PreCommit-HIVE-Build
> ACID: Allow whole table ReadLocks to skip all partition locks
> -
>
> Key: HIVE-20890
> URL: https://issues.apache.org/jira/browse/HIVE-20890
> Project: Hive
> Issue Type: Improvement
> Components: Transactions
>Reporter: Gopal Vijayaraghavan
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-20890.1.patch, HIVE-20890.2.patch,
> HIVE-20890.3.patch, HIVE-20890.4.patch
>
>
> HIVE-19369 proposes adding a EXCL_WRITE lock which does not wait for any
> SHARED_READ locks for read operations - in the presence of that lock, the
> insert overwrite no longer takes an exclusive lock.
> The only exclusive operation will be a schema change or drop table, which
> should take an exclusive lock on the entire table directly.
> {code}
> explain locks select * from tpcds_bin_partitioned_orc_1000.store_sales where
> ss_sold_date_sk=2452626
> ++
> | Explain |
> ++
> | LOCK INFORMATION: |
> | tpcds_bin_partitioned_orc_1000.store_sales -> SHARED_READ |
> | tpcds_bin_partitioned_orc_1000.store_sales.ss_sold_date_sk=2452626 ->
> SHARED_READ |
> ++
> {code}
> So the per-partition SHARED_READ locks are no longer necessary, if the lock
> builder already includes the table-wide SHARED_READ locks.
> The removal of entire partitions is the only part which needs to be taken
> care of within this semantics as row-removal instead of directory removal
> (i.e "drop partition" -> "truncate partition" and have the truncation trigger
> a whole directory cleaner, so that the partition disappears when there are 0
> rows left).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23596) LLAP: Encode guaranteed task information in containerId
[ https://issues.apache.org/jira/browse/HIVE-23596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal Vijayaraghavan updated HIVE-23596: Summary: LLAP: Encode guaranteed task information in containerId (was: Encode guaranteed task information in containerId) > LLAP: Encode guaranteed task information in containerId > --- > > Key: HIVE-23596 > URL: https://issues.apache.org/jira/browse/HIVE-23596 > Project: Hive > Issue Type: Improvement >Reporter: Mustafa Iman >Assignee: Mustafa Iman >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > We should avoid calling LlapTaskScheduler to get initial isguaranteed flag > for all the tasks. It causes arbitrary delays in sending tasks out. Since > communicator is a single thread, any blocking there delays all the tasks. > There are [https://jira.apache.org/jira/browse/TEZ-4192] and > [https://jira.apache.org/jira/browse/HIVE-23589] for a proper solution to > this. However, that requires a Tez release which seems far right now. We can > replace the current hack with another hack that does not require locking. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23609) SemiJoin: Relax big table size check for self-joins
[ https://issues.apache.org/jira/browse/HIVE-23609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal Vijayaraghavan updated HIVE-23609: Attachment: HIVE-23609.WIP.patch > SemiJoin: Relax big table size check for self-joins > --- > > Key: HIVE-23609 > URL: https://issues.apache.org/jira/browse/HIVE-23609 > Project: Hive > Issue Type: Improvement >Reporter: Gopal Vijayaraghavan >Assignee: Gopal Vijayaraghavan >Priority: Major > Attachments: HIVE-23609.WIP.patch > > > For self-joins, several other heuristics applied to Semijoins don't apply as > the difference between rows on either side is likely to result in an exact > reduction of rows scanned. > This change results in slightly different Tez priorities for self-joins which > are heavily filtered on one side over the other, which helps ensure the > smaller table is completed before the bigger table consumes resources. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-20890) ACID: Allow whole table ReadLocks to skip all partition locks
[
https://issues.apache.org/jira/browse/HIVE-20890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17126068#comment-17126068
]
Hive QA commented on HIVE-20890:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
0s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
57s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
8s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
34s{color} | {color:blue} common in master has 63 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
54s{color} | {color:blue} ql in master has 1522 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
12s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
54s{color} | {color:red} ql: The patch generated 13 new + 219 unchanged - 5
fixed = 232 total (was 224) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 29m 34s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22747/dev-support/hive-personality.sh
|
| git revision | master / 1271d08 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22747/yetus/diff-checkstyle-ql.txt
|
| modules | C: common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22747/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> ACID: Allow whole table ReadLocks to skip all partition locks
> -
>
> Key: HIVE-20890
> URL: https://issues.apache.org/jira/browse/HIVE-20890
> Project: Hive
> Issue Type: Improvement
> Components: Transactions
>Reporter: Gopal Vijayaraghavan
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-20890.1.patch, HIVE-20890.2.patch,
> HIVE-20890.3.patch, HIVE-20890.4.patch
>
>
> HIVE-19369 proposes adding a EXCL_WRITE lock which does not wait for any
> SHARED_READ locks for read operations - in the presence of that lock, the
> insert overwrite no longer takes an exclusive lock.
> The only exclusive operation will be a schema change or drop table, which
> should take an exclusive lock on the entire table directly.
> {code}
> explain locks select * from tpcds_bin_partitioned_orc_1000.store_sales where
> ss_sold_date_sk=2452626
> ++
> |
[jira] [Commented] (HIVE-23607) Permission Issue: Create view on another view succeeds but alter view fails
[
https://issues.apache.org/jira/browse/HIVE-23607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17126047#comment-17126047
]
Naresh P R commented on HIVE-23607:
---
Incase on select on view, there will be 2 ReadEntity..
1 for view with noParent & isDirect is true, 2nd for underlying table with view
as parent & isDirect is false.
Same will be validated while preparing privObject
[https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/security/authorization/command/CommandAuthorizerV2.java#L106-L109]
ie., Incase of view, underlying table/view privs need not to be verified. Only
parent view privs to be checked for user.
> Permission Issue: Create view on another view succeeds but alter view fails
> -
>
> Key: HIVE-23607
> URL: https://issues.apache.org/jira/browse/HIVE-23607
> Project: Hive
> Issue Type: Bug
>Reporter: Naresh P R
>Assignee: Naresh P R
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-23607.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> create table test_table (id int);
> create view test_view as select * from test_table;
>
> {code:java}
> -- user "abc" has read access on test_view
> -- Create view succeeds
> create view test_view_1 as select * from test_view;
> -- Alter view fails
> alter view test_view_1 as select * from test_view
> Error: Error while compiling statement: FAILED: HiveAccessControlException
> Permission denied: user [abc] does not have [SELECT] privilege on
> [test/test_table] (state=42000,code=4)
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23612) Option for HiveStrictManagedMigration to impersonate a user for FS operations
[
https://issues.apache.org/jira/browse/HIVE-23612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17126040#comment-17126040
]
Hive QA commented on HIVE-23612:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13004827/HIVE-23612.1.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17103 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22746/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22746/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22746/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13004827 - PreCommit-HIVE-Build
> Option for HiveStrictManagedMigration to impersonate a user for FS operations
> -
>
> Key: HIVE-23612
> URL: https://issues.apache.org/jira/browse/HIVE-23612
> Project: Hive
> Issue Type: Improvement
>Reporter: Ádám Szita
>Assignee: Ádám Szita
>Priority: Major
> Attachments: HIVE-23612.0.patch, HIVE-23612.1.patch
>
>
> HiveStrictManagedMigration tool can be used to move HDFS paths and to change
> ownership on said paths. It may be beneficial to do such file system
> operations as a different user than the one the tool itself is run.
> Moreover, while creating the external DB directory, the tool will chown the
> new directory to the user set as DB owner in HMS. If this is unset, no chown
> command is used. In this case we should make the 'hive' user the directory
> owner.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23590) Close stale PRs automatically
[ https://issues.apache.org/jira/browse/HIVE-23590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17126028#comment-17126028 ] Zoltan Haindrich commented on HIVE-23590: - [~jcamachorodriguez] Could you please take a look? > Close stale PRs automatically > - > > Key: HIVE-23590 > URL: https://issues.apache.org/jira/browse/HIVE-23590 > Project: Hive > Issue Type: Sub-task >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23613) Cleanup FindBugs
[
https://issues.apache.org/jira/browse/HIVE-23613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17126017#comment-17126017
]
Zoltan Haindrich commented on HIVE-23613:
-
I think this should be also enforced during precommit - start with some low
level module like storage-api or common and fix the findbugs issues below that
- and enforce that precommit will fail in case someone tries to re-introduce
issues with it
> Cleanup FindBugs
>
>
> Key: HIVE-23613
> URL: https://issues.apache.org/jira/browse/HIVE-23613
> Project: Hive
> Issue Type: Bug
>Reporter: László Bodor
>Priority: Major
> Fix For: 4.0.0
>
>
> Let this Jira be an umbrella for an effort of cleaning up the enormous amount
> of FindBugs warnings in Hive project.
> {code}
> mvn -DskipTests -Pitests,findbugs test-compile findbugs:findbugs
> -DskipTests=true
> {code}
> outputs
> {code}
> find . -name findbugsXml.xml
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23612) Option for HiveStrictManagedMigration to impersonate a user for FS operations
[
https://issues.apache.org/jira/browse/HIVE-23612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17126011#comment-17126011
]
Hive QA commented on HIVE-23612:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
52s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
48s{color} | {color:blue} ql in master has 1522 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 25s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22746/dev-support/hive-personality.sh
|
| git revision | master / 1271d08 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22746/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Option for HiveStrictManagedMigration to impersonate a user for FS operations
> -
>
> Key: HIVE-23612
> URL: https://issues.apache.org/jira/browse/HIVE-23612
> Project: Hive
> Issue Type: Improvement
>Reporter: Ádám Szita
>Assignee: Ádám Szita
>Priority: Major
> Attachments: HIVE-23612.0.patch, HIVE-23612.1.patch
>
>
> HiveStrictManagedMigration tool can be used to move HDFS paths and to change
> ownership on said paths. It may be beneficial to do such file system
> operations as a different user than the one the tool itself is run.
> Moreover, while creating the external DB directory, the tool will chown the
> new directory to the user set as DB owner in HMS. If this is unset, no chown
> command is used. In this case we should make the 'hive' user the directory
> owner.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-20890) ACID: Allow whole table ReadLocks to skip all partition locks
[
https://issues.apache.org/jira/browse/HIVE-20890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Denys Kuzmenko updated HIVE-20890:
--
Attachment: HIVE-20890.4.patch
> ACID: Allow whole table ReadLocks to skip all partition locks
> -
>
> Key: HIVE-20890
> URL: https://issues.apache.org/jira/browse/HIVE-20890
> Project: Hive
> Issue Type: Improvement
> Components: Transactions
>Reporter: Gopal Vijayaraghavan
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-20890.1.patch, HIVE-20890.2.patch,
> HIVE-20890.3.patch, HIVE-20890.4.patch
>
>
> HIVE-19369 proposes adding a EXCL_WRITE lock which does not wait for any
> SHARED_READ locks for read operations - in the presence of that lock, the
> insert overwrite no longer takes an exclusive lock.
> The only exclusive operation will be a schema change or drop table, which
> should take an exclusive lock on the entire table directly.
> {code}
> explain locks select * from tpcds_bin_partitioned_orc_1000.store_sales where
> ss_sold_date_sk=2452626
> ++
> | Explain |
> ++
> | LOCK INFORMATION: |
> | tpcds_bin_partitioned_orc_1000.store_sales -> SHARED_READ |
> | tpcds_bin_partitioned_orc_1000.store_sales.ss_sold_date_sk=2452626 ->
> SHARED_READ |
> ++
> {code}
> So the per-partition SHARED_READ locks are no longer necessary, if the lock
> builder already includes the table-wide SHARED_READ locks.
> The removal of entire partitions is the only part which needs to be taken
> care of within this semantics as row-removal instead of directory removal
> (i.e "drop partition" -> "truncate partition" and have the truncation trigger
> a whole directory cleaner, so that the partition disappears when there are 0
> rows left).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-23526) Beeline may throw the misleading exception
[
https://issues.apache.org/jira/browse/HIVE-23526?focusedWorklogId=441318&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441318
]
ASF GitHub Bot logged work on HIVE-23526:
-
Author: ASF GitHub Bot
Created on: 04/Jun/20 14:53
Start Date: 04/Jun/20 14:53
Worklog Time Spent: 10m
Work Description: dengzhhu653 commented on a change in pull request #1029:
URL: https://github.com/apache/hive/pull/1029#discussion_r435321340
##
File path: jdbc/src/java/org/apache/hive/jdbc/HiveStatement.java
##
@@ -198,9 +198,9 @@ private void closeStatementIfNeeded() throws SQLException {
} catch (SQLException e) {
throw e;
} catch (TApplicationException tae) {
- String errorMsg = tae.getMessage();
+ String errorMsg = "Failed to close statement";
if (tae.getType() == TApplicationException.BAD_SEQUENCE_ID) {
-errorMsg = "Mismatch thrift sequence id. A previous call to the Thrift
library failed"
+errorMsg += ". Mismatch thrift sequence id. A previous call to the
Thrift library failed"
Review comment:
Done. Thank you @belugabehr
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 441318)
Time Spent: 6h 20m (was: 6h 10m)
> Beeline may throw the misleading exception
> --
>
> Key: HIVE-23526
> URL: https://issues.apache.org/jira/browse/HIVE-23526
> Project: Hive
> Issue Type: Improvement
> Components: Beeline
> Environment: Hive 1.2.2
>Reporter: Zhihua Deng
>Priority: Minor
> Labels: pull-request-available
> Attachments: HIVE-23526.2.patch, HIVE-23526.3.patch,
> HIVE-23526.patch, outofsequence.log
>
> Time Spent: 6h 20m
> Remaining Estimate: 0h
>
> Sometimes we can see 'out of sequence response' message in beeline, for
> example:
> Error: org.apache.thrift.TApplicationException: CloseOperation failed: out of
> sequence response (state=08S01,code=0)
> java.sql.SQLException: org.apache.thrift.TApplicationException:
> CloseOperation failed: out of sequence response
> at
> org.apache.hive.jdbc.HiveStatement.closeClientOperation(HiveStatement.java:198)
> at org.apache.hive.jdbc.HiveStatement.close(HiveStatement.java:217)
> at org.apache.hive.beeline.Commands.execute(Commands.java:891)
> at org.apache.hive.beeline.Commands.sql(Commands.java:713)
> at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:976)
> at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:816)
> at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:774)
> at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:487)
> at org.apache.hive.beeline.BeeLine.main(BeeLine.java:470)
> and there is no other usage message to figured it out, even with --verbose,
> this makes problem puzzled as beeline does not have concurrency problem on
> underlying thrift transport.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23612) Option for HiveStrictManagedMigration to impersonate a user for FS operations
[ https://issues.apache.org/jira/browse/HIVE-23612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ádám Szita updated HIVE-23612: -- Attachment: HIVE-23612.1.patch > Option for HiveStrictManagedMigration to impersonate a user for FS operations > - > > Key: HIVE-23612 > URL: https://issues.apache.org/jira/browse/HIVE-23612 > Project: Hive > Issue Type: Improvement >Reporter: Ádám Szita >Assignee: Ádám Szita >Priority: Major > Attachments: HIVE-23612.0.patch, HIVE-23612.1.patch > > > HiveStrictManagedMigration tool can be used to move HDFS paths and to change > ownership on said paths. It may be beneficial to do such file system > operations as a different user than the one the tool itself is run. > Moreover, while creating the external DB directory, the tool will chown the > new directory to the user set as DB owner in HMS. If this is unset, no chown > command is used. In this case we should make the 'hive' user the directory > owner. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23526) Beeline may throw the misleading exception
[
https://issues.apache.org/jira/browse/HIVE-23526?focusedWorklogId=441297&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441297
]
ASF GitHub Bot logged work on HIVE-23526:
-
Author: ASF GitHub Bot
Created on: 04/Jun/20 14:32
Start Date: 04/Jun/20 14:32
Worklog Time Spent: 10m
Work Description: belugabehr commented on a change in pull request #1029:
URL: https://github.com/apache/hive/pull/1029#discussion_r435305205
##
File path: jdbc/src/java/org/apache/hive/jdbc/HiveStatement.java
##
@@ -198,9 +198,9 @@ private void closeStatementIfNeeded() throws SQLException {
} catch (SQLException e) {
throw e;
} catch (TApplicationException tae) {
- String errorMsg = tae.getMessage();
+ String errorMsg = "Failed to close statement";
if (tae.getType() == TApplicationException.BAD_SEQUENCE_ID) {
-errorMsg = "Mismatch thrift sequence id. A previous call to the Thrift
library failed"
+errorMsg += ". Mismatch thrift sequence id. A previous call to the
Thrift library failed"
Review comment:
```
errorMsg = "Failed to close statement. Mismatch thrift sequence id. A
previous call to the Thrift library failed...";
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 441297)
Time Spent: 6h 10m (was: 6h)
> Beeline may throw the misleading exception
> --
>
> Key: HIVE-23526
> URL: https://issues.apache.org/jira/browse/HIVE-23526
> Project: Hive
> Issue Type: Improvement
> Components: Beeline
> Environment: Hive 1.2.2
>Reporter: Zhihua Deng
>Priority: Minor
> Labels: pull-request-available
> Attachments: HIVE-23526.2.patch, HIVE-23526.3.patch,
> HIVE-23526.patch, outofsequence.log
>
> Time Spent: 6h 10m
> Remaining Estimate: 0h
>
> Sometimes we can see 'out of sequence response' message in beeline, for
> example:
> Error: org.apache.thrift.TApplicationException: CloseOperation failed: out of
> sequence response (state=08S01,code=0)
> java.sql.SQLException: org.apache.thrift.TApplicationException:
> CloseOperation failed: out of sequence response
> at
> org.apache.hive.jdbc.HiveStatement.closeClientOperation(HiveStatement.java:198)
> at org.apache.hive.jdbc.HiveStatement.close(HiveStatement.java:217)
> at org.apache.hive.beeline.Commands.execute(Commands.java:891)
> at org.apache.hive.beeline.Commands.sql(Commands.java:713)
> at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:976)
> at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:816)
> at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:774)
> at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:487)
> at org.apache.hive.beeline.BeeLine.main(BeeLine.java:470)
> and there is no other usage message to figured it out, even with --verbose,
> this makes problem puzzled as beeline does not have concurrency problem on
> underlying thrift transport.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-23526) Beeline may throw the misleading exception
[
https://issues.apache.org/jira/browse/HIVE-23526?focusedWorklogId=441296&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441296
]
ASF GitHub Bot logged work on HIVE-23526:
-
Author: ASF GitHub Bot
Created on: 04/Jun/20 14:31
Start Date: 04/Jun/20 14:31
Worklog Time Spent: 10m
Work Description: belugabehr commented on a change in pull request #1029:
URL: https://github.com/apache/hive/pull/1029#discussion_r435304118
##
File path: jdbc/src/java/org/apache/hive/jdbc/HiveStatement.java
##
@@ -198,9 +198,9 @@ private void closeStatementIfNeeded() throws SQLException {
} catch (SQLException e) {
throw e;
} catch (TApplicationException tae) {
- String errorMsg = tae.getMessage();
+ String errorMsg = "Failed to close statement";
if (tae.getType() == TApplicationException.BAD_SEQUENCE_ID) {
-errorMsg = "Mismatch thrift sequence id. A previous call to the Thrift
library failed"
+errorMsg += ". Mismatch thrift sequence id. A previous call to the
Thrift library failed"
Review comment:
Lol. I'm so sorry to do this to you, I've been in your seat quite a few
times myself, but can you please do one more push? Please do not do the string
concatenation. This could be ripe for errors if the code changes later. Just
use a single static string which includes the words "Failed to close statement."
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 441296)
Time Spent: 6h (was: 5h 50m)
> Beeline may throw the misleading exception
> --
>
> Key: HIVE-23526
> URL: https://issues.apache.org/jira/browse/HIVE-23526
> Project: Hive
> Issue Type: Improvement
> Components: Beeline
> Environment: Hive 1.2.2
>Reporter: Zhihua Deng
>Priority: Minor
> Labels: pull-request-available
> Attachments: HIVE-23526.2.patch, HIVE-23526.3.patch,
> HIVE-23526.patch, outofsequence.log
>
> Time Spent: 6h
> Remaining Estimate: 0h
>
> Sometimes we can see 'out of sequence response' message in beeline, for
> example:
> Error: org.apache.thrift.TApplicationException: CloseOperation failed: out of
> sequence response (state=08S01,code=0)
> java.sql.SQLException: org.apache.thrift.TApplicationException:
> CloseOperation failed: out of sequence response
> at
> org.apache.hive.jdbc.HiveStatement.closeClientOperation(HiveStatement.java:198)
> at org.apache.hive.jdbc.HiveStatement.close(HiveStatement.java:217)
> at org.apache.hive.beeline.Commands.execute(Commands.java:891)
> at org.apache.hive.beeline.Commands.sql(Commands.java:713)
> at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:976)
> at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:816)
> at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:774)
> at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:487)
> at org.apache.hive.beeline.BeeLine.main(BeeLine.java:470)
> and there is no other usage message to figured it out, even with --verbose,
> this makes problem puzzled as beeline does not have concurrency problem on
> underlying thrift transport.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-23526) Beeline may throw the misleading exception
[
https://issues.apache.org/jira/browse/HIVE-23526?focusedWorklogId=441294&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441294
]
ASF GitHub Bot logged work on HIVE-23526:
-
Author: ASF GitHub Bot
Created on: 04/Jun/20 14:29
Start Date: 04/Jun/20 14:29
Worklog Time Spent: 10m
Work Description: dengzhhu653 commented on a change in pull request #1029:
URL: https://github.com/apache/hive/pull/1029#discussion_r435303133
##
File path: jdbc/src/java/org/apache/hive/jdbc/HiveStatement.java
##
@@ -196,6 +197,14 @@ private void closeStatementIfNeeded() throws SQLException {
}
} catch (SQLException e) {
throw e;
+} catch (TApplicationException tae) {
+ String errorMsg = tae.getMessage();
Review comment:
Done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 441294)
Time Spent: 5h 50m (was: 5h 40m)
> Beeline may throw the misleading exception
> --
>
> Key: HIVE-23526
> URL: https://issues.apache.org/jira/browse/HIVE-23526
> Project: Hive
> Issue Type: Improvement
> Components: Beeline
> Environment: Hive 1.2.2
>Reporter: Zhihua Deng
>Priority: Minor
> Labels: pull-request-available
> Attachments: HIVE-23526.2.patch, HIVE-23526.3.patch,
> HIVE-23526.patch, outofsequence.log
>
> Time Spent: 5h 50m
> Remaining Estimate: 0h
>
> Sometimes we can see 'out of sequence response' message in beeline, for
> example:
> Error: org.apache.thrift.TApplicationException: CloseOperation failed: out of
> sequence response (state=08S01,code=0)
> java.sql.SQLException: org.apache.thrift.TApplicationException:
> CloseOperation failed: out of sequence response
> at
> org.apache.hive.jdbc.HiveStatement.closeClientOperation(HiveStatement.java:198)
> at org.apache.hive.jdbc.HiveStatement.close(HiveStatement.java:217)
> at org.apache.hive.beeline.Commands.execute(Commands.java:891)
> at org.apache.hive.beeline.Commands.sql(Commands.java:713)
> at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:976)
> at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:816)
> at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:774)
> at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:487)
> at org.apache.hive.beeline.BeeLine.main(BeeLine.java:470)
> and there is no other usage message to figured it out, even with --verbose,
> this makes problem puzzled as beeline does not have concurrency problem on
> underlying thrift transport.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-23526) Beeline may throw the misleading exception
[ https://issues.apache.org/jira/browse/HIVE-23526?focusedWorklogId=441291&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441291 ] ASF GitHub Bot logged work on HIVE-23526: - Author: ASF GitHub Bot Created on: 04/Jun/20 14:23 Start Date: 04/Jun/20 14:23 Worklog Time Spent: 10m Work Description: belugabehr commented on pull request #1029: URL: https://github.com/apache/hive/pull/1029#issuecomment-638881978 Thanks for your work on this and for bringing it to our attention. I think the changes we ended up with will be helpful for others. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 441291) Time Spent: 5h 40m (was: 5.5h) > Beeline may throw the misleading exception > -- > > Key: HIVE-23526 > URL: https://issues.apache.org/jira/browse/HIVE-23526 > Project: Hive > Issue Type: Improvement > Components: Beeline > Environment: Hive 1.2.2 >Reporter: Zhihua Deng >Priority: Minor > Labels: pull-request-available > Attachments: HIVE-23526.2.patch, HIVE-23526.3.patch, > HIVE-23526.patch, outofsequence.log > > Time Spent: 5h 40m > Remaining Estimate: 0h > > Sometimes we can see 'out of sequence response' message in beeline, for > example: > Error: org.apache.thrift.TApplicationException: CloseOperation failed: out of > sequence response (state=08S01,code=0) > java.sql.SQLException: org.apache.thrift.TApplicationException: > CloseOperation failed: out of sequence response > at > org.apache.hive.jdbc.HiveStatement.closeClientOperation(HiveStatement.java:198) > at org.apache.hive.jdbc.HiveStatement.close(HiveStatement.java:217) > at org.apache.hive.beeline.Commands.execute(Commands.java:891) > at org.apache.hive.beeline.Commands.sql(Commands.java:713) > at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:976) > at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:816) > at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:774) > at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:487) > at org.apache.hive.beeline.BeeLine.main(BeeLine.java:470) > and there is no other usage message to figured it out, even with --verbose, > this makes problem puzzled as beeline does not have concurrency problem on > underlying thrift transport. -- This message was sent by Atlassian Jira (v8.3.4#803005)
