[jira] [Commented] (HIVE-22783) Add test for HIVE-22366
[
https://issues.apache.org/jira/browse/HIVE-22783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17031350#comment-17031350
]
Hive QA commented on HIVE-22783:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12992714/HIVE-22783.2.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17965 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20489/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20489/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20489/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12992714 - PreCommit-HIVE-Build
> Add test for HIVE-22366
> ---
>
> Key: HIVE-22783
> URL: https://issues.apache.org/jira/browse/HIVE-22783
> Project: Hive
> Issue Type: Sub-task
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22783.1.patch, HIVE-22783.2.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22783) Add test for HIVE-22366
[
https://issues.apache.org/jira/browse/HIVE-22783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17031312#comment-17031312
]
Hive QA commented on HIVE-22783:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
38s{color} | {color:blue} ql in master has 1533 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
38s{color} | {color:red} ql: The patch generated 34 new + 0 unchanged - 0 fixed
= 34 total (was 0) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 1s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20489/dev-support/hive-personality.sh
|
| git revision | master / dabfa86 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20489/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20489/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Add test for HIVE-22366
> ---
>
> Key: HIVE-22783
> URL: https://issues.apache.org/jira/browse/HIVE-22783
> Project: Hive
> Issue Type: Sub-task
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22783.1.patch, HIVE-22783.2.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22835) Extract Executor from Driver
[
https://issues.apache.org/jira/browse/HIVE-22835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17031299#comment-17031299
]
Hive QA commented on HIVE-22835:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12992703/HIVE-22835.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17965 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20488/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20488/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20488/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12992703 - PreCommit-HIVE-Build
> Extract Executor from Driver
>
>
> Key: HIVE-22835
> URL: https://issues.apache.org/jira/browse/HIVE-22835
> Project: Hive
> Issue Type: Sub-task
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22835.01.patch
>
>
> The Driver class contains ~600 lines of code responsible for executing the
> command. That means that from the Query Plan as input the appropriate tasks
> should be executed. This is a thing done by the execute function, which is
> ~300 lines long, and also has some sub functions to help this task. All these
> codes should be put into a separate class, where it can do it's job without
> getting mixed with the other codes in the Driver.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22835) Extract Executor from Driver
[
https://issues.apache.org/jira/browse/HIVE-22835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17031285#comment-17031285
]
Hive QA commented on HIVE-22835:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
42s{color} | {color:blue} ql in master has 1533 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
41s{color} | {color:red} ql: The patch generated 3 new + 31 unchanged - 7 fixed
= 34 total (was 38) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
53s{color} | {color:green} ql generated 0 new + 1532 unchanged - 1 fixed = 1532
total (was 1533) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
54s{color} | {color:red} ql generated 1 new + 99 unchanged - 1 fixed = 100
total (was 100) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 20s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20488/dev-support/hive-personality.sh
|
| git revision | master / dabfa86 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20488/yetus/diff-checkstyle-ql.txt
|
| javadoc |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20488/yetus/diff-javadoc-javadoc-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20488/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Extract Executor from Driver
>
>
> Key: HIVE-22835
> URL: https://issues.apache.org/jira/browse/HIVE-22835
> Project: Hive
> Issue Type: Sub-task
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22835.01.patch
>
>
> The Driver class contains ~600 lines of code responsible for executing the
> command. That means that from the Query Plan as input the appropriate tasks
> should be executed. This is a thing done by the execute function, which is
> ~300 lines long, and also has some sub functions to help this task. All these
> codes should be put into a separate class, where it can do it's job without
> getting mixed with the other codes in the Driver.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22736) Support replication across multiple encryption zones
[ https://issues.apache.org/jira/browse/HIVE-22736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-22736: --- Status: In Progress (was: Patch Available) > Support replication across multiple encryption zones > > > Key: HIVE-22736 > URL: https://issues.apache.org/jira/browse/HIVE-22736 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch > > Time Spent: 3h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22736) Support replication across multiple encryption zones
[ https://issues.apache.org/jira/browse/HIVE-22736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-22736: --- Attachment: HIVE-22736.patch Status: Patch Available (was: In Progress) > Support replication across multiple encryption zones > > > Key: HIVE-22736 > URL: https://issues.apache.org/jira/browse/HIVE-22736 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch > > Time Spent: 3h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22736) Support replication across multiple encryption zones
[ https://issues.apache.org/jira/browse/HIVE-22736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-22736: --- Status: In Progress (was: Patch Available) > Support replication across multiple encryption zones > > > Key: HIVE-22736 > URL: https://issues.apache.org/jira/browse/HIVE-22736 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch > > Time Spent: 3h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22736) Support replication across multiple encryption zones
[ https://issues.apache.org/jira/browse/HIVE-22736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-22736: --- Attachment: HIVE-22736.patch Status: Patch Available (was: In Progress) > Support replication across multiple encryption zones > > > Key: HIVE-22736 > URL: https://issues.apache.org/jira/browse/HIVE-22736 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch > > Time Spent: 3h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22610) Minor compaction for MM (insert-only) tables
[
https://issues.apache.org/jira/browse/HIVE-22610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17031271#comment-17031271
]
Hive QA commented on HIVE-22610:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12992700/HIVE-22610.01.patch
{color:green}SUCCESS:{color} +1 due to 4 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 6 failed/errored test(s), 17983 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.TestTxnCommands2.testMmTableCompaction (batchId=340)
org.apache.hadoop.hive.ql.TestTxnCommands2WithSplitUpdateAndVectorization.testMmTableCompaction
(batchId=354)
org.apache.hadoop.hive.ql.TestTxnCommandsForMmTable.testMmTableCompaction
(batchId=314)
org.apache.hadoop.hive.ql.TestTxnCommandsForMmTable.testSnapshotIsolationWithAbortedTxnOnMmTable
(batchId=314)
org.apache.hadoop.hive.ql.TestTxnCommandsForOrcMmTable.testMmTableCompaction
(batchId=336)
org.apache.hadoop.hive.ql.TestTxnCommandsForOrcMmTable.testSnapshotIsolationWithAbortedTxnOnMmTable
(batchId=336)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20487/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20487/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20487/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 6 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12992700 - PreCommit-HIVE-Build
> Minor compaction for MM (insert-only) tables
>
>
> Key: HIVE-22610
> URL: https://issues.apache.org/jira/browse/HIVE-22610
> Project: Hive
> Issue Type: Sub-task
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Major
> Attachments: HIVE-22610.01.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22736) Support replication across multiple encryption zones
[ https://issues.apache.org/jira/browse/HIVE-22736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-22736: --- Attachment: HIVE-22736.patch Status: Patch Available (was: In Progress) > Support replication across multiple encryption zones > > > Key: HIVE-22736 > URL: https://issues.apache.org/jira/browse/HIVE-22736 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch > > Time Spent: 3h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22736) Support replication across multiple encryption zones
[ https://issues.apache.org/jira/browse/HIVE-22736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-22736: --- Status: In Progress (was: Patch Available) > Support replication across multiple encryption zones > > > Key: HIVE-22736 > URL: https://issues.apache.org/jira/browse/HIVE-22736 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch > > Time Spent: 3h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22610) Minor compaction for MM (insert-only) tables
[
https://issues.apache.org/jira/browse/HIVE-22610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17031252#comment-17031252
]
Hive QA commented on HIVE-22610:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
41s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
54s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
46s{color} | {color:blue} ql in master has 1533 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
41s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
19s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
44s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
42s{color} | {color:red} ql: The patch generated 2 new + 518 unchanged - 0
fixed = 520 total (was 518) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
17s{color} | {color:red} itests/hive-unit: The patch generated 15 new + 8
unchanged - 3 fixed = 23 total (was 11) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
14s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 30m 6s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20487/dev-support/hive-personality.sh
|
| git revision | master / dabfa86 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20487/yetus/diff-checkstyle-ql.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20487/yetus/diff-checkstyle-itests_hive-unit.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20487/yetus/patch-asflicense-problems.txt
|
| modules | C: ql itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20487/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Minor compaction for MM (insert-only) tables
>
>
> Key: HIVE-22610
> URL: https://issues.apache.org/jira/browse/HIVE-22610
> Project: Hive
> Issue Type: Sub-task
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Major
> Attachments: HIVE-22610.01.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22825) Reduce directory lookup cost for acid tables
[ https://issues.apache.org/jira/browse/HIVE-22825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HIVE-22825: Attachment: HIVE-22825.4.patch > Reduce directory lookup cost for acid tables > > > Key: HIVE-22825 > URL: https://issues.apache.org/jira/browse/HIVE-22825 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Rajesh Balamohan >Assignee: Rajesh Balamohan >Priority: Minor > Attachments: HIVE-22825.1.patch, HIVE-22825.2.patch, > HIVE-22825.3.patch, HIVE-22825.4.patch > > > With objectstores, directory lookup costs are expensive. For acid tables, it > would be good to have a directory cache to reduce number of lookup calls. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-22836) HiveServer2 sttart failed for tez ClassNotFoundException
[
https://issues.apache.org/jira/browse/HIVE-22836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaohui Liu reassigned HIVE-22836:
--
> HiveServer2 sttart failed for tez ClassNotFoundException
>
>
> Key: HIVE-22836
> URL: https://issues.apache.org/jira/browse/HIVE-22836
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.2
>Reporter: Shaohui Liu
>Assignee: Shaohui Liu
>Priority: Major
>
> {code:java}
> 2020-02-05 21:36:54 [main] INFO HiveServer2:877 - Stopping/Disconnecting tez
> sessions.
> 2020-02-05 21:36:54 [main] INFO HiveServer2:883 - Stopped tez session pool
> manager.
> 2020-02-05 21:36:54 [main] ERROR HiveServer2:1307 - Error starting HiveServer2
> java.lang.Error: Max start attempts 1 exhausted
> at
> org.apache.hive.service.server.HiveServer2.startHiveServer2(HiveServer2.java:1062)
> ~[hive-service-3.1.2-mdh3.1.2.0-SNAPSHOT.jar:3.1.2-mdh3.1.2.0-SNAPSHOT]
> at
> org.apache.hive.service.server.HiveServer2.access$1600(HiveServer2.java:140)
> ~[hive-service-3.1.2-mdh3.1.2.0-SNAPSHOT.jar:3.1.2-mdh3.1.2.0-SNAPSHOT]
> at
> org.apache.hive.service.server.HiveServer2$StartOptionExecutor.execute(HiveServer2.java:1305)
> [hive-service-3.1.2-mdh3.1.2.0-SNAPSHOT.jar:3.1.2-mdh3.1.2.0-SNAPSHOT]
> at
> org.apache.hive.service.server.HiveServer2.main(HiveServer2.java:1149)
> [hive-service-3.1.2-mdh3.1.2.0-SNAPSHOT.jar:3.1.2-mdh3.1.2.0-SNAPSHOT]
> Caused by: java.lang.NoClassDefFoundError:
> org/apache/tez/dag/api/SessionNotRunning
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolSession$AbstractTriggerValidator.startTriggerValidator(TezSessionPoolSession.java:74)
> ~[hive-exec-3.1.2-mdh3.1.2.0-SNAPSHOT.jar:3.1.2-mdh3.1.2.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.initTriggers(TezSessionPoolManager.java:207)
> ~[hive-exec-3.1.2-mdh3.1.2.0-SNAPSHOT.jar:3.1.2-mdh3.1.2.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.startPool(TezSessionPoolManager.java:114)
> ~[hive-exec-3.1.2-mdh3.1.2.0-SNAPSHOT.jar:3.1.2-mdh3.1.2.0-SNAPSHOT]
> at
> org.apache.hive.service.server.HiveServer2.initAndStartTezSessionPoolManager(HiveServer2.java:839)
> ~[hive-service-3.1.2-mdh3.1.2.0-SNAPSHOT.jar:3.1.2-mdh3.1.2.0-SNAPSHOT]
> at
> org.apache.hive.service.server.HiveServer2.startOrReconnectTezSessions(HiveServer2.java:822)
> ~[hive-service-3.1.2-mdh3.1.2.0-SNAPSHOT.jar:3.1.2-mdh3.1.2.0-SNAPSHOT]
> at
> org.apache.hive.service.server.HiveServer2.start(HiveServer2.java:745)
> ~[hive-service-3.1.2-mdh3.1.2.0-SNAPSHOT.jar:3.1.2-mdh3.1.2.0-SNAPSHOT]
> at
> org.apache.hive.service.server.HiveServer2.startHiveServer2(HiveServer2.java:1037)
> ~[hive-service-3.1.2-mdh3.1.2.0-SNAPSHOT.jar:3.1.2-mdh3.1.2.0-SNAPSHOT]
> ... 3 more
> Caused by: java.lang.ClassNotFoundException:
> org.apache.tez.dag.api.SessionNotRunning
> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> ~[?:1.8.0_202]
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> ~[?:1.8.0_202]
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
> ~[?:1.8.0_202]
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ~[?:1.8.0_202]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolSession$AbstractTriggerValidator.startTriggerValidator(TezSessionPoolSession.java:74)
> ~[hive-exec-3.1.2-mdh3.1.2.0-SNAPSHOT.jar:3.1.2-mdh3.1.2.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.initTriggers(TezSessionPoolManager.java:207)
> ~[hive-exec-3.1.2-mdh3.1.2.0-SNAPSHOT.jar:3.1.2-mdh3.1.2.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.startPool(TezSessionPoolManager.java:114)
> ~[hive-exec-3.1.2-mdh3.1.2.0-SNAPSHOT.jar:3.1.2-mdh3.1.2.0-SNAPSHOT]
> at
> org.apache.hive.service.server.HiveServer2.initAndStartTezSessionPoolManager(HiveServer2.java:839)
> ~[hive-service-3.1.2-mdh3.1.2.0-SNAPSHOT.jar:3.1.2-mdh3.1.2.0-SNAPSHOT]
> at
> org.apache.hive.service.server.HiveServer2.startOrReconnectTezSessions(HiveServer2.java:822)
> ~[hive-service-3.1.2-mdh3.1.2.0-SNAPSHOT.jar:3.1.2-mdh3.1.2.0-SNAPSHOT]
> at
> org.apache.hive.service.server.HiveServer2.start(HiveServer2.java:745)
> ~[hive-service-3.1.2-mdh3.1.2.0-SNAPSHOT.jar:3.1.2-mdh3.1.2.0-SNAPSHOT]
> at
> org.apache.hive.service.server.HiveServer2.startHiveServer2(HiveServer2.java:1037)
> ~[hive-service-3.1.2-mdh3.1.2.0-SNAPSHOT.jar:3.1.2-mdh3.1.2.0-SNAPSHOT]
> ... 3 more
> {code}
> In our use case, TEZ engine is not used. Maybe we can add a conf to disable
> tez execution enigine.
>
--
This message was sent by Atlassian Jira
[jira] [Commented] (HIVE-22832) Parallelise mm directory cleaning process
[
https://issues.apache.org/jira/browse/HIVE-22832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17031227#comment-17031227
]
Hive QA commented on HIVE-22832:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12992699/HIVE-22832.1.patch
{color:green}SUCCESS:{color} +1 due to 19 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17965 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20486/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20486/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20486/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12992699 - PreCommit-HIVE-Build
> Parallelise mm directory cleaning process
> -
>
> Key: HIVE-22832
> URL: https://issues.apache.org/jira/browse/HIVE-22832
> Project: Hive
> Issue Type: Improvement
>Reporter: Marton Bod
>Assignee: Marton Bod
>Priority: Major
> Attachments: HIVE-22832.1.patch
>
>
> Inside Utilities::handleMmTableFinalPath, the cleanMmDirectories method is
> called sequentially for each element of the mmDirectory list, which might
> have a large number of elements depending on how many partitions were
> written. This current sequential execution could be improved by parallelising
> the clean up process.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22832) Parallelise mm directory cleaning process
[
https://issues.apache.org/jira/browse/HIVE-22832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17031205#comment-17031205
]
Hive QA commented on HIVE-22832:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
35s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
49s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
40s{color} | {color:blue} ql in master has 1533 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
28s{color} | {color:blue} hcatalog/streaming in master has 11 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
26s{color} | {color:blue} streaming in master has 2 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
41s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
47s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
23s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
3s{color} | {color:red} ql: The patch generated 48 new + 2704 unchanged - 27
fixed = 2752 total (was 2731) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
19s{color} | {color:red} itests/hive-unit: The patch generated 22 new + 177
unchanged - 21 fixed = 199 total (was 198) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
45s{color} | {color:red} ql generated 1 new + 1532 unchanged - 1 fixed = 1533
total (was 1533) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
46s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 37m 2s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Dead store to filename in
org.apache.hadoop.hive.ql.io.AcidUtils.parseBucketId(Path) At
AcidUtils.java:org.apache.hadoop.hive.ql.io.AcidUtils.parseBucketId(Path) At
AcidUtils.java:[line 434] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20486/dev-support/hive-personality.sh
|
| git revision | master / dabfa86 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20486/yetus/diff-checkstyle-ql.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20486/yetus/diff-checkstyle-itests_hive-unit.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20486/yetus/new-findbugs-ql.html
|
| modules | C: ql hcatalog/streaming streaming itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20486/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically
[jira] [Comment Edited] (HIVE-22771) Partition location incorrectly formed in FileOutputCommitterContainer
[
https://issues.apache.org/jira/browse/HIVE-22771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17031198#comment-17031198
]
Shivam edited comment on HIVE-22771 at 2/6/20 2:21 AM:
---
I also agree with Daniel's point, that scientific notation is surprising.
However, I am unable to understand the benefit we will get by making the id a
random String when compared with a random numeric.
Am I missing something?
was (Author: shivam-mohan):
I also agree with Daniel's point, that scientific notation is surprising.
However, I am unable to understand the benefit we will get by making the id a
random String from a random numeric.
Am I missing something?
> Partition location incorrectly formed in FileOutputCommitterContainer
> -
>
> Key: HIVE-22771
> URL: https://issues.apache.org/jira/browse/HIVE-22771
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
>Affects Versions: 1.2.1
>Reporter: Shivam
>Assignee: Shivam
>Priority: Critical
> Labels: pull-request-available
> Attachments: HIVE-22771.2.patch, HIVE-22771.3.patch, HIVE-22771.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Class _HCatOutputFormat_ in package _org.apache.hive.hcatalog.mapreduce_ uses
> function _setOutput_ to generate _idHash_ using below statement:
> *+In file org/apache/hive/hcatalog/mapreduce/HCatOutputFormat.java+*
> *line 116: idHash = String.valueOf(Math.random());*
> The output of idHash can be similar to values like this : 7.145347157239135E-4
>
> And, in class _FileOutputCommitterContainer_ in package
> _org.apache.hive.hcatalog.mapreduce;_
> Uses below statement to compute final partition path:
> +*In org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java*+
> *line 366: String finalLocn = jobLocation.replaceAll(Path.SEPARATOR +
> SCRATCH_DIR_NAME + "{color:#ff}\\d\\.?
> d+"{color},"");*
> *line 367: partPath = new Path(finalLocn);*
>
> Regex used here is incorrect, since it will only remove integers after the
> *SCRATCH_DIR_NAME,* and hence will append 'E-4' (for the above example) in
> the final partition location.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22771) Partition location incorrectly formed in FileOutputCommitterContainer
[
https://issues.apache.org/jira/browse/HIVE-22771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17031198#comment-17031198
]
Shivam commented on HIVE-22771:
---
I also agree with Daniel's point, that scientific notation is surprising.
However, I am unable to understand the benefit we will get by making the id a
random String from a random numeric.
Am I missing something?
> Partition location incorrectly formed in FileOutputCommitterContainer
> -
>
> Key: HIVE-22771
> URL: https://issues.apache.org/jira/browse/HIVE-22771
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
>Affects Versions: 1.2.1
>Reporter: Shivam
>Assignee: Shivam
>Priority: Critical
> Labels: pull-request-available
> Attachments: HIVE-22771.2.patch, HIVE-22771.3.patch, HIVE-22771.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Class _HCatOutputFormat_ in package _org.apache.hive.hcatalog.mapreduce_ uses
> function _setOutput_ to generate _idHash_ using below statement:
> *+In file org/apache/hive/hcatalog/mapreduce/HCatOutputFormat.java+*
> *line 116: idHash = String.valueOf(Math.random());*
> The output of idHash can be similar to values like this : 7.145347157239135E-4
>
> And, in class _FileOutputCommitterContainer_ in package
> _org.apache.hive.hcatalog.mapreduce;_
> Uses below statement to compute final partition path:
> +*In org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java*+
> *line 366: String finalLocn = jobLocation.replaceAll(Path.SEPARATOR +
> SCRATCH_DIR_NAME + "{color:#ff}\\d\\.?
> d+"{color},"");*
> *line 367: partPath = new Path(finalLocn);*
>
> Regex used here is incorrect, since it will only remove integers after the
> *SCRATCH_DIR_NAME,* and hence will append 'E-4' (for the above example) in
> the final partition location.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22736) Support replication across multiple encryption zones
[
https://issues.apache.org/jira/browse/HIVE-22736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17031185#comment-17031185
]
Hive QA commented on HIVE-22736:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12992697/HIVE-22736.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20485/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20485/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20485/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12992697/HIVE-22736.patch was
found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12992697 - PreCommit-HIVE-Build
> Support replication across multiple encryption zones
>
>
> Key: HIVE-22736
> URL: https://issues.apache.org/jira/browse/HIVE-22736
> Project: Hive
> Issue Type: Task
>Reporter: Aasha Medhi
>Assignee: Aasha Medhi
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch,
> HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch,
> HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch,
> HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch,
> HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch,
> HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch,
> HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch
>
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22825) Reduce directory lookup cost for acid tables
[
https://issues.apache.org/jira/browse/HIVE-22825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17031184#comment-17031184
]
Hive QA commented on HIVE-22825:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12992693/HIVE-22825.3.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17965 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20484/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20484/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20484/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12992693 - PreCommit-HIVE-Build
> Reduce directory lookup cost for acid tables
>
>
> Key: HIVE-22825
> URL: https://issues.apache.org/jira/browse/HIVE-22825
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Reporter: Rajesh Balamohan
>Assignee: Rajesh Balamohan
>Priority: Minor
> Attachments: HIVE-22825.1.patch, HIVE-22825.2.patch,
> HIVE-22825.3.patch
>
>
> With objectstores, directory lookup costs are expensive. For acid tables, it
> would be good to have a directory cache to reduce number of lookup calls.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22825) Reduce directory lookup cost for acid tables
[
https://issues.apache.org/jira/browse/HIVE-22825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17031158#comment-17031158
]
Hive QA commented on HIVE-22825:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
54s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
34s{color} | {color:blue} common in master has 63 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
35s{color} | {color:blue} ql in master has 1533 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
42s{color} | {color:red} ql: The patch generated 3 new + 315 unchanged - 0
fixed = 318 total (was 315) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 28m 12s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20484/dev-support/hive-personality.sh
|
| git revision | master / dabfa86 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20484/yetus/diff-checkstyle-ql.txt
|
| modules | C: common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20484/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Reduce directory lookup cost for acid tables
>
>
> Key: HIVE-22825
> URL: https://issues.apache.org/jira/browse/HIVE-22825
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Reporter: Rajesh Balamohan
>Assignee: Rajesh Balamohan
>Priority: Minor
> Attachments: HIVE-22825.1.patch, HIVE-22825.2.patch,
> HIVE-22825.3.patch
>
>
> With objectstores, directory lookup costs are expensive. For acid tables, it
> would be good to have a directory cache to reduce number of lookup calls.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-21216) Write Parquet INT64 timestamp
[
https://issues.apache.org/jira/browse/HIVE-21216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17031141#comment-17031141
]
Hive QA commented on HIVE-21216:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12992692/HIVE-21216.2.patch
{color:green}SUCCESS:{color} +1 due to 6 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 17971 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_int64_timestamp]
(batchId=53)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_int64_timestamp_int96_compatibility]
(batchId=3)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20483/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20483/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20483/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12992692 - PreCommit-HIVE-Build
> Write Parquet INT64 timestamp
> -
>
> Key: HIVE-21216
> URL: https://issues.apache.org/jira/browse/HIVE-21216
> Project: Hive
> Issue Type: New Feature
> Components: Hive
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Major
> Labels: breaking_change
> Attachments: HIVE-21216.1.patch, HIVE-21216.2.patch
>
>
> This patch enables Hive to start writing int64 timestamps in Parquet.
> With Parquet version 1.11, a new timestamp LogicalType with base INT64 and
> the following metadata is introduced:
> boolean isAdjustedToUtc: marks whether the timestamp is converted to UTC
> (aka Instant semantics) or not (LocalDateTime semantics)
> enum TimeUnit (NANOS, MICROS, MILLIS): granularity of timestamp
> The timestamp will have LocalDateTime semantics (not converted to UTC).
> Timestamps outside of range 1677-09-21 00:12:43.145224192 – 2262-04-11
> 23:47:16.854775807 cannot be written in nanos and will be recorded as NULL.
> Time unit (granularity) will be determined by the user. Default is
> milliseconds.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22808) HiveRelFieldTrimmer does not handle HiveTableFunctionScan
[
https://issues.apache.org/jira/browse/HIVE-22808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-22808:
---
Fix Version/s: 4.0.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
Pushed to master, thanks [~kkasa]
> HiveRelFieldTrimmer does not handle HiveTableFunctionScan
> -
>
> Key: HIVE-22808
> URL: https://issues.apache.org/jira/browse/HIVE-22808
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22808.1.patch, HIVE-22808.2.patch,
> HIVE-22808.2.patch, HIVE-22808.3.patch, HIVE-22808.4.patch, HIVE-22808.5.patch
>
>
> *Repro*
> {code:java}
> CREATE TABLE table_16 (
> timestamp_col_19timestamp,
> timestamp_col_29timestamp,
> int_col_27 int,
> int_col_39 int,
> boolean_col_18 boolean,
> varchar0045_col_23 varchar(45)
> );
> CREATE TABLE table_7 (
> int_col_10 int,
> bigint_col_3bigint
> );
> CREATE TABLE table_10 (
> boolean_col_8 boolean,
> boolean_col_16 boolean,
> timestamp_col_5 timestamp,
> timestamp_col_15timestamp,
> timestamp_col_30timestamp,
> decimal3825_col_26 decimal(38, 25),
> smallint_col_9 smallint,
> int_col_18 int
> );
> explain cbo
> SELECT
> DISTINCT COALESCE(a4.timestamp_col_15, IF(a4.boolean_col_16,
> a4.timestamp_col_30, a4.timestamp_col_5)) AS timestamp_col
> FROM table_7 a3
> RIGHT JOIN table_10 a4
> WHERE (a3.bigint_col_3) >= (a4.int_col_18)
> INTERSECT ALL
> SELECT
> COALESCE(LEAST(
> COALESCE(a1.timestamp_col_19, CAST('2010-03-29 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2014-08-16 00:00:00' AS
> TIMESTAMP))
> ),
> GREATEST(COALESCE(a1.timestamp_col_19, CAST('2013-07-01 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2028-06-18 00:00:00' AS
> TIMESTAMP)))
> ) AS timestamp_col
> FROM table_16 a1
> GROUP BY COALESCE(LEAST(
> COALESCE(a1.timestamp_col_19, CAST('2010-03-29 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2014-08-16 00:00:00' AS
> TIMESTAMP))
> ),
> GREATEST(
> COALESCE(a1.timestamp_col_19, CAST('2013-07-01 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2028-06-18 00:00:00' AS
> TIMESTAMP)))
> );
> {code}
> CBO Plan contains unnecessary columns or all columns from a table in
> projections like:
> {code:java}
> HiveProject(int_col_10=[$0], bigint_col_3=[$1],
> BLOCK__OFFSET__INSIDE__FILE=[$2], INPUT__FILE__NAME=[$3],
> CAST=[CAST($4):RecordType(BIGINT writeid, INTEGER bucketid, BIGINT rowid)])
> {code}
> *Cause*
> The plan contains a HiveTableFunctionScan operator:
> {code:java}
> HiveTableFunctionScan(invocation=[replicate_rows($0, $1)],
> rowType=[RecordType(BIGINT $f0, TIMESTAMP(9) $f1)])
> {code}
> HiveTableFunctionScan is not handled by HiveRelFieldTrimmer nor
> RelFieldTrimmer which suppose to remove unused columns in the
> CalcitePlanner.applyPreJoinOrderingTransforms(...) phase. The whole subtree
> rooted from HiveTableFunctionScan is ignored.
> Whole plan:
> {code:java}
> CBO PLAN:
> HiveProject($f0=[$1])
> HiveTableFunctionScan(invocation=[replicate_rows($0, $1)],
> rowType=[RecordType(BIGINT $f0, TIMESTAMP(9) $f1)])
> HiveProject($f0=[$2], $f1=[$0])
> HiveFilter(condition=[=($1, 2)])
> HiveAggregate(group=[{0}], agg#0=[count($1)], agg#1=[min($1)])
> HiveProject($f0=[$0], $f1=[$1])
> HiveUnion(all=[true])
> HiveProject($f0=[$0], $f1=[$1])
> HiveAggregate(group=[{0}], agg#0=[count()])
> HiveProject($f0=[$0])
> HiveAggregate(group=[{0}])
> HiveProject($f0=[CASE(IS NOT NULL($7), $7, if($5, $8,
> $6))])
> HiveJoin(condition=[>=($1, $13)], joinType=[inner],
> algorithm=[none], cost=[not available])
> HiveProject(int_col_10=[$0], bigint_col_3=[$1],
> BLOCK__OFFSET__INSIDE__FILE=[$2], INPUT__FILE__NAME=[$3],
> CAST=[CAST($4):RecordType(BIGINT writeid, INTEGER bucketid, BIGINT rowid)])
> HiveFilter(condition=[IS NOT NULL($1)])
> HiveTableScan(table=[[default, table_7]],
> table:alias=[a3])
> HiveProject(boolean_col_16=[$0],
> timestamp_col_5=[$1], timestamp_col_15=[$2], timestamp_col_30=[$3],
> int_col_18=[$4], BLOCK__OFFSET__INSIDE__FILE=[$5], INPUT__FILE__NAME=[$6],
> ROW__ID=[$7], CAST=[CAST($4):BIGINT])
> HiveFilter(condition=[IS NOT
> NULL(CAST($4):BIGINT)])
>
[jira] [Commented] (HIVE-22808) HiveRelFieldTrimmer does not handle HiveTableFunctionScan
[
https://issues.apache.org/jira/browse/HIVE-22808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17031137#comment-17031137
]
Vineet Garg commented on HIVE-22808:
+1
> HiveRelFieldTrimmer does not handle HiveTableFunctionScan
> -
>
> Key: HIVE-22808
> URL: https://issues.apache.org/jira/browse/HIVE-22808
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22808.1.patch, HIVE-22808.2.patch,
> HIVE-22808.2.patch, HIVE-22808.3.patch, HIVE-22808.4.patch, HIVE-22808.5.patch
>
>
> *Repro*
> {code:java}
> CREATE TABLE table_16 (
> timestamp_col_19timestamp,
> timestamp_col_29timestamp,
> int_col_27 int,
> int_col_39 int,
> boolean_col_18 boolean,
> varchar0045_col_23 varchar(45)
> );
> CREATE TABLE table_7 (
> int_col_10 int,
> bigint_col_3bigint
> );
> CREATE TABLE table_10 (
> boolean_col_8 boolean,
> boolean_col_16 boolean,
> timestamp_col_5 timestamp,
> timestamp_col_15timestamp,
> timestamp_col_30timestamp,
> decimal3825_col_26 decimal(38, 25),
> smallint_col_9 smallint,
> int_col_18 int
> );
> explain cbo
> SELECT
> DISTINCT COALESCE(a4.timestamp_col_15, IF(a4.boolean_col_16,
> a4.timestamp_col_30, a4.timestamp_col_5)) AS timestamp_col
> FROM table_7 a3
> RIGHT JOIN table_10 a4
> WHERE (a3.bigint_col_3) >= (a4.int_col_18)
> INTERSECT ALL
> SELECT
> COALESCE(LEAST(
> COALESCE(a1.timestamp_col_19, CAST('2010-03-29 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2014-08-16 00:00:00' AS
> TIMESTAMP))
> ),
> GREATEST(COALESCE(a1.timestamp_col_19, CAST('2013-07-01 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2028-06-18 00:00:00' AS
> TIMESTAMP)))
> ) AS timestamp_col
> FROM table_16 a1
> GROUP BY COALESCE(LEAST(
> COALESCE(a1.timestamp_col_19, CAST('2010-03-29 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2014-08-16 00:00:00' AS
> TIMESTAMP))
> ),
> GREATEST(
> COALESCE(a1.timestamp_col_19, CAST('2013-07-01 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2028-06-18 00:00:00' AS
> TIMESTAMP)))
> );
> {code}
> CBO Plan contains unnecessary columns or all columns from a table in
> projections like:
> {code:java}
> HiveProject(int_col_10=[$0], bigint_col_3=[$1],
> BLOCK__OFFSET__INSIDE__FILE=[$2], INPUT__FILE__NAME=[$3],
> CAST=[CAST($4):RecordType(BIGINT writeid, INTEGER bucketid, BIGINT rowid)])
> {code}
> *Cause*
> The plan contains a HiveTableFunctionScan operator:
> {code:java}
> HiveTableFunctionScan(invocation=[replicate_rows($0, $1)],
> rowType=[RecordType(BIGINT $f0, TIMESTAMP(9) $f1)])
> {code}
> HiveTableFunctionScan is not handled by HiveRelFieldTrimmer nor
> RelFieldTrimmer which suppose to remove unused columns in the
> CalcitePlanner.applyPreJoinOrderingTransforms(...) phase. The whole subtree
> rooted from HiveTableFunctionScan is ignored.
> Whole plan:
> {code:java}
> CBO PLAN:
> HiveProject($f0=[$1])
> HiveTableFunctionScan(invocation=[replicate_rows($0, $1)],
> rowType=[RecordType(BIGINT $f0, TIMESTAMP(9) $f1)])
> HiveProject($f0=[$2], $f1=[$0])
> HiveFilter(condition=[=($1, 2)])
> HiveAggregate(group=[{0}], agg#0=[count($1)], agg#1=[min($1)])
> HiveProject($f0=[$0], $f1=[$1])
> HiveUnion(all=[true])
> HiveProject($f0=[$0], $f1=[$1])
> HiveAggregate(group=[{0}], agg#0=[count()])
> HiveProject($f0=[$0])
> HiveAggregate(group=[{0}])
> HiveProject($f0=[CASE(IS NOT NULL($7), $7, if($5, $8,
> $6))])
> HiveJoin(condition=[>=($1, $13)], joinType=[inner],
> algorithm=[none], cost=[not available])
> HiveProject(int_col_10=[$0], bigint_col_3=[$1],
> BLOCK__OFFSET__INSIDE__FILE=[$2], INPUT__FILE__NAME=[$3],
> CAST=[CAST($4):RecordType(BIGINT writeid, INTEGER bucketid, BIGINT rowid)])
> HiveFilter(condition=[IS NOT NULL($1)])
> HiveTableScan(table=[[default, table_7]],
> table:alias=[a3])
> HiveProject(boolean_col_16=[$0],
> timestamp_col_5=[$1], timestamp_col_15=[$2], timestamp_col_30=[$3],
> int_col_18=[$4], BLOCK__OFFSET__INSIDE__FILE=[$5], INPUT__FILE__NAME=[$6],
> ROW__ID=[$7], CAST=[CAST($4):BIGINT])
> HiveFilter(condition=[IS NOT
> NULL(CAST($4):BIGINT)])
> HiveTableScan(table=[[default, table_10]],
> table:alias=[a4])
>
[jira] [Commented] (HIVE-21216) Write Parquet INT64 timestamp
[
https://issues.apache.org/jira/browse/HIVE-21216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17031124#comment-17031124
]
Hive QA commented on HIVE-21216:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
44s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
51s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
32s{color} | {color:blue} common in master has 63 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
44s{color} | {color:blue} ql in master has 1533 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
16s{color} | {color:red} common: The patch generated 1 new + 370 unchanged - 0
fixed = 371 total (was 370) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
41s{color} | {color:red} ql: The patch generated 10 new + 206 unchanged - 2
fixed = 216 total (was 208) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 6 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 28m 17s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20483/dev-support/hive-personality.sh
|
| git revision | master / f04323f |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20483/yetus/diff-checkstyle-common.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20483/yetus/diff-checkstyle-ql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20483/yetus/whitespace-eol.txt
|
| modules | C: common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20483/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Write Parquet INT64 timestamp
> -
>
> Key: HIVE-21216
> URL: https://issues.apache.org/jira/browse/HIVE-21216
> Project: Hive
> Issue Type: New Feature
> Components: Hive
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Major
> Labels: breaking_change
> Attachments: HIVE-21216.1.patch, HIVE-21216.2.patch
>
>
> This patch enables Hive to start writing int64 timestamps in Parquet.
> With Parquet version 1.11, a new timestamp LogicalType with base INT64 and
>
[jira] [Updated] (HIVE-22828) Decimal64: NVL & CASE statements implicitly convert decimal64 to 128
[
https://issues.apache.org/jira/browse/HIVE-22828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ramesh Kumar Thangarajan updated HIVE-22828:
Attachment: HIVE-22828.1.patch
Status: Patch Available (was: Open)
> Decimal64: NVL & CASE statements implicitly convert decimal64 to 128
> -
>
> Key: HIVE-22828
> URL: https://issues.apache.org/jira/browse/HIVE-22828
> Project: Hive
> Issue Type: Bug
> Components: Vectorization
>Reporter: Gopal Vijayaraghavan
>Assignee: Ramesh Kumar Thangarajan
>Priority: Major
> Attachments: HIVE-22828.1.patch
>
>
> {code}
> select
> sum(case when (ss_item_sk=1) then ss_sales_price else null end),
> sum(case when (ss_item_sk=2) then ss_sales_price else ss_sales_price+1 end),
> sum(case when (ss_item_sk=2) then 1.0BD+ss_sales_price else null end)
> from store_sales where ss_sold_date_sk % = 1;
> {code}
> {code}
> Caused by: java.lang.ClassCastException:
> org.apache.hadoop.hive.ql.exec.vector.Decimal64ColumnVector cannot be cast to
> org.apache.hadoop.hive.ql.exec.vector.DecimalColumnVector
> at
> org.apache.hadoop.hive.ql.exec.vector.DecimalColumnVector.setElement(DecimalColumnVector.java:130)
> at
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprColumnNull.evaluate(IfExprColumnNull.java:125)
> at
> org.apache.hadoop.hive.ql.exec.vector.expressions.aggregates.VectorUDAFSumDecimal.aggregateInputSelection(VectorUDAFSumDecimal.java:113)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator$ProcessingModeBase.processAggregators(VectorGroupByOperator.java:221)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator$ProcessingModeHashAggregate.doProcessBatch(VectorGroupByOperator.java:414)
> {code}
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizationContext.java#L3950
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22808) HiveRelFieldTrimmer does not handle HiveTableFunctionScan
[
https://issues.apache.org/jira/browse/HIVE-22808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17031096#comment-17031096
]
Hive QA commented on HIVE-22808:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12992688/HIVE-22808.5.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17965 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20482/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20482/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20482/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12992688 - PreCommit-HIVE-Build
> HiveRelFieldTrimmer does not handle HiveTableFunctionScan
> -
>
> Key: HIVE-22808
> URL: https://issues.apache.org/jira/browse/HIVE-22808
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22808.1.patch, HIVE-22808.2.patch,
> HIVE-22808.2.patch, HIVE-22808.3.patch, HIVE-22808.4.patch, HIVE-22808.5.patch
>
>
> *Repro*
> {code:java}
> CREATE TABLE table_16 (
> timestamp_col_19timestamp,
> timestamp_col_29timestamp,
> int_col_27 int,
> int_col_39 int,
> boolean_col_18 boolean,
> varchar0045_col_23 varchar(45)
> );
> CREATE TABLE table_7 (
> int_col_10 int,
> bigint_col_3bigint
> );
> CREATE TABLE table_10 (
> boolean_col_8 boolean,
> boolean_col_16 boolean,
> timestamp_col_5 timestamp,
> timestamp_col_15timestamp,
> timestamp_col_30timestamp,
> decimal3825_col_26 decimal(38, 25),
> smallint_col_9 smallint,
> int_col_18 int
> );
> explain cbo
> SELECT
> DISTINCT COALESCE(a4.timestamp_col_15, IF(a4.boolean_col_16,
> a4.timestamp_col_30, a4.timestamp_col_5)) AS timestamp_col
> FROM table_7 a3
> RIGHT JOIN table_10 a4
> WHERE (a3.bigint_col_3) >= (a4.int_col_18)
> INTERSECT ALL
> SELECT
> COALESCE(LEAST(
> COALESCE(a1.timestamp_col_19, CAST('2010-03-29 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2014-08-16 00:00:00' AS
> TIMESTAMP))
> ),
> GREATEST(COALESCE(a1.timestamp_col_19, CAST('2013-07-01 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2028-06-18 00:00:00' AS
> TIMESTAMP)))
> ) AS timestamp_col
> FROM table_16 a1
> GROUP BY COALESCE(LEAST(
> COALESCE(a1.timestamp_col_19, CAST('2010-03-29 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2014-08-16 00:00:00' AS
> TIMESTAMP))
> ),
> GREATEST(
> COALESCE(a1.timestamp_col_19, CAST('2013-07-01 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2028-06-18 00:00:00' AS
> TIMESTAMP)))
> );
> {code}
> CBO Plan contains unnecessary columns or all columns from a table in
> projections like:
> {code:java}
> HiveProject(int_col_10=[$0], bigint_col_3=[$1],
> BLOCK__OFFSET__INSIDE__FILE=[$2], INPUT__FILE__NAME=[$3],
> CAST=[CAST($4):RecordType(BIGINT writeid, INTEGER bucketid, BIGINT rowid)])
> {code}
> *Cause*
> The plan contains a HiveTableFunctionScan operator:
> {code:java}
> HiveTableFunctionScan(invocation=[replicate_rows($0, $1)],
> rowType=[RecordType(BIGINT $f0, TIMESTAMP(9) $f1)])
> {code}
> HiveTableFunctionScan is not handled by HiveRelFieldTrimmer nor
> RelFieldTrimmer which suppose to remove unused columns in the
> CalcitePlanner.applyPreJoinOrderingTransforms(...) phase. The whole subtree
> rooted from HiveTableFunctionScan is ignored.
> Whole plan:
> {code:java}
> CBO PLAN:
> HiveProject($f0=[$1])
> HiveTableFunctionScan(invocation=[replicate_rows($0, $1)],
> rowType=[RecordType(BIGINT $f0, TIMESTAMP(9) $f1)])
> HiveProject($f0=[$2], $f1=[$0])
> HiveFilter(condition=[=($1, 2)])
> HiveAggregate(group=[{0}], agg#0=[count($1)], agg#1=[min($1)])
> HiveProject($f0=[$0], $f1=[$1])
> HiveUnion(all=[true])
> HiveProject($f0=[$0], $f1=[$1])
> HiveAggregate(group=[{0}], agg#0=[count()])
> HiveProject($f0=[$0])
> HiveAggregate(group=[{0}])
> HiveProject($f0=[CASE(IS NOT NULL($7), $7, if($5, $8,
> $6))])
> HiveJoin(condition=[>=($1, $13)], joinType=[inner],
>
[jira] [Commented] (HIVE-22808) HiveRelFieldTrimmer does not handle HiveTableFunctionScan
[
https://issues.apache.org/jira/browse/HIVE-22808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17031063#comment-17031063
]
Hive QA commented on HIVE-22808:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
41s{color} | {color:blue} ql in master has 1533 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 15s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20482/dev-support/hive-personality.sh
|
| git revision | master / f04323f |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20482/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> HiveRelFieldTrimmer does not handle HiveTableFunctionScan
> -
>
> Key: HIVE-22808
> URL: https://issues.apache.org/jira/browse/HIVE-22808
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22808.1.patch, HIVE-22808.2.patch,
> HIVE-22808.2.patch, HIVE-22808.3.patch, HIVE-22808.4.patch, HIVE-22808.5.patch
>
>
> *Repro*
> {code:java}
> CREATE TABLE table_16 (
> timestamp_col_19timestamp,
> timestamp_col_29timestamp,
> int_col_27 int,
> int_col_39 int,
> boolean_col_18 boolean,
> varchar0045_col_23 varchar(45)
> );
> CREATE TABLE table_7 (
> int_col_10 int,
> bigint_col_3bigint
> );
> CREATE TABLE table_10 (
> boolean_col_8 boolean,
> boolean_col_16 boolean,
> timestamp_col_5 timestamp,
> timestamp_col_15timestamp,
> timestamp_col_30timestamp,
> decimal3825_col_26 decimal(38, 25),
> smallint_col_9 smallint,
> int_col_18 int
> );
> explain cbo
> SELECT
> DISTINCT COALESCE(a4.timestamp_col_15, IF(a4.boolean_col_16,
> a4.timestamp_col_30, a4.timestamp_col_5)) AS timestamp_col
> FROM table_7 a3
> RIGHT JOIN table_10 a4
> WHERE (a3.bigint_col_3) >= (a4.int_col_18)
> INTERSECT ALL
> SELECT
> COALESCE(LEAST(
> COALESCE(a1.timestamp_col_19, CAST('2010-03-29 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2014-08-16 00:00:00' AS
> TIMESTAMP))
> ),
> GREATEST(COALESCE(a1.timestamp_col_19, CAST('2013-07-01
[jira] [Updated] (HIVE-22824) JoinProjectTranspose rule should skip Projects containing windowing expression
[ https://issues.apache.org/jira/browse/HIVE-22824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-22824: --- Status: Patch Available (was: Open) > JoinProjectTranspose rule should skip Projects containing windowing expression > -- > > Key: HIVE-22824 > URL: https://issues.apache.org/jira/browse/HIVE-22824 > Project: Hive > Issue Type: Bug > Components: Query Planning >Affects Versions: 4.0.0 >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-22824.1.patch, HIVE-22824.2.patch > > > Otherwise this rule could end up creating plan with windowing expression > within join condition which hive doesn't know how to process. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22824) JoinProjectTranspose rule should skip Projects containing windowing expression
[ https://issues.apache.org/jira/browse/HIVE-22824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-22824: --- Attachment: HIVE-22824.2.patch > JoinProjectTranspose rule should skip Projects containing windowing expression > -- > > Key: HIVE-22824 > URL: https://issues.apache.org/jira/browse/HIVE-22824 > Project: Hive > Issue Type: Bug > Components: Query Planning >Affects Versions: 4.0.0 >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-22824.1.patch, HIVE-22824.2.patch > > > Otherwise this rule could end up creating plan with windowing expression > within join condition which hive doesn't know how to process. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22824) JoinProjectTranspose rule should skip Projects containing windowing expression
[ https://issues.apache.org/jira/browse/HIVE-22824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-22824: --- Status: Open (was: Patch Available) > JoinProjectTranspose rule should skip Projects containing windowing expression > -- > > Key: HIVE-22824 > URL: https://issues.apache.org/jira/browse/HIVE-22824 > Project: Hive > Issue Type: Bug > Components: Query Planning >Affects Versions: 4.0.0 >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-22824.1.patch, HIVE-22824.2.patch > > > Otherwise this rule could end up creating plan with windowing expression > within join condition which hive doesn't know how to process. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-10362) Support Type check/conversion in dynamic partition column
[
https://issues.apache.org/jira/browse/HIVE-10362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17031047#comment-17031047
]
Hive QA commented on HIVE-10362:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12992686/HIVE-10362.10.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 19 failed/errored test(s), 17966 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dynpart_sort_optimization_acid2]
(batchId=36)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_sortmerge_join_16]
(batchId=180)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dynpart_sort_opt_bucketing]
(batchId=190)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dynpart_sort_optimization_acid]
(batchId=177)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_smb]
(batchId=191)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[auto_sortmerge_join_16]
(batchId=198)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[auto_sortmerge_join_16]
(batchId=141)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testCancelRenewTokenFlow
(batchId=304)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testConnection
(batchId=304)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testIsValid (batchId=304)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testIsValidNeg
(batchId=304)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testNegativeProxyAuth
(batchId=304)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testNegativeTokenAuth
(batchId=304)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testNoKrbSASLTokenAuthNeg
(batchId=304)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testNonKrbSASLAuth
(batchId=304)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testNonKrbSASLFullNameAuth
(batchId=304)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testProxyAuth
(batchId=304)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testRenewDelegationToken
(batchId=304)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testTokenAuth
(batchId=304)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20481/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20481/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20481/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 19 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12992686 - PreCommit-HIVE-Build
> Support Type check/conversion in dynamic partition column
> -
>
> Key: HIVE-10362
> URL: https://issues.apache.org/jira/browse/HIVE-10362
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor, Types
>Affects Versions: 1.0.0, 3.0.0, 4.0.0
>Reporter: Chaoyu Tang
>Assignee: Karen Coppage
>Priority: Major
> Labels: breaking_change
> Fix For: 4.0.0
>
> Attachments: HIVE-10362.01.patch, HIVE-10362.02.patch,
> HIVE-10362.03.patch, HIVE-10362.04.patch, HIVE-10362.04.patch,
> HIVE-10362.06.patch, HIVE-10362.07.patch, HIVE-10362.07.patch,
> HIVE-10362.08.patch, HIVE-10362.09.patch, HIVE-10362.10.patch
>
>
> There are quite a lot of issues associated with the non-noramlized or
> type-mismatched values for partition column. Hive has many ways to introduce
> such problematic data. HIVE-10307 mainly provides the support to type
> check/convert/normalize the partition column value in static partition
> specification. This JIRA tries to deal with the partition column type in
> dynamic partition insert. Currently any data can be inserted as a partition
> column value as long as it is quoted as a string. For example,
> create table dynparttypechecknum (key int, value string) partitioned by (part
> int);
> insert into dynparttypechecknum partition (part) select key, value, '1'
> from src limit 1;
> show partitions dynparttypechecknum;
> --
> part=1
> The partition column value is non-normalized int 1. It causes some
> unnecessary problems such as integer partition column JDO filter pushdown
> (see HIVE-6052) and others like HIVE-10210.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-21569) Bump guava version to 28.1-jre
[ https://issues.apache.org/jira/browse/HIVE-21569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor updated HIVE-21569: Status: Patch Available (was: Open) > Bump guava version to 28.1-jre > -- > > Key: HIVE-21569 > URL: https://issues.apache.org/jira/browse/HIVE-21569 > Project: Hive > Issue Type: Task > Components: Hive >Reporter: Laszlo Pinter >Assignee: László Bodor >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-21569.01.patch, HIVE-21569.02.patch, > HIVE-21569.03.patch, HIVE-21569.04.patch, HIVE-21569.05.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Changes required to bump guava to 28.1-jre -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-10362) Support Type check/conversion in dynamic partition column
[
https://issues.apache.org/jira/browse/HIVE-10362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17030997#comment-17030997
]
Hive QA commented on HIVE-10362:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
55s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
35s{color} | {color:blue} common in master has 63 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
46s{color} | {color:blue} ql in master has 1533 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
8s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
29s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch 10 line(s) with tabs. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
0s{color} | {color:red} ql generated 1 new + 1533 unchanged - 0 fixed = 1534
total (was 1533) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 29m 43s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Redundant nullcheck of destinationTable which is known to be null in
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genFileSinkPlan(String, QB,
Operator) Redundant null check at SemanticAnalyzer.java:is known to be null in
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genFileSinkPlan(String, QB,
Operator) Redundant null check at SemanticAnalyzer.java:[line 7706] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20481/dev-support/hive-personality.sh
|
| git revision | master / f04323f |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20481/yetus/whitespace-tabs.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20481/yetus/new-findbugs-ql.html
|
| modules | C: common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20481/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Support Type check/conversion in dynamic partition column
> -
>
> Key: HIVE-10362
> URL: https://issues.apache.org/jira/browse/HIVE-10362
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor, Types
>Affects Versions: 1.0.0, 3.0.0, 4.0.0
>Reporter: Chaoyu Tang
>Assignee: Karen Coppage
>Priority: Major
> Labels: breaking_change
> Fix For: 4.0.0
>
> Attachments: HIVE-10362.01.patch,
[jira] [Updated] (HIVE-21569) Bump guava version to 28.1-jre
[ https://issues.apache.org/jira/browse/HIVE-21569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor updated HIVE-21569: Attachment: HIVE-21569.05.patch > Bump guava version to 28.1-jre > -- > > Key: HIVE-21569 > URL: https://issues.apache.org/jira/browse/HIVE-21569 > Project: Hive > Issue Type: Task > Components: Hive >Reporter: Laszlo Pinter >Assignee: László Bodor >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-21569.01.patch, HIVE-21569.02.patch, > HIVE-21569.03.patch, HIVE-21569.04.patch, HIVE-21569.05.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Changes required to bump guava to 28.1-jre -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22736) Support replication across multiple encryption zones
[
https://issues.apache.org/jira/browse/HIVE-22736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17030967#comment-17030967
]
Hive QA commented on HIVE-22736:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12992697/HIVE-22736.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20480/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20480/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20480/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12992697/HIVE-22736.patch was
found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12992697 - PreCommit-HIVE-Build
> Support replication across multiple encryption zones
>
>
> Key: HIVE-22736
> URL: https://issues.apache.org/jira/browse/HIVE-22736
> Project: Hive
> Issue Type: Task
>Reporter: Aasha Medhi
>Assignee: Aasha Medhi
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch,
> HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch,
> HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch,
> HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch,
> HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch,
> HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch,
> HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch
>
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22748) Remove disabling of IdentityProjectRemover transformation for Return Path
[
https://issues.apache.org/jira/browse/HIVE-22748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17030965#comment-17030965
]
Hive QA commented on HIVE-22748:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12992683/HIVE-22748.03.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1223 failed/errored test(s), 17966 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_single_sourced_multi_insert]
(batchId=295)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[explain_outputs]
(batchId=303)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[smb_mapjoin_16]
(batchId=303)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[smb_mapjoin_7]
(batchId=303)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_into_table]
(batchId=306)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_table]
(batchId=306)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_mapjoin]
(batchId=12)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_nullscan]
(batchId=76)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_stats3] (batchId=17)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_stats4] (batchId=70)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_stats5] (batchId=24)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_table_stats]
(batchId=61)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alterColumnStatsPart]
(batchId=99)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_deep_filters]
(batchId=103)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_groupby2]
(batchId=54)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_groupby]
(batchId=56)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_join]
(batchId=61)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_join_pkfk]
(batchId=16)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_union]
(batchId=56)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_11]
(batchId=91)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_4]
(batchId=14)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_7]
(batchId=73)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_9]
(batchId=41)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join0] (batchId=101)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join10] (batchId=40)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join11] (batchId=10)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join12] (batchId=28)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join13] (batchId=92)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join14] (batchId=17)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join15] (batchId=18)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join16] (batchId=45)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join17] (batchId=94)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join18] (batchId=14)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join18_multi_distinct]
(batchId=30)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join19] (batchId=74)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join19_inclause]
(batchId=20)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join1] (batchId=89)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join20]
(batchId=102)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join22] (batchId=64)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join24] (batchId=86)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join26] (batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join27]
(batchId=103)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join2] (batchId=73)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join31] (batchId=51)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join32] (batchId=98)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join33] (batchId=13)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join3] (batchId=93)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join4] (batchId=81)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join5] (batchId=83)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join6] (batchId=99)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join7] (batchId=30)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join8]
[jira] [Updated] (HIVE-22783) Add test for HIVE-22366
[ https://issues.apache.org/jira/browse/HIVE-22783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-22783: -- Labels: pull-request-available (was: ) > Add test for HIVE-22366 > --- > > Key: HIVE-22783 > URL: https://issues.apache.org/jira/browse/HIVE-22783 > Project: Hive > Issue Type: Sub-task >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22783.1.patch, HIVE-22783.2.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-22783) Add test for HIVE-22366
[ https://issues.apache.org/jira/browse/HIVE-22783?focusedWorklogId=382462=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-382462 ] ASF GitHub Bot logged work on HIVE-22783: - Author: ASF GitHub Bot Created on: 05/Feb/20 19:22 Start Date: 05/Feb/20 19:22 Worklog Time Spent: 10m Work Description: vineetgarg02 commented on pull request #895: HIVE-22783 : Test to ensure number of metastore calls by compiler URL: https://github.com/apache/hive/pull/895 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 382462) Remaining Estimate: 0h Time Spent: 10m > Add test for HIVE-22366 > --- > > Key: HIVE-22783 > URL: https://issues.apache.org/jira/browse/HIVE-22783 > Project: Hive > Issue Type: Sub-task >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22783.1.patch, HIVE-22783.2.patch > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22783) Add test for HIVE-22366
[ https://issues.apache.org/jira/browse/HIVE-22783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-22783: --- Attachment: HIVE-22783.2.patch > Add test for HIVE-22366 > --- > > Key: HIVE-22783 > URL: https://issues.apache.org/jira/browse/HIVE-22783 > Project: Hive > Issue Type: Sub-task >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-22783.1.patch, HIVE-22783.2.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22783) Add test for HIVE-22366
[ https://issues.apache.org/jira/browse/HIVE-22783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-22783: --- Status: Open (was: Patch Available) > Add test for HIVE-22366 > --- > > Key: HIVE-22783 > URL: https://issues.apache.org/jira/browse/HIVE-22783 > Project: Hive > Issue Type: Sub-task >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-22783.1.patch, HIVE-22783.2.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22783) Add test for HIVE-22366
[ https://issues.apache.org/jira/browse/HIVE-22783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-22783: --- Status: Open (was: Patch Available) > Add test for HIVE-22366 > --- > > Key: HIVE-22783 > URL: https://issues.apache.org/jira/browse/HIVE-22783 > Project: Hive > Issue Type: Sub-task >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-22783.1.patch, HIVE-22783.2.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22783) Add test for HIVE-22366
[ https://issues.apache.org/jira/browse/HIVE-22783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-22783: --- Status: Patch Available (was: Open) > Add test for HIVE-22366 > --- > > Key: HIVE-22783 > URL: https://issues.apache.org/jira/browse/HIVE-22783 > Project: Hive > Issue Type: Sub-task >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-22783.1.patch, HIVE-22783.2.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22783) Add test for HIVE-22366
[ https://issues.apache.org/jira/browse/HIVE-22783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-22783: --- Status: Patch Available (was: Open) > Add test for HIVE-22366 > --- > > Key: HIVE-22783 > URL: https://issues.apache.org/jira/browse/HIVE-22783 > Project: Hive > Issue Type: Sub-task >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-22783.1.patch, HIVE-22783.2.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-21569) Bump guava version to 28.1-jre
[ https://issues.apache.org/jira/browse/HIVE-21569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17030944#comment-17030944 ] László Bodor commented on HIVE-21569: - [~dlavati]: "as mentioned in the above ticket, an option would be to have a spark version that shades all dependencies...try the same approach with hive itself" I'm about to go this way, maybe it's time to take care of guava shade and relocation in hive > Bump guava version to 28.1-jre > -- > > Key: HIVE-21569 > URL: https://issues.apache.org/jira/browse/HIVE-21569 > Project: Hive > Issue Type: Task > Components: Hive >Reporter: Laszlo Pinter >Assignee: László Bodor >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-21569.01.patch, HIVE-21569.02.patch, > HIVE-21569.03.patch, HIVE-21569.04.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Changes required to bump guava to 28.1-jre -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22771) Partition location incorrectly formed in FileOutputCommitterContainer
[
https://issues.apache.org/jira/browse/HIVE-22771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17030941#comment-17030941
]
Mithun Radhakrishnan commented on HIVE-22771:
-
{quote}Sufficiently long format should not increase the chance of collision I
think. Also, do you know if there's any reason for keeping this random ID
numeric?
{quote}
This is a good point, [~dvoros]. There is no reason that I can think of to keep
this numeric. There certainly isn't a reason for this to be in scientific
notation. IIRC, the whole point was to keep the temp directory from clashing
against other attempts. Perhaps a better way forward would be to use a
sufficiently long random {{String}}.
> Partition location incorrectly formed in FileOutputCommitterContainer
> -
>
> Key: HIVE-22771
> URL: https://issues.apache.org/jira/browse/HIVE-22771
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
>Affects Versions: 1.2.1
>Reporter: Shivam
>Assignee: Shivam
>Priority: Critical
> Labels: pull-request-available
> Attachments: HIVE-22771.2.patch, HIVE-22771.3.patch, HIVE-22771.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Class _HCatOutputFormat_ in package _org.apache.hive.hcatalog.mapreduce_ uses
> function _setOutput_ to generate _idHash_ using below statement:
> *+In file org/apache/hive/hcatalog/mapreduce/HCatOutputFormat.java+*
> *line 116: idHash = String.valueOf(Math.random());*
> The output of idHash can be similar to values like this : 7.145347157239135E-4
>
> And, in class _FileOutputCommitterContainer_ in package
> _org.apache.hive.hcatalog.mapreduce;_
> Uses below statement to compute final partition path:
> +*In org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java*+
> *line 366: String finalLocn = jobLocation.replaceAll(Path.SEPARATOR +
> SCRATCH_DIR_NAME + "{color:#ff}\\d\\.?
> d+"{color},"");*
> *line 367: partPath = new Path(finalLocn);*
>
> Regex used here is incorrect, since it will only remove integers after the
> *SCRATCH_DIR_NAME,* and hence will append 'E-4' (for the above example) in
> the final partition location.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-21569) Bump guava version to 28.1-jre
[ https://issues.apache.org/jira/browse/HIVE-21569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor reassigned HIVE-21569: --- Assignee: László Bodor > Bump guava version to 28.1-jre > -- > > Key: HIVE-21569 > URL: https://issues.apache.org/jira/browse/HIVE-21569 > Project: Hive > Issue Type: Task > Components: Hive >Reporter: Laszlo Pinter >Assignee: László Bodor >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-21569.01.patch, HIVE-21569.02.patch, > HIVE-21569.03.patch, HIVE-21569.04.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Changes required to bump guava to 28.1-jre -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22748) Remove disabling of IdentityProjectRemover transformation for Return Path
[
https://issues.apache.org/jira/browse/HIVE-22748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17030908#comment-17030908
]
Hive QA commented on HIVE-22748:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
55s{color} | {color:blue} ql in master has 1533 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} ql: The patch generated 0 new + 27 unchanged - 2
fixed = 27 total (was 29) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 56s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20479/dev-support/hive-personality.sh
|
| git revision | master / f04323f |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20479/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Remove disabling of IdentityProjectRemover transformation for Return Path
> -
>
> Key: HIVE-22748
> URL: https://issues.apache.org/jira/browse/HIVE-22748
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22748.01.patch, HIVE-22748.03.patch
>
>
> For some reason IdentityProjectRemover transformation was not enabled for
> Return Path.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22835) Extract Executor from Driver
[ https://issues.apache.org/jira/browse/HIVE-22835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22835: -- Status: Patch Available (was: Open) > Extract Executor from Driver > > > Key: HIVE-22835 > URL: https://issues.apache.org/jira/browse/HIVE-22835 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22835.01.patch > > > The Driver class contains ~600 lines of code responsible for executing the > command. That means that from the Query Plan as input the appropriate tasks > should be executed. This is a thing done by the execute function, which is > ~300 lines long, and also has some sub functions to help this task. All these > codes should be put into a separate class, where it can do it's job without > getting mixed with the other codes in the Driver. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22835) Extract Executor from Driver
[ https://issues.apache.org/jira/browse/HIVE-22835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22835: -- Attachment: HIVE-22835.01.patch > Extract Executor from Driver > > > Key: HIVE-22835 > URL: https://issues.apache.org/jira/browse/HIVE-22835 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22835.01.patch > > > The Driver class contains ~600 lines of code responsible for executing the > command. That means that from the Query Plan as input the appropriate tasks > should be executed. This is a thing done by the execute function, which is > ~300 lines long, and also has some sub functions to help this task. All these > codes should be put into a separate class, where it can do it's job without > getting mixed with the other codes in the Driver. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-22835) Extract Executor from Driver
[ https://issues.apache.org/jira/browse/HIVE-22835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely reassigned HIVE-22835: - > Extract Executor from Driver > > > Key: HIVE-22835 > URL: https://issues.apache.org/jira/browse/HIVE-22835 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 4.0.0 > > > The Driver class contains ~600 lines of code responsible for executing the > command. That means that from the Query Plan as input the appropriate tasks > should be executed. This is a thing done by the execute function, which is > ~300 lines long, and also has some sub functions to help this task. All these > codes should be put into a separate class, where it can do it's job without > getting mixed with the other codes in the Driver. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22740) Test failing tests with return path
[
https://issues.apache.org/jira/browse/HIVE-22740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17030847#comment-17030847
]
Hive QA commented on HIVE-22740:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
52s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
35s{color} | {color:blue} common in master has 63 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
44s{color} | {color:blue} ql in master has 1533 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
11s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
30s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
16s{color} | {color:red} common: The patch generated 1 new + 369 unchanged - 1
fixed = 370 total (was 370) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} ql: The patch generated 0 new + 27 unchanged - 2
fixed = 27 total (was 29) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
22s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 29m 57s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20478/dev-support/hive-personality.sh
|
| git revision | master / f04323f |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20478/yetus/diff-checkstyle-common.txt
|
| modules | C: common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20478/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Test failing tests with return path
>
>
> Key: HIVE-22740
> URL: https://issues.apache.org/jira/browse/HIVE-22740
> Project: Hive
> Issue Type: Bug
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Attachments: HIVE-22740.01.patch, HIVE-22740.02.patch,
> HIVE-22740.03.patch
>
>
> This is a dummy Jira for testing which tests are still failing with return
> path enabled.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22736) Support replication across multiple encryption zones
[
https://issues.apache.org/jira/browse/HIVE-22736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17030807#comment-17030807
]
Hive QA commented on HIVE-22736:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12992697/HIVE-22736.patch
{color:green}SUCCESS:{color} +1 due to 7 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 286 failed/errored test(s), 17981 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[import_addpartition_blobstore_to_warehouse]
(batchId=306)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[import_blobstore_to_warehouse]
(batchId=306)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[import_blobstore_to_warehouse_nonpart]
(batchId=306)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_into_table]
(batchId=306)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_10]
(batchId=84)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_1]
(batchId=25)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[columnstats_infinity]
(batchId=88)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[comma_in_column_name]
(batchId=5)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[confirm_initial_tbl_stats]
(batchId=34)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[drop_table_removes_partition_dirs]
(batchId=99)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[except_all] (batchId=53)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[fm-sketch] (batchId=59)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby_sort_5]
(batchId=52)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[hll] (batchId=101)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sample6] (batchId=76)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[schema_evol_par_vec_table_dictionary_encoding]
(batchId=30)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[schema_evol_par_vec_table_non_dictionary_encoding]
(batchId=57)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[setop_no_distinct]
(batchId=93)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoinopt17]
(batchId=97)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[statsfs] (batchId=53)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[symlink_text_input_format]
(batchId=54)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[topnkey_order_null]
(batchId=42)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_character_length]
(batchId=44)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_length] (batchId=39)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_octet_length]
(batchId=37)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_udf_character_length]
(batchId=94)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_udf_octet_length]
(batchId=2)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_auto_purge_tables]
(batchId=193)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_ctas]
(batchId=193)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_drop_partition]
(batchId=193)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_drop_table]
(batchId=194)
org.apache.hadoop.hive.cli.TestMiniHiveKafkaCliDriver.testCliDriver[kafka_storage_handler]
(batchId=303)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[except_distinct]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[external_table_purge]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[file_with_header_footer_aggregation]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[autoColumnStats_10]
(batchId=185)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[autoColumnStats_1]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[database]
(batchId=173)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[default_constraint]
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dp_counter_mm]
(batchId=166)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dp_counter_non_mm]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dynpart_sort_optimization]
(batchId=180)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=182)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[load_data_using_job]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[materialized_view_rewrite_5]
(batchId=170)
[jira] [Commented] (HIVE-22736) Support replication across multiple encryption zones
[
https://issues.apache.org/jira/browse/HIVE-22736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17030779#comment-17030779
]
Hive QA commented on HIVE-22736:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
53s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
9s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
18s{color} | {color:blue} shims/common in master has 6 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
22s{color} | {color:blue} shims/0.23 in master has 7 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
37s{color} | {color:blue} standalone-metastore/metastore-common in master has
35 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
34s{color} | {color:blue} common in master has 63 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
17s{color} | {color:blue} standalone-metastore/metastore-server in master has
181 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
41s{color} | {color:blue} ql in master has 1533 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
44s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
17s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
29s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} shims/common: The patch generated 0 new + 93
unchanged - 1 fixed = 93 total (was 94) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} The patch 0.23 passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} The patch metastore-common passed checkstyle {color}
|
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} The patch common passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
24s{color} | {color:green} The patch metastore-server passed checkstyle {color}
|
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} The patch ql passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s{color} | {color:green} The patch hive-unit passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 10m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
17s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
|
[jira] [Updated] (HIVE-22610) Minor compaction for MM (insert-only) tables
[ https://issues.apache.org/jira/browse/HIVE-22610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karen Coppage updated HIVE-22610: - Attachment: HIVE-22610.01.patch Status: Patch Available (was: In Progress) > Minor compaction for MM (insert-only) tables > > > Key: HIVE-22610 > URL: https://issues.apache.org/jira/browse/HIVE-22610 > Project: Hive > Issue Type: Sub-task >Reporter: Karen Coppage >Assignee: Karen Coppage >Priority: Major > Attachments: HIVE-22610.01.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22832) Parallelise mm directory cleaning process
[ https://issues.apache.org/jira/browse/HIVE-22832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Bod updated HIVE-22832: -- Attachment: HIVE-22832.1.patch Status: Patch Available (was: Open) > Parallelise mm directory cleaning process > - > > Key: HIVE-22832 > URL: https://issues.apache.org/jira/browse/HIVE-22832 > Project: Hive > Issue Type: Improvement >Reporter: Marton Bod >Assignee: Marton Bod >Priority: Major > Attachments: HIVE-22832.1.patch > > > Inside Utilities::handleMmTableFinalPath, the cleanMmDirectories method is > called sequentially for each element of the mmDirectory list, which might > have a large number of elements depending on how many partitions were > written. This current sequential execution could be improved by parallelising > the clean up process. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22825) Reduce directory lookup cost for acid tables
[ https://issues.apache.org/jira/browse/HIVE-22825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17030744#comment-17030744 ] Ashutosh Chauhan commented on HIVE-22825: - 1. There is no invalidation of cache yet. Is that not needed or this is experimental only? 2. I will suggest to change config hive.txn.acid.dir.cache.enabled to hive.txn.acid.dir.cache.duration with default value of -1 meaning disabled. And positive value used as expiry for cache duration. (making 30mins configurable) > Reduce directory lookup cost for acid tables > > > Key: HIVE-22825 > URL: https://issues.apache.org/jira/browse/HIVE-22825 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Rajesh Balamohan >Assignee: Rajesh Balamohan >Priority: Minor > Attachments: HIVE-22825.1.patch, HIVE-22825.2.patch, > HIVE-22825.3.patch > > > With objectstores, directory lookup costs are expensive. For acid tables, it > would be good to have a directory cache to reduce number of lookup calls. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22736) Support replication across multiple encryption zones
[
https://issues.apache.org/jira/browse/HIVE-22736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17030733#comment-17030733
]
Hive QA commented on HIVE-22736:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12992682/HIVE-22736.patch
{color:green}SUCCESS:{color} +1 due to 7 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 283 failed/errored test(s), 17981 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[import_addpartition_blobstore_to_warehouse]
(batchId=306)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[import_blobstore_to_warehouse]
(batchId=306)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[import_blobstore_to_warehouse_nonpart]
(batchId=306)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_into_table]
(batchId=306)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_10]
(batchId=84)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_1]
(batchId=25)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[columnstats_infinity]
(batchId=88)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[comma_in_column_name]
(batchId=5)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[confirm_initial_tbl_stats]
(batchId=34)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[drop_table_removes_partition_dirs]
(batchId=99)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[except_all] (batchId=53)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[fm-sketch] (batchId=59)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby_sort_5]
(batchId=52)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[hll] (batchId=101)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sample6] (batchId=76)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[schema_evol_par_vec_table_dictionary_encoding]
(batchId=30)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[schema_evol_par_vec_table_non_dictionary_encoding]
(batchId=57)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[setop_no_distinct]
(batchId=93)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoinopt17]
(batchId=97)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[statsfs] (batchId=53)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[symlink_text_input_format]
(batchId=54)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[topnkey_order_null]
(batchId=42)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_character_length]
(batchId=44)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_length] (batchId=39)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_octet_length]
(batchId=37)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_udf_character_length]
(batchId=94)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_udf_octet_length]
(batchId=2)
org.apache.hadoop.hive.cli.TestMiniHiveKafkaCliDriver.testCliDriver[kafka_storage_handler]
(batchId=303)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[except_distinct]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[external_table_purge]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[file_with_header_footer_aggregation]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[autoColumnStats_10]
(batchId=185)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[autoColumnStats_1]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[database]
(batchId=173)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[default_constraint]
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dp_counter_mm]
(batchId=166)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dp_counter_non_mm]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dynpart_sort_optimization]
(batchId=180)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=182)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[load_data_using_job]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[materialized_view_rewrite_5]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[materialized_view_rewrite_no_join_opt]
(batchId=191)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_in]
(batchId=182)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_notin]
(batchId=184)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_reduce_side]
(batchId=179)
[jira] [Assigned] (HIVE-22834) Consistency: Expose unique table-identifiers in org.apache.hadoop.hive.ql.metadata.Table
[
https://issues.apache.org/jira/browse/HIVE-22834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gopal Vijayaraghavan reassigned HIVE-22834:
---
Assignee: Gopal Vijayaraghavan
> Consistency: Expose unique table-identifiers in
> org.apache.hadoop.hive.ql.metadata.Table
>
>
> Key: HIVE-22834
> URL: https://issues.apache.org/jira/browse/HIVE-22834
> Project: Hive
> Issue Type: Bug
>Reporter: Gopal Vijayaraghavan
>Assignee: Gopal Vijayaraghavan
>Priority: Major
>
> Distinguish between the two tables in
> {code}
> create table foo as select 1 as x;
> drop table foo;
> create table foo as select 2 as x;
> {code}
> in caching subsystems.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22736) Support replication across multiple encryption zones
[ https://issues.apache.org/jira/browse/HIVE-22736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-22736: --- Attachment: HIVE-22736.patch Status: Patch Available (was: In Progress) > Support replication across multiple encryption zones > > > Key: HIVE-22736 > URL: https://issues.apache.org/jira/browse/HIVE-22736 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch > > Time Spent: 3h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22736) Support replication across multiple encryption zones
[ https://issues.apache.org/jira/browse/HIVE-22736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-22736: --- Status: In Progress (was: Patch Available) > Support replication across multiple encryption zones > > > Key: HIVE-22736 > URL: https://issues.apache.org/jira/browse/HIVE-22736 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch > > Time Spent: 3h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-22825) Reduce directory lookup cost for acid tables
[ https://issues.apache.org/jira/browse/HIVE-22825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal Vijayaraghavan reassigned HIVE-22825: --- Assignee: Rajesh Balamohan > Reduce directory lookup cost for acid tables > > > Key: HIVE-22825 > URL: https://issues.apache.org/jira/browse/HIVE-22825 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Rajesh Balamohan >Assignee: Rajesh Balamohan >Priority: Minor > Attachments: HIVE-22825.1.patch, HIVE-22825.2.patch, > HIVE-22825.3.patch > > > With objectstores, directory lookup costs are expensive. For acid tables, it > would be good to have a directory cache to reduce number of lookup calls. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22736) Support replication across multiple encryption zones
[
https://issues.apache.org/jira/browse/HIVE-22736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17030717#comment-17030717
]
Hive QA commented on HIVE-22736:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
58s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
20s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
20s{color} | {color:blue} shims/common in master has 6 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
22s{color} | {color:blue} shims/0.23 in master has 7 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
46s{color} | {color:blue} standalone-metastore/metastore-common in master has
35 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
34s{color} | {color:blue} common in master has 63 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
11s{color} | {color:blue} standalone-metastore/metastore-server in master has
181 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
44s{color} | {color:blue} ql in master has 1533 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
44s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
18s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
29s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} shims/common: The patch generated 0 new + 93
unchanged - 1 fixed = 93 total (was 94) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} The patch 0.23 passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} The patch metastore-common passed checkstyle {color}
|
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s{color} | {color:green} The patch common passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s{color} | {color:green} The patch metastore-server passed checkstyle {color}
|
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} The patch ql passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s{color} | {color:green} The patch hive-unit passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
4s{color} | {color:red} ql generated 1 new + 1533 unchanged - 0 fixed = 1534
total (was 1533) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
21s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not
[jira] [Commented] (HIVE-22805) Vectorization with conditional array or map is not implemented and throws an error
[
https://issues.apache.org/jira/browse/HIVE-22805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17030670#comment-17030670
]
Marta Kuczora commented on HIVE-22805:
--
+1
Thanks a lot [~rameshkumar] for the review and [~pvary] for the patch.
> Vectorization with conditional array or map is not implemented and throws an
> error
> --
>
> Key: HIVE-22805
> URL: https://issues.apache.org/jira/browse/HIVE-22805
> Project: Hive
> Issue Type: Bug
> Components: Vectorization
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-22805.2.patch, HIVE-22805.3.patch, HIVE-22805.patch
>
>
> The following queries throw an "Not supported" Exception:
> {code}
> DROP TABLE IF EXISTS cond_vector;
> CREATE TABLE cond_vector(a STRING) STORED AS ORC;
> INSERT OVERWRITE TABLE cond_vector VALUES("a/b");
> set hive.fetch.task.conversion=minimal;
> set hive.execution.mode=container;
> SELECT IF(1=1, MAP("a","b"), NULL) FROM cond_vector;
> EXPLAIN VECTORIZATION DETAIL SELECT IF(1=1, MAP("Mathematics","78"), NULL)
> FROM cond_vector;
> SELECT IF(1=1, ARRAY("c", "d"), NULL) FROM cond_vector;
> EXPLAIN VECTORIZATION DETAIL SELECT IF(1=1, ARRAY("a", "b"), NULL) FROM
> cond_vector;
> {code}
> One example of the Exception (maybe not exactly this on 4.0.0 branch, but
> close enough to check):
> {code}
> Caused by: java.lang.RuntimeException: Not supported
> at
> org.apache.hadoop.hive.ql.exec.vector.ListColumnVector.copySelected(ListColumnVector.java:161)
> at
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprNull.evaluate(IfExprCondExprNull.java:87)
> at
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprCondExpr.evaluate(IfExprCondExprCondExpr.java:95)
> at
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprBase.conditionalEvaluate(IfExprCondExprBase.java:68)
> at
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprCondExpr.evaluate(IfExprCondExprCondExpr.java:110)
> at
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprCondExpr.evaluate(IfExprCondExprCondExpr.java:95)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.process(VectorSelectOperator.java:146)
>
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22821) Add necessary endpoints for proactive cache eviction
[
https://issues.apache.org/jira/browse/HIVE-22821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17030666#comment-17030666
]
Hive QA commented on HIVE-22821:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12992672/HIVE-22821.2.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17965 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20475/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20475/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20475/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12992672 - PreCommit-HIVE-Build
> Add necessary endpoints for proactive cache eviction
>
>
> Key: HIVE-22821
> URL: https://issues.apache.org/jira/browse/HIVE-22821
> Project: Hive
> Issue Type: Sub-task
> Components: llap
>Reporter: Ádám Szita
>Assignee: Ádám Szita
>Priority: Major
> Attachments: HIVE-22821.0.patch, HIVE-22821.1.patch,
> HIVE-22821.2.patch
>
>
> Implement the parts required for iHS2 -> LLAP daemons communication:
> * protobuf message schema and endpoints
> * Hive configuration
> * for use cases:
> ** dropping db
> ** dropping table
> ** dropping partition from a table
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-22832) Parallelise mm directory cleaning process
[ https://issues.apache.org/jira/browse/HIVE-22832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Bod reassigned HIVE-22832: - > Parallelise mm directory cleaning process > - > > Key: HIVE-22832 > URL: https://issues.apache.org/jira/browse/HIVE-22832 > Project: Hive > Issue Type: Improvement >Reporter: Marton Bod >Assignee: Marton Bod >Priority: Major > > Inside Utilities::handleMmTableFinalPath, the cleanMmDirectories method is > called sequentially for each element of the mmDirectory list, which might > have a large number of elements depending on how many partitions were > written. This current sequential execution could be improved by parallelising > the clean up process. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22825) Reduce directory lookup cost for acid tables
[ https://issues.apache.org/jira/browse/HIVE-22825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HIVE-22825: Attachment: HIVE-22825.3.patch > Reduce directory lookup cost for acid tables > > > Key: HIVE-22825 > URL: https://issues.apache.org/jira/browse/HIVE-22825 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Rajesh Balamohan >Priority: Minor > Attachments: HIVE-22825.1.patch, HIVE-22825.2.patch, > HIVE-22825.3.patch > > > With objectstores, directory lookup costs are expensive. For acid tables, it > would be good to have a directory cache to reduce number of lookup calls. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-21216) Write Parquet INT64 timestamp
[ https://issues.apache.org/jira/browse/HIVE-21216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karen Coppage updated HIVE-21216: - Attachment: HIVE-21216.2.patch Status: Patch Available (was: In Progress) > Write Parquet INT64 timestamp > - > > Key: HIVE-21216 > URL: https://issues.apache.org/jira/browse/HIVE-21216 > Project: Hive > Issue Type: New Feature > Components: Hive >Reporter: Karen Coppage >Assignee: Karen Coppage >Priority: Major > Labels: breaking_change > Attachments: HIVE-21216.1.patch, HIVE-21216.2.patch > > > This patch enables Hive to start writing int64 timestamps in Parquet. > With Parquet version 1.11, a new timestamp LogicalType with base INT64 and > the following metadata is introduced: > boolean isAdjustedToUtc: marks whether the timestamp is converted to UTC > (aka Instant semantics) or not (LocalDateTime semantics) > enum TimeUnit (NANOS, MICROS, MILLIS): granularity of timestamp > The timestamp will have LocalDateTime semantics (not converted to UTC). > Timestamps outside of range 1677-09-21 00:12:43.145224192 – 2262-04-11 > 23:47:16.854775807 cannot be written in nanos and will be recorded as NULL. > Time unit (granularity) will be determined by the user. Default is > milliseconds. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-21216) Write Parquet INT64 timestamp
[ https://issues.apache.org/jira/browse/HIVE-21216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karen Coppage updated HIVE-21216: - Description: This patch enables Hive to start writing int64 timestamps in Parquet. With Parquet version 1.11, a new timestamp LogicalType with base INT64 and the following metadata is introduced: boolean isAdjustedToUtc: marks whether the timestamp is converted to UTC (aka Instant semantics) or not (LocalDateTime semantics) enum TimeUnit (NANOS, MICROS, MILLIS): granularity of timestamp The timestamp will have LocalDateTime semantics (not converted to UTC). Timestamps outside of range 1677-09-21 00:12:43.145224192 – 2262-04-11 23:47:16.854775807 cannot be written in nanos and will be recorded as NULL. Time unit (granularity) will be determined by the user. Default is milliseconds. was: [WIP] NO PRECOMMIT TESTS This patch enables Hive to start writing int64 timestamps in Parquet. With Parquet version 1.11, a new timestamp LogicalType with base INT64 and the following metadata is introduced: boolean isAdjustedToUtc: marks whether the timestamp is converted to UTC (aka Instant semantics) or not (LocalDateTime semantics) enum TimeUnit (NANOS, MICROS, MILLIS): granularity of timestamp The timestamp will have LocalDateTime semantics (not converted to UTC). Timestamps outside of range 1677-09-21 00:12:43.145224192 – 2262-04-11 23:47:16.854775807 cannot be written in nanos and will be recorded as NULL. Time unit (granularity) will be determined by the user. Default is milliseconds. > Write Parquet INT64 timestamp > - > > Key: HIVE-21216 > URL: https://issues.apache.org/jira/browse/HIVE-21216 > Project: Hive > Issue Type: New Feature > Components: Hive >Reporter: Karen Coppage >Assignee: Karen Coppage >Priority: Major > Labels: breaking_change > Attachments: HIVE-21216.1.patch > > > This patch enables Hive to start writing int64 timestamps in Parquet. > With Parquet version 1.11, a new timestamp LogicalType with base INT64 and > the following metadata is introduced: > boolean isAdjustedToUtc: marks whether the timestamp is converted to UTC > (aka Instant semantics) or not (LocalDateTime semantics) > enum TimeUnit (NANOS, MICROS, MILLIS): granularity of timestamp > The timestamp will have LocalDateTime semantics (not converted to UTC). > Timestamps outside of range 1677-09-21 00:12:43.145224192 – 2262-04-11 > 23:47:16.854775807 cannot be written in nanos and will be recorded as NULL. > Time unit (granularity) will be determined by the user. Default is > milliseconds. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-22831) Add option in HiveStrictManagedMigration to also move managed tables converted to external
[ https://issues.apache.org/jira/browse/HIVE-22831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ádám Szita reassigned HIVE-22831: - > Add option in HiveStrictManagedMigration to also move managed tables > converted to external > -- > > Key: HIVE-22831 > URL: https://issues.apache.org/jira/browse/HIVE-22831 > Project: Hive > Issue Type: Improvement >Reporter: Ádám Szita >Assignee: Ádám Szita >Priority: Major > > HiveStrictManagedMigration supports these use cases (among others) currently: > * convert managed tables to external (+set external purge) > * move managed tables from old to new warehouse root (HDFS) > I propose we add a feature that combines both: > * convert managed tables to external and move them (from old managed > warehouse root) to a new path ( e.g. default external warehouse location) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22821) Add necessary endpoints for proactive cache eviction
[
https://issues.apache.org/jira/browse/HIVE-22821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17030636#comment-17030636
]
Hive QA commented on HIVE-22821:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
31s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
36s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
37s{color} | {color:blue} common in master has 63 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
33s{color} | {color:blue} llap-common in master has 90 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
24s{color} | {color:blue} llap-client in master has 27 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
41s{color} | {color:blue} ql in master has 1533 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
44s{color} | {color:blue} llap-server in master has 90 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
48s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
29s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
13s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
42s{color} | {color:red} ql: The patch generated 2 new + 2 unchanged - 0 fixed
= 4 total (was 2) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 17 line(s) that end in whitespace. Use
git apply --whitespace=fix <>. Refer
https://git-scm.com/docs/git-apply {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
40s{color} | {color:red} llap-common generated 8 new + 90 unchanged - 0 fixed =
98 total (was 90) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
52s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 38m 51s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:llap-common |
| |
org.apache.hadoop.hive.llap.daemon.rpc.LlapDaemonProtocolProtos$EvictEntityRequestProto.PARSER
isn't final but should be At LlapDaemonProtocolProtos.java:be At
LlapDaemonProtocolProtos.java:[line 22695] |
| | Class
org.apache.hadoop.hive.llap.daemon.rpc.LlapDaemonProtocolProtos$EvictEntityRequestProto
defines non-transient non-serializable instance field unknownFields In
LlapDaemonProtocolProtos.java:instance field unknownFields In
LlapDaemonProtocolProtos.java |
| |
org.apache.hadoop.hive.llap.daemon.rpc.LlapDaemonProtocolProtos$EvictEntityResponseProto.PARSER
isn't final but should be At LlapDaemonProtocolProtos.java:be At
LlapDaemonProtocolProtos.java:[line 24450] |
| | Class
org.apache.hadoop.hive.llap.daemon.rpc.LlapDaemonProtocolProtos$EvictEntityResponseProto
defines non-transient non-serializable instance field unknownFields In
LlapDaemonProtocolProtos.java:instance field unknownFields In
LlapDaemonProtocolProtos.java |
| | Useless control flow in
org.apache.hadoop.hive.llap.daemon.rpc.LlapDaemonProtocolProtos$EvictEntityResponseProto$Builder.maybeForceBuilderInitialization()
At LlapDaemonProtocolProtos.java: At
[jira] [Updated] (HIVE-22808) HiveRelFieldTrimmer does not handle HiveTableFunctionScan
[
https://issues.apache.org/jira/browse/HIVE-22808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22808:
--
Attachment: HIVE-22808.5.patch
> HiveRelFieldTrimmer does not handle HiveTableFunctionScan
> -
>
> Key: HIVE-22808
> URL: https://issues.apache.org/jira/browse/HIVE-22808
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22808.1.patch, HIVE-22808.2.patch,
> HIVE-22808.2.patch, HIVE-22808.3.patch, HIVE-22808.4.patch, HIVE-22808.5.patch
>
>
> *Repro*
> {code:java}
> CREATE TABLE table_16 (
> timestamp_col_19timestamp,
> timestamp_col_29timestamp,
> int_col_27 int,
> int_col_39 int,
> boolean_col_18 boolean,
> varchar0045_col_23 varchar(45)
> );
> CREATE TABLE table_7 (
> int_col_10 int,
> bigint_col_3bigint
> );
> CREATE TABLE table_10 (
> boolean_col_8 boolean,
> boolean_col_16 boolean,
> timestamp_col_5 timestamp,
> timestamp_col_15timestamp,
> timestamp_col_30timestamp,
> decimal3825_col_26 decimal(38, 25),
> smallint_col_9 smallint,
> int_col_18 int
> );
> explain cbo
> SELECT
> DISTINCT COALESCE(a4.timestamp_col_15, IF(a4.boolean_col_16,
> a4.timestamp_col_30, a4.timestamp_col_5)) AS timestamp_col
> FROM table_7 a3
> RIGHT JOIN table_10 a4
> WHERE (a3.bigint_col_3) >= (a4.int_col_18)
> INTERSECT ALL
> SELECT
> COALESCE(LEAST(
> COALESCE(a1.timestamp_col_19, CAST('2010-03-29 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2014-08-16 00:00:00' AS
> TIMESTAMP))
> ),
> GREATEST(COALESCE(a1.timestamp_col_19, CAST('2013-07-01 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2028-06-18 00:00:00' AS
> TIMESTAMP)))
> ) AS timestamp_col
> FROM table_16 a1
> GROUP BY COALESCE(LEAST(
> COALESCE(a1.timestamp_col_19, CAST('2010-03-29 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2014-08-16 00:00:00' AS
> TIMESTAMP))
> ),
> GREATEST(
> COALESCE(a1.timestamp_col_19, CAST('2013-07-01 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2028-06-18 00:00:00' AS
> TIMESTAMP)))
> );
> {code}
> CBO Plan contains unnecessary columns or all columns from a table in
> projections like:
> {code:java}
> HiveProject(int_col_10=[$0], bigint_col_3=[$1],
> BLOCK__OFFSET__INSIDE__FILE=[$2], INPUT__FILE__NAME=[$3],
> CAST=[CAST($4):RecordType(BIGINT writeid, INTEGER bucketid, BIGINT rowid)])
> {code}
> *Cause*
> The plan contains a HiveTableFunctionScan operator:
> {code:java}
> HiveTableFunctionScan(invocation=[replicate_rows($0, $1)],
> rowType=[RecordType(BIGINT $f0, TIMESTAMP(9) $f1)])
> {code}
> HiveTableFunctionScan is not handled by HiveRelFieldTrimmer nor
> RelFieldTrimmer which suppose to remove unused columns in the
> CalcitePlanner.applyPreJoinOrderingTransforms(...) phase. The whole subtree
> rooted from HiveTableFunctionScan is ignored.
> Whole plan:
> {code:java}
> CBO PLAN:
> HiveProject($f0=[$1])
> HiveTableFunctionScan(invocation=[replicate_rows($0, $1)],
> rowType=[RecordType(BIGINT $f0, TIMESTAMP(9) $f1)])
> HiveProject($f0=[$2], $f1=[$0])
> HiveFilter(condition=[=($1, 2)])
> HiveAggregate(group=[{0}], agg#0=[count($1)], agg#1=[min($1)])
> HiveProject($f0=[$0], $f1=[$1])
> HiveUnion(all=[true])
> HiveProject($f0=[$0], $f1=[$1])
> HiveAggregate(group=[{0}], agg#0=[count()])
> HiveProject($f0=[$0])
> HiveAggregate(group=[{0}])
> HiveProject($f0=[CASE(IS NOT NULL($7), $7, if($5, $8,
> $6))])
> HiveJoin(condition=[>=($1, $13)], joinType=[inner],
> algorithm=[none], cost=[not available])
> HiveProject(int_col_10=[$0], bigint_col_3=[$1],
> BLOCK__OFFSET__INSIDE__FILE=[$2], INPUT__FILE__NAME=[$3],
> CAST=[CAST($4):RecordType(BIGINT writeid, INTEGER bucketid, BIGINT rowid)])
> HiveFilter(condition=[IS NOT NULL($1)])
> HiveTableScan(table=[[default, table_7]],
> table:alias=[a3])
> HiveProject(boolean_col_16=[$0],
> timestamp_col_5=[$1], timestamp_col_15=[$2], timestamp_col_30=[$3],
> int_col_18=[$4], BLOCK__OFFSET__INSIDE__FILE=[$5], INPUT__FILE__NAME=[$6],
> ROW__ID=[$7], CAST=[CAST($4):BIGINT])
> HiveFilter(condition=[IS NOT
> NULL(CAST($4):BIGINT)])
> HiveTableScan(table=[[default, table_10]],
> table:alias=[a4])
>
[jira] [Updated] (HIVE-22808) HiveRelFieldTrimmer does not handle HiveTableFunctionScan
[
https://issues.apache.org/jira/browse/HIVE-22808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22808:
--
Status: Patch Available (was: Open)
> HiveRelFieldTrimmer does not handle HiveTableFunctionScan
> -
>
> Key: HIVE-22808
> URL: https://issues.apache.org/jira/browse/HIVE-22808
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22808.1.patch, HIVE-22808.2.patch,
> HIVE-22808.2.patch, HIVE-22808.3.patch, HIVE-22808.4.patch, HIVE-22808.5.patch
>
>
> *Repro*
> {code:java}
> CREATE TABLE table_16 (
> timestamp_col_19timestamp,
> timestamp_col_29timestamp,
> int_col_27 int,
> int_col_39 int,
> boolean_col_18 boolean,
> varchar0045_col_23 varchar(45)
> );
> CREATE TABLE table_7 (
> int_col_10 int,
> bigint_col_3bigint
> );
> CREATE TABLE table_10 (
> boolean_col_8 boolean,
> boolean_col_16 boolean,
> timestamp_col_5 timestamp,
> timestamp_col_15timestamp,
> timestamp_col_30timestamp,
> decimal3825_col_26 decimal(38, 25),
> smallint_col_9 smallint,
> int_col_18 int
> );
> explain cbo
> SELECT
> DISTINCT COALESCE(a4.timestamp_col_15, IF(a4.boolean_col_16,
> a4.timestamp_col_30, a4.timestamp_col_5)) AS timestamp_col
> FROM table_7 a3
> RIGHT JOIN table_10 a4
> WHERE (a3.bigint_col_3) >= (a4.int_col_18)
> INTERSECT ALL
> SELECT
> COALESCE(LEAST(
> COALESCE(a1.timestamp_col_19, CAST('2010-03-29 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2014-08-16 00:00:00' AS
> TIMESTAMP))
> ),
> GREATEST(COALESCE(a1.timestamp_col_19, CAST('2013-07-01 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2028-06-18 00:00:00' AS
> TIMESTAMP)))
> ) AS timestamp_col
> FROM table_16 a1
> GROUP BY COALESCE(LEAST(
> COALESCE(a1.timestamp_col_19, CAST('2010-03-29 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2014-08-16 00:00:00' AS
> TIMESTAMP))
> ),
> GREATEST(
> COALESCE(a1.timestamp_col_19, CAST('2013-07-01 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2028-06-18 00:00:00' AS
> TIMESTAMP)))
> );
> {code}
> CBO Plan contains unnecessary columns or all columns from a table in
> projections like:
> {code:java}
> HiveProject(int_col_10=[$0], bigint_col_3=[$1],
> BLOCK__OFFSET__INSIDE__FILE=[$2], INPUT__FILE__NAME=[$3],
> CAST=[CAST($4):RecordType(BIGINT writeid, INTEGER bucketid, BIGINT rowid)])
> {code}
> *Cause*
> The plan contains a HiveTableFunctionScan operator:
> {code:java}
> HiveTableFunctionScan(invocation=[replicate_rows($0, $1)],
> rowType=[RecordType(BIGINT $f0, TIMESTAMP(9) $f1)])
> {code}
> HiveTableFunctionScan is not handled by HiveRelFieldTrimmer nor
> RelFieldTrimmer which suppose to remove unused columns in the
> CalcitePlanner.applyPreJoinOrderingTransforms(...) phase. The whole subtree
> rooted from HiveTableFunctionScan is ignored.
> Whole plan:
> {code:java}
> CBO PLAN:
> HiveProject($f0=[$1])
> HiveTableFunctionScan(invocation=[replicate_rows($0, $1)],
> rowType=[RecordType(BIGINT $f0, TIMESTAMP(9) $f1)])
> HiveProject($f0=[$2], $f1=[$0])
> HiveFilter(condition=[=($1, 2)])
> HiveAggregate(group=[{0}], agg#0=[count($1)], agg#1=[min($1)])
> HiveProject($f0=[$0], $f1=[$1])
> HiveUnion(all=[true])
> HiveProject($f0=[$0], $f1=[$1])
> HiveAggregate(group=[{0}], agg#0=[count()])
> HiveProject($f0=[$0])
> HiveAggregate(group=[{0}])
> HiveProject($f0=[CASE(IS NOT NULL($7), $7, if($5, $8,
> $6))])
> HiveJoin(condition=[>=($1, $13)], joinType=[inner],
> algorithm=[none], cost=[not available])
> HiveProject(int_col_10=[$0], bigint_col_3=[$1],
> BLOCK__OFFSET__INSIDE__FILE=[$2], INPUT__FILE__NAME=[$3],
> CAST=[CAST($4):RecordType(BIGINT writeid, INTEGER bucketid, BIGINT rowid)])
> HiveFilter(condition=[IS NOT NULL($1)])
> HiveTableScan(table=[[default, table_7]],
> table:alias=[a3])
> HiveProject(boolean_col_16=[$0],
> timestamp_col_5=[$1], timestamp_col_15=[$2], timestamp_col_30=[$3],
> int_col_18=[$4], BLOCK__OFFSET__INSIDE__FILE=[$5], INPUT__FILE__NAME=[$6],
> ROW__ID=[$7], CAST=[CAST($4):BIGINT])
> HiveFilter(condition=[IS NOT
> NULL(CAST($4):BIGINT)])
> HiveTableScan(table=[[default, table_10]],
> table:alias=[a4])
>
[jira] [Updated] (HIVE-22808) HiveRelFieldTrimmer does not handle HiveTableFunctionScan
[
https://issues.apache.org/jira/browse/HIVE-22808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22808:
--
Status: Open (was: Patch Available)
> HiveRelFieldTrimmer does not handle HiveTableFunctionScan
> -
>
> Key: HIVE-22808
> URL: https://issues.apache.org/jira/browse/HIVE-22808
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22808.1.patch, HIVE-22808.2.patch,
> HIVE-22808.2.patch, HIVE-22808.3.patch, HIVE-22808.4.patch, HIVE-22808.5.patch
>
>
> *Repro*
> {code:java}
> CREATE TABLE table_16 (
> timestamp_col_19timestamp,
> timestamp_col_29timestamp,
> int_col_27 int,
> int_col_39 int,
> boolean_col_18 boolean,
> varchar0045_col_23 varchar(45)
> );
> CREATE TABLE table_7 (
> int_col_10 int,
> bigint_col_3bigint
> );
> CREATE TABLE table_10 (
> boolean_col_8 boolean,
> boolean_col_16 boolean,
> timestamp_col_5 timestamp,
> timestamp_col_15timestamp,
> timestamp_col_30timestamp,
> decimal3825_col_26 decimal(38, 25),
> smallint_col_9 smallint,
> int_col_18 int
> );
> explain cbo
> SELECT
> DISTINCT COALESCE(a4.timestamp_col_15, IF(a4.boolean_col_16,
> a4.timestamp_col_30, a4.timestamp_col_5)) AS timestamp_col
> FROM table_7 a3
> RIGHT JOIN table_10 a4
> WHERE (a3.bigint_col_3) >= (a4.int_col_18)
> INTERSECT ALL
> SELECT
> COALESCE(LEAST(
> COALESCE(a1.timestamp_col_19, CAST('2010-03-29 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2014-08-16 00:00:00' AS
> TIMESTAMP))
> ),
> GREATEST(COALESCE(a1.timestamp_col_19, CAST('2013-07-01 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2028-06-18 00:00:00' AS
> TIMESTAMP)))
> ) AS timestamp_col
> FROM table_16 a1
> GROUP BY COALESCE(LEAST(
> COALESCE(a1.timestamp_col_19, CAST('2010-03-29 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2014-08-16 00:00:00' AS
> TIMESTAMP))
> ),
> GREATEST(
> COALESCE(a1.timestamp_col_19, CAST('2013-07-01 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2028-06-18 00:00:00' AS
> TIMESTAMP)))
> );
> {code}
> CBO Plan contains unnecessary columns or all columns from a table in
> projections like:
> {code:java}
> HiveProject(int_col_10=[$0], bigint_col_3=[$1],
> BLOCK__OFFSET__INSIDE__FILE=[$2], INPUT__FILE__NAME=[$3],
> CAST=[CAST($4):RecordType(BIGINT writeid, INTEGER bucketid, BIGINT rowid)])
> {code}
> *Cause*
> The plan contains a HiveTableFunctionScan operator:
> {code:java}
> HiveTableFunctionScan(invocation=[replicate_rows($0, $1)],
> rowType=[RecordType(BIGINT $f0, TIMESTAMP(9) $f1)])
> {code}
> HiveTableFunctionScan is not handled by HiveRelFieldTrimmer nor
> RelFieldTrimmer which suppose to remove unused columns in the
> CalcitePlanner.applyPreJoinOrderingTransforms(...) phase. The whole subtree
> rooted from HiveTableFunctionScan is ignored.
> Whole plan:
> {code:java}
> CBO PLAN:
> HiveProject($f0=[$1])
> HiveTableFunctionScan(invocation=[replicate_rows($0, $1)],
> rowType=[RecordType(BIGINT $f0, TIMESTAMP(9) $f1)])
> HiveProject($f0=[$2], $f1=[$0])
> HiveFilter(condition=[=($1, 2)])
> HiveAggregate(group=[{0}], agg#0=[count($1)], agg#1=[min($1)])
> HiveProject($f0=[$0], $f1=[$1])
> HiveUnion(all=[true])
> HiveProject($f0=[$0], $f1=[$1])
> HiveAggregate(group=[{0}], agg#0=[count()])
> HiveProject($f0=[$0])
> HiveAggregate(group=[{0}])
> HiveProject($f0=[CASE(IS NOT NULL($7), $7, if($5, $8,
> $6))])
> HiveJoin(condition=[>=($1, $13)], joinType=[inner],
> algorithm=[none], cost=[not available])
> HiveProject(int_col_10=[$0], bigint_col_3=[$1],
> BLOCK__OFFSET__INSIDE__FILE=[$2], INPUT__FILE__NAME=[$3],
> CAST=[CAST($4):RecordType(BIGINT writeid, INTEGER bucketid, BIGINT rowid)])
> HiveFilter(condition=[IS NOT NULL($1)])
> HiveTableScan(table=[[default, table_7]],
> table:alias=[a3])
> HiveProject(boolean_col_16=[$0],
> timestamp_col_5=[$1], timestamp_col_15=[$2], timestamp_col_30=[$3],
> int_col_18=[$4], BLOCK__OFFSET__INSIDE__FILE=[$5], INPUT__FILE__NAME=[$6],
> ROW__ID=[$7], CAST=[CAST($4):BIGINT])
> HiveFilter(condition=[IS NOT
> NULL(CAST($4):BIGINT)])
> HiveTableScan(table=[[default, table_10]],
> table:alias=[a4])
>
[jira] [Updated] (HIVE-22804) Ensure ANSI quotes are used for mysql connections
[
https://issues.apache.org/jira/browse/HIVE-22804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Vary updated HIVE-22804:
--
Fix Version/s: 4.0.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
Pushed to master.
Thanks for the patch [~zchovan]!
Please create a follow-up jira to remove old hacks for ANSI quote settings.
Thanks,
Peter
> Ensure ANSI quotes are used for mysql connections
> -
>
> Key: HIVE-22804
> URL: https://issues.apache.org/jira/browse/HIVE-22804
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Chovan
>Assignee: Zoltan Chovan
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22804.patch
>
>
> Recent changes in direct sql queries to resolve postgres issues(e.g.
> TxnHandler in HIVE-22663 ) break compatibility with mysql backend db.
> A workaround for these issues is to add a session config to the mysql
> connection string, e.g.:
> {code:java}
> jdbc:mysql://localhost:3306/db?sessionVariables=sql_mode=ANSI_QUOTES
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HIVE-10210) Compute partition column stats fails when partition value is zero-leading integer
[
https://issues.apache.org/jira/browse/HIVE-10210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karen Coppage resolved HIVE-10210.
--
Resolution: Duplicate
> Compute partition column stats fails when partition value is zero-leading
> integer
> -
>
> Key: HIVE-10210
> URL: https://issues.apache.org/jira/browse/HIVE-10210
> Project: Hive
> Issue Type: Bug
>Reporter: Chaoyu Tang
>Assignee: Chaoyu Tang
>Priority: Major
>
> The command "Analyze table .. partition compute statistics for columns" fails
> if the partition value is not a normalize integer with leading zeros. For
> example:
> create table colstatspartint (key int, value string) partitioned by (part
> int);
> insert into colstatspartint partition (part='0003') select key, value from
> src limit 30;
> analyze table colstatspartint partition (part='0003') compute statistics for
> columns; or
> analyze table colstatspartint partition (part=0003) compute statistics for
> columns;
> you will get the error:
> {code}
> 15/04/03 10:13:19 ERROR metastore.RetryingHMSHandler:
> NoSuchObjectException(message:Partition for which stats is gathered doesn't
> exist.)
> at
> org.apache.hadoop.hive.metastore.ObjectStore.updatePartitionColumnStatistics(ObjectStore.java:5952)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:114)
> at com.sun.proxy.$Proxy6.updatePartitionColumnStatistics(Unknown Source)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.update_partition_column_statistics(HiveMetaStore.java:4346)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.set_aggr_stats_for(HiveMetaStore.java:5678)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22805) Vectorization with conditional array or map is not implemented and throws an error
[
https://issues.apache.org/jira/browse/HIVE-22805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17030616#comment-17030616
]
Peter Vary commented on HIVE-22805:
---
[~rameshkumar]: Sorry, I have missed your non-binding +1 on RB. Thanks for the
review!
[~kuczoram]: Could you please review?
Thanks,
Peter
> Vectorization with conditional array or map is not implemented and throws an
> error
> --
>
> Key: HIVE-22805
> URL: https://issues.apache.org/jira/browse/HIVE-22805
> Project: Hive
> Issue Type: Bug
> Components: Vectorization
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-22805.2.patch, HIVE-22805.3.patch, HIVE-22805.patch
>
>
> The following queries throw an "Not supported" Exception:
> {code}
> DROP TABLE IF EXISTS cond_vector;
> CREATE TABLE cond_vector(a STRING) STORED AS ORC;
> INSERT OVERWRITE TABLE cond_vector VALUES("a/b");
> set hive.fetch.task.conversion=minimal;
> set hive.execution.mode=container;
> SELECT IF(1=1, MAP("a","b"), NULL) FROM cond_vector;
> EXPLAIN VECTORIZATION DETAIL SELECT IF(1=1, MAP("Mathematics","78"), NULL)
> FROM cond_vector;
> SELECT IF(1=1, ARRAY("c", "d"), NULL) FROM cond_vector;
> EXPLAIN VECTORIZATION DETAIL SELECT IF(1=1, ARRAY("a", "b"), NULL) FROM
> cond_vector;
> {code}
> One example of the Exception (maybe not exactly this on 4.0.0 branch, but
> close enough to check):
> {code}
> Caused by: java.lang.RuntimeException: Not supported
> at
> org.apache.hadoop.hive.ql.exec.vector.ListColumnVector.copySelected(ListColumnVector.java:161)
> at
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprNull.evaluate(IfExprCondExprNull.java:87)
> at
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprCondExpr.evaluate(IfExprCondExprCondExpr.java:95)
> at
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprBase.conditionalEvaluate(IfExprCondExprBase.java:68)
> at
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprCondExpr.evaluate(IfExprCondExprCondExpr.java:110)
> at
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprCondExpr.evaluate(IfExprCondExprCondExpr.java:95)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.process(VectorSelectOperator.java:146)
>
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-10362) Support Type check/conversion in dynamic partition column
[ https://issues.apache.org/jira/browse/HIVE-10362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karen Coppage updated HIVE-10362: - Status: Open (was: Patch Available) > Support Type check/conversion in dynamic partition column > - > > Key: HIVE-10362 > URL: https://issues.apache.org/jira/browse/HIVE-10362 > Project: Hive > Issue Type: Improvement > Components: Query Processor, Types >Affects Versions: 3.0.0, 1.0.0, 4.0.0 >Reporter: Chaoyu Tang >Assignee: Karen Coppage >Priority: Major > Labels: breaking_change > Fix For: 4.0.0 > > Attachments: HIVE-10362.01.patch, HIVE-10362.02.patch, > HIVE-10362.03.patch, HIVE-10362.04.patch, HIVE-10362.04.patch, > HIVE-10362.06.patch, HIVE-10362.07.patch, HIVE-10362.07.patch, > HIVE-10362.08.patch, HIVE-10362.09.patch, HIVE-10362.10.patch > > > There are quite a lot of issues associated with the non-noramlized or > type-mismatched values for partition column. Hive has many ways to introduce > such problematic data. HIVE-10307 mainly provides the support to type > check/convert/normalize the partition column value in static partition > specification. This JIRA tries to deal with the partition column type in > dynamic partition insert. Currently any data can be inserted as a partition > column value as long as it is quoted as a string. For example, > create table dynparttypechecknum (key int, value string) partitioned by (part > int); > insert into dynparttypechecknum partition (part) select key, value, '1' > from src limit 1; > show partitions dynparttypechecknum; > -- > part=1 > The partition column value is non-normalized int 1. It causes some > unnecessary problems such as integer partition column JDO filter pushdown > (see HIVE-6052) and others like HIVE-10210. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-10362) Support Type check/conversion in dynamic partition column
[ https://issues.apache.org/jira/browse/HIVE-10362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karen Coppage updated HIVE-10362: - Attachment: HIVE-10362.10.patch Status: Patch Available (was: Open) > Support Type check/conversion in dynamic partition column > - > > Key: HIVE-10362 > URL: https://issues.apache.org/jira/browse/HIVE-10362 > Project: Hive > Issue Type: Improvement > Components: Query Processor, Types >Affects Versions: 3.0.0, 1.0.0, 4.0.0 >Reporter: Chaoyu Tang >Assignee: Karen Coppage >Priority: Major > Labels: breaking_change > Fix For: 4.0.0 > > Attachments: HIVE-10362.01.patch, HIVE-10362.02.patch, > HIVE-10362.03.patch, HIVE-10362.04.patch, HIVE-10362.04.patch, > HIVE-10362.06.patch, HIVE-10362.07.patch, HIVE-10362.07.patch, > HIVE-10362.08.patch, HIVE-10362.09.patch, HIVE-10362.10.patch > > > There are quite a lot of issues associated with the non-noramlized or > type-mismatched values for partition column. Hive has many ways to introduce > such problematic data. HIVE-10307 mainly provides the support to type > check/convert/normalize the partition column value in static partition > specification. This JIRA tries to deal with the partition column type in > dynamic partition insert. Currently any data can be inserted as a partition > column value as long as it is quoted as a string. For example, > create table dynparttypechecknum (key int, value string) partitioned by (part > int); > insert into dynparttypechecknum partition (part) select key, value, '1' > from src limit 1; > show partitions dynparttypechecknum; > -- > part=1 > The partition column value is non-normalized int 1. It causes some > unnecessary problems such as integer partition column JDO filter pushdown > (see HIVE-6052) and others like HIVE-10210. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22825) Reduce directory lookup cost for acid tables
[
https://issues.apache.org/jira/browse/HIVE-22825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17030607#comment-17030607
]
Hive QA commented on HIVE-22825:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12992670/HIVE-22825.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17965 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[timestamptz_2]
(batchId=91)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20474/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20474/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20474/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12992670 - PreCommit-HIVE-Build
> Reduce directory lookup cost for acid tables
>
>
> Key: HIVE-22825
> URL: https://issues.apache.org/jira/browse/HIVE-22825
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Reporter: Rajesh Balamohan
>Priority: Minor
> Attachments: HIVE-22825.1.patch, HIVE-22825.2.patch
>
>
> With objectstores, directory lookup costs are expensive. For acid tables, it
> would be good to have a directory cache to reduce number of lookup calls.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22825) Reduce directory lookup cost for acid tables
[
https://issues.apache.org/jira/browse/HIVE-22825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17030583#comment-17030583
]
Hive QA commented on HIVE-22825:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
29s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
39s{color} | {color:blue} common in master has 63 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
6s{color} | {color:blue} ql in master has 1533 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
15s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
31s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
45s{color} | {color:red} ql: The patch generated 3 new + 315 unchanged - 0
fixed = 318 total (was 315) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 30m 36s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20474/dev-support/hive-personality.sh
|
| git revision | master / 2a66c3f |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20474/yetus/diff-checkstyle-ql.txt
|
| modules | C: common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20474/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Reduce directory lookup cost for acid tables
>
>
> Key: HIVE-22825
> URL: https://issues.apache.org/jira/browse/HIVE-22825
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Reporter: Rajesh Balamohan
>Priority: Minor
> Attachments: HIVE-22825.1.patch, HIVE-22825.2.patch
>
>
> With objectstores, directory lookup costs are expensive. For acid tables, it
> would be good to have a directory cache to reduce number of lookup calls.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-22409) Logging: Implement QueryID Based Hive Logging
[
https://issues.apache.org/jira/browse/HIVE-22409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Syed Shameerur Rahman reassigned HIVE-22409:
Assignee: (was: Syed Shameerur Rahman)
> Logging: Implement QueryID Based Hive Logging
> -
>
> Key: HIVE-22409
> URL: https://issues.apache.org/jira/browse/HIVE-22409
> Project: Hive
> Issue Type: Improvement
> Components: Logging
>Reporter: Syed Shameerur Rahman
>Priority: Minor
>
> Currently all the hive logs are logged in
> ${sys:hive.log.dir}/${sys:hive.log.file} which is basically a single log
> file. Over the time it becomes tedious to search for logs since multiple hive
> query logs are logged into single log file.
> Hence we propose a queryID based hive logging where logs of different queries
> are logged into a separate log file based on their queryID.
> CC [~prasanth_j] [~gopalv] [~sseth]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22740) Test failing tests with return path
[ https://issues.apache.org/jira/browse/HIVE-22740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22740: -- Attachment: HIVE-22740.03.patch > Test failing tests with return path > > > Key: HIVE-22740 > URL: https://issues.apache.org/jira/browse/HIVE-22740 > Project: Hive > Issue Type: Bug >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Attachments: HIVE-22740.01.patch, HIVE-22740.02.patch, > HIVE-22740.03.patch > > > This is a dummy Jira for testing which tests are still failing with return > path enabled. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22740) Test failing tests with return path
[ https://issues.apache.org/jira/browse/HIVE-22740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22740: -- Attachment: (was: HIVE-22740.03.patch) > Test failing tests with return path > > > Key: HIVE-22740 > URL: https://issues.apache.org/jira/browse/HIVE-22740 > Project: Hive > Issue Type: Bug >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Attachments: HIVE-22740.01.patch, HIVE-22740.02.patch > > > This is a dummy Jira for testing which tests are still failing with return > path enabled. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22748) Remove disabling of IdentityProjectRemover transformation for Return Path
[ https://issues.apache.org/jira/browse/HIVE-22748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22748: -- Attachment: HIVE-22748.03.patch > Remove disabling of IdentityProjectRemover transformation for Return Path > - > > Key: HIVE-22748 > URL: https://issues.apache.org/jira/browse/HIVE-22748 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22748.01.patch, HIVE-22748.03.patch > > > For some reason IdentityProjectRemover transformation was not enabled for > Return Path. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22736) Support replication across multiple encryption zones
[ https://issues.apache.org/jira/browse/HIVE-22736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-22736: --- Status: In Progress (was: Patch Available) > Support replication across multiple encryption zones > > > Key: HIVE-22736 > URL: https://issues.apache.org/jira/browse/HIVE-22736 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch > > Time Spent: 3h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22736) Support replication across multiple encryption zones
[ https://issues.apache.org/jira/browse/HIVE-22736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-22736: --- Attachment: HIVE-22736.patch Status: Patch Available (was: In Progress) > Support replication across multiple encryption zones > > > Key: HIVE-22736 > URL: https://issues.apache.org/jira/browse/HIVE-22736 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch > > Time Spent: 3h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22736) Support replication across multiple encryption zones
[
https://issues.apache.org/jira/browse/HIVE-22736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17030557#comment-17030557
]
Hive QA commented on HIVE-22736:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12992678/HIVE-22736.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20473/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20473/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20473/
Messages:
{noformat}
This message was trimmed, see log for full details
error: patch failed:
standalone-metastore/metastore-common/src/main/thrift/hive_metastore.thrift:1155
Falling back to three-way merge...
Applied patch to
'standalone-metastore/metastore-common/src/main/thrift/hive_metastore.thrift'
cleanly.
error: patch failed:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/AuthFactory.java:24
Falling back to three-way merge...
Applied patch to
'standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/AuthFactory.java'
cleanly.
error: patch failed:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java:22
Falling back to three-way merge...
Applied patch to
'standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java'
cleanly.
error: patch failed:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java:17
Falling back to three-way merge...
Applied patch to
'standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java'
cleanly.
error: patch failed:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java:18
Falling back to three-way merge...
Applied patch to
'standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java'
cleanly.
error: patch failed:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/MetastoreDirectSqlUtils.java:20
Falling back to three-way merge...
Applied patch to
'standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/MetastoreDirectSqlUtils.java'
cleanly.
error: patch failed:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java:18
Falling back to three-way merge...
Applied patch to
'standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java'
cleanly.
error: patch failed:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/PersistenceManagerProvider.java:20
Falling back to three-way merge...
Applied patch to
'standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/PersistenceManagerProvider.java'
cleanly.
error: patch failed:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/RawStore.java:1820
Falling back to three-way merge...
Applied patch to
'standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/RawStore.java'
cleanly.
error: patch failed:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/RawStoreProxy.java:26
Falling back to three-way merge...
Applied patch to
'standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/RawStoreProxy.java'
cleanly.
error: patch failed:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/RetryingHMSHandler.java:25
Falling back to three-way merge...
Applied patch to
'standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/RetryingHMSHandler.java'
cleanly.
error: patch failed:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ScheduledQueryExecutionsMaintTask.java:22
Falling back to three-way merge...
Applied patch to
'standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ScheduledQueryExecutionsMaintTask.java'
cleanly.
error: patch failed:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/cache/CacheUtils.java:74
Falling back to three-way merge...
Applied patch to
'standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/cache/CacheUtils.java'
cleanly.
error: patch failed:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java:35
Falling back to three-way merge...
Applied patch to
'standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java'
cleanly.
error: patch failed:
[jira] [Commented] (HIVE-22808) HiveRelFieldTrimmer does not handle HiveTableFunctionScan
[
https://issues.apache.org/jira/browse/HIVE-22808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17030554#comment-17030554
]
Hive QA commented on HIVE-22808:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12992656/HIVE-22808.4.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17965 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[orc_merge6]
(batchId=198)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20472/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20472/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20472/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12992656 - PreCommit-HIVE-Build
> HiveRelFieldTrimmer does not handle HiveTableFunctionScan
> -
>
> Key: HIVE-22808
> URL: https://issues.apache.org/jira/browse/HIVE-22808
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22808.1.patch, HIVE-22808.2.patch,
> HIVE-22808.2.patch, HIVE-22808.3.patch, HIVE-22808.4.patch
>
>
> *Repro*
> {code:java}
> CREATE TABLE table_16 (
> timestamp_col_19timestamp,
> timestamp_col_29timestamp,
> int_col_27 int,
> int_col_39 int,
> boolean_col_18 boolean,
> varchar0045_col_23 varchar(45)
> );
> CREATE TABLE table_7 (
> int_col_10 int,
> bigint_col_3bigint
> );
> CREATE TABLE table_10 (
> boolean_col_8 boolean,
> boolean_col_16 boolean,
> timestamp_col_5 timestamp,
> timestamp_col_15timestamp,
> timestamp_col_30timestamp,
> decimal3825_col_26 decimal(38, 25),
> smallint_col_9 smallint,
> int_col_18 int
> );
> explain cbo
> SELECT
> DISTINCT COALESCE(a4.timestamp_col_15, IF(a4.boolean_col_16,
> a4.timestamp_col_30, a4.timestamp_col_5)) AS timestamp_col
> FROM table_7 a3
> RIGHT JOIN table_10 a4
> WHERE (a3.bigint_col_3) >= (a4.int_col_18)
> INTERSECT ALL
> SELECT
> COALESCE(LEAST(
> COALESCE(a1.timestamp_col_19, CAST('2010-03-29 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2014-08-16 00:00:00' AS
> TIMESTAMP))
> ),
> GREATEST(COALESCE(a1.timestamp_col_19, CAST('2013-07-01 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2028-06-18 00:00:00' AS
> TIMESTAMP)))
> ) AS timestamp_col
> FROM table_16 a1
> GROUP BY COALESCE(LEAST(
> COALESCE(a1.timestamp_col_19, CAST('2010-03-29 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2014-08-16 00:00:00' AS
> TIMESTAMP))
> ),
> GREATEST(
> COALESCE(a1.timestamp_col_19, CAST('2013-07-01 00:00:00' AS
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2028-06-18 00:00:00' AS
> TIMESTAMP)))
> );
> {code}
> CBO Plan contains unnecessary columns or all columns from a table in
> projections like:
> {code:java}
> HiveProject(int_col_10=[$0], bigint_col_3=[$1],
> BLOCK__OFFSET__INSIDE__FILE=[$2], INPUT__FILE__NAME=[$3],
> CAST=[CAST($4):RecordType(BIGINT writeid, INTEGER bucketid, BIGINT rowid)])
> {code}
> *Cause*
> The plan contains a HiveTableFunctionScan operator:
> {code:java}
> HiveTableFunctionScan(invocation=[replicate_rows($0, $1)],
> rowType=[RecordType(BIGINT $f0, TIMESTAMP(9) $f1)])
> {code}
> HiveTableFunctionScan is not handled by HiveRelFieldTrimmer nor
> RelFieldTrimmer which suppose to remove unused columns in the
> CalcitePlanner.applyPreJoinOrderingTransforms(...) phase. The whole subtree
> rooted from HiveTableFunctionScan is ignored.
> Whole plan:
> {code:java}
> CBO PLAN:
> HiveProject($f0=[$1])
> HiveTableFunctionScan(invocation=[replicate_rows($0, $1)],
> rowType=[RecordType(BIGINT $f0, TIMESTAMP(9) $f1)])
> HiveProject($f0=[$2], $f1=[$0])
> HiveFilter(condition=[=($1, 2)])
> HiveAggregate(group=[{0}], agg#0=[count($1)], agg#1=[min($1)])
> HiveProject($f0=[$0], $f1=[$1])
> HiveUnion(all=[true])
> HiveProject($f0=[$0], $f1=[$1])
> HiveAggregate(group=[{0}], agg#0=[count()])
> HiveProject($f0=[$0])
>
[jira] [Commented] (HIVE-22805) Vectorization with conditional array or map is not implemented and throws an error
[
https://issues.apache.org/jira/browse/HIVE-22805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17030547#comment-17030547
]
Peter Vary commented on HIVE-22805:
---
[~rameshkumar], [~mustafaiman]: Could you please review?
Thanks,
Peter
> Vectorization with conditional array or map is not implemented and throws an
> error
> --
>
> Key: HIVE-22805
> URL: https://issues.apache.org/jira/browse/HIVE-22805
> Project: Hive
> Issue Type: Bug
> Components: Vectorization
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-22805.2.patch, HIVE-22805.3.patch, HIVE-22805.patch
>
>
> The following queries throw an "Not supported" Exception:
> {code}
> DROP TABLE IF EXISTS cond_vector;
> CREATE TABLE cond_vector(a STRING) STORED AS ORC;
> INSERT OVERWRITE TABLE cond_vector VALUES("a/b");
> set hive.fetch.task.conversion=minimal;
> set hive.execution.mode=container;
> SELECT IF(1=1, MAP("a","b"), NULL) FROM cond_vector;
> EXPLAIN VECTORIZATION DETAIL SELECT IF(1=1, MAP("Mathematics","78"), NULL)
> FROM cond_vector;
> SELECT IF(1=1, ARRAY("c", "d"), NULL) FROM cond_vector;
> EXPLAIN VECTORIZATION DETAIL SELECT IF(1=1, ARRAY("a", "b"), NULL) FROM
> cond_vector;
> {code}
> One example of the Exception (maybe not exactly this on 4.0.0 branch, but
> close enough to check):
> {code}
> Caused by: java.lang.RuntimeException: Not supported
> at
> org.apache.hadoop.hive.ql.exec.vector.ListColumnVector.copySelected(ListColumnVector.java:161)
> at
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprNull.evaluate(IfExprCondExprNull.java:87)
> at
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprCondExpr.evaluate(IfExprCondExprCondExpr.java:95)
> at
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprBase.conditionalEvaluate(IfExprCondExprBase.java:68)
> at
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprCondExpr.evaluate(IfExprCondExprCondExpr.java:110)
> at
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprCondExpr.evaluate(IfExprCondExprCondExpr.java:95)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.process(VectorSelectOperator.java:146)
>
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22736) Support replication across multiple encryption zones
[ https://issues.apache.org/jira/browse/HIVE-22736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-22736: --- Attachment: HIVE-22736.patch Status: Patch Available (was: In Progress) > Support replication across multiple encryption zones > > > Key: HIVE-22736 > URL: https://issues.apache.org/jira/browse/HIVE-22736 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch > > Time Spent: 3h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22736) Support replication across multiple encryption zones
[ https://issues.apache.org/jira/browse/HIVE-22736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-22736: --- Status: In Progress (was: Patch Available) > Support replication across multiple encryption zones > > > Key: HIVE-22736 > URL: https://issues.apache.org/jira/browse/HIVE-22736 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch > > Time Spent: 3h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22825) Reduce directory lookup cost for acid tables
[ https://issues.apache.org/jira/browse/HIVE-22825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17030504#comment-17030504 ] Peter Vary commented on HIVE-22825: --- +1 Good start for caching > Reduce directory lookup cost for acid tables > > > Key: HIVE-22825 > URL: https://issues.apache.org/jira/browse/HIVE-22825 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Rajesh Balamohan >Priority: Minor > Attachments: HIVE-22825.1.patch, HIVE-22825.2.patch > > > With objectstores, directory lookup costs are expensive. For acid tables, it > would be good to have a directory cache to reduce number of lookup calls. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22808) HiveRelFieldTrimmer does not handle HiveTableFunctionScan
[
https://issues.apache.org/jira/browse/HIVE-22808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17030499#comment-17030499
]
Hive QA commented on HIVE-22808:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
13s{color} | {color:blue} ql in master has 1533 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
43s{color} | {color:red} ql: The patch generated 1 new + 3 unchanged - 0 fixed
= 4 total (was 3) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 26m 54s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20472/dev-support/hive-personality.sh
|
| git revision | master / 2a66c3f |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20472/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20472/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> HiveRelFieldTrimmer does not handle HiveTableFunctionScan
> -
>
> Key: HIVE-22808
> URL: https://issues.apache.org/jira/browse/HIVE-22808
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22808.1.patch, HIVE-22808.2.patch,
> HIVE-22808.2.patch, HIVE-22808.3.patch, HIVE-22808.4.patch
>
>
> *Repro*
> {code:java}
> CREATE TABLE table_16 (
> timestamp_col_19timestamp,
> timestamp_col_29timestamp,
> int_col_27 int,
> int_col_39 int,
> boolean_col_18 boolean,
> varchar0045_col_23 varchar(45)
> );
> CREATE TABLE table_7 (
> int_col_10 int,
> bigint_col_3bigint
> );
> CREATE TABLE table_10 (
> boolean_col_8 boolean,
> boolean_col_16 boolean,
> timestamp_col_5 timestamp,
> timestamp_col_15timestamp,
> timestamp_col_30timestamp,
> decimal3825_col_26 decimal(38, 25),
> smallint_col_9 smallint,
> int_col_18 int
> );
> explain cbo
> SELECT
> DISTINCT COALESCE(a4.timestamp_col_15, IF(a4.boolean_col_16,
> a4.timestamp_col_30, a4.timestamp_col_5)) AS timestamp_col
> FROM table_7 a3
> RIGHT JOIN table_10 a4
> WHERE (a3.bigint_col_3) >= (a4.int_col_18)
> INTERSECT ALL
> SELECT
> COALESCE(LEAST(
> COALESCE(a1.timestamp_col_19, CAST('2010-03-29 00:00:00' AS
> TIMESTAMP)),
>
[jira] [Updated] (HIVE-22736) Support replication across multiple encryption zones
[ https://issues.apache.org/jira/browse/HIVE-22736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-22736: --- Attachment: HIVE-22736.patch Status: Patch Available (was: In Progress) > Support replication across multiple encryption zones > > > Key: HIVE-22736 > URL: https://issues.apache.org/jira/browse/HIVE-22736 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch > > Time Spent: 3h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22783) Add test for HIVE-22366
[
https://issues.apache.org/jira/browse/HIVE-22783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17030471#comment-17030471
]
Hive QA commented on HIVE-22783:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12992644/HIVE-22783.1.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17965 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.parse.TestMetaStoreEventListenerInRepl.org.apache.hadoop.hive.ql.parse.TestMetaStoreEventListenerInRepl
(batchId=279)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20471/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20471/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20471/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12992644 - PreCommit-HIVE-Build
> Add test for HIVE-22366
> ---
>
> Key: HIVE-22783
> URL: https://issues.apache.org/jira/browse/HIVE-22783
> Project: Hive
> Issue Type: Sub-task
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-22783.1.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22821) Add necessary endpoints for proactive cache eviction
[ https://issues.apache.org/jira/browse/HIVE-22821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ádám Szita updated HIVE-22821: -- Attachment: HIVE-22821.2.patch > Add necessary endpoints for proactive cache eviction > > > Key: HIVE-22821 > URL: https://issues.apache.org/jira/browse/HIVE-22821 > Project: Hive > Issue Type: Sub-task > Components: llap >Reporter: Ádám Szita >Assignee: Ádám Szita >Priority: Major > Attachments: HIVE-22821.0.patch, HIVE-22821.1.patch, > HIVE-22821.2.patch > > > Implement the parts required for iHS2 -> LLAP daemons communication: > * protobuf message schema and endpoints > * Hive configuration > * for use cases: > ** dropping db > ** dropping table > ** dropping partition from a table -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22825) Reduce directory lookup cost for acid tables
[ https://issues.apache.org/jira/browse/HIVE-22825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HIVE-22825: Attachment: HIVE-22825.2.patch > Reduce directory lookup cost for acid tables > > > Key: HIVE-22825 > URL: https://issues.apache.org/jira/browse/HIVE-22825 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Rajesh Balamohan >Priority: Minor > Attachments: HIVE-22825.1.patch, HIVE-22825.2.patch > > > With objectstores, directory lookup costs are expensive. For acid tables, it > would be good to have a directory cache to reduce number of lookup calls. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22783) Add test for HIVE-22366
[
https://issues.apache.org/jira/browse/HIVE-22783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17030438#comment-17030438
]
Hive QA commented on HIVE-22783:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
54s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
37s{color} | {color:blue} ql in master has 1533 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
47s{color} | {color:red} ql: The patch generated 36 new + 0 unchanged - 0 fixed
= 36 total (was 0) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
17s{color} | {color:red} The patch generated 2 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 32m 54s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20471/dev-support/hive-personality.sh
|
| git revision | master / 2a66c3f |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20471/yetus/diff-checkstyle-ql.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20471/yetus/patch-asflicense-problems.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20471/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Add test for HIVE-22366
> ---
>
> Key: HIVE-22783
> URL: https://issues.apache.org/jira/browse/HIVE-22783
> Project: Hive
> Issue Type: Sub-task
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-22783.1.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22589) Add storage support for ProlepticCalendar in ORC, Parquet, and Avro
[
https://issues.apache.org/jira/browse/HIVE-22589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17030430#comment-17030430
]
Hive QA commented on HIVE-22589:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
45s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m
54s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 5m
7s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
26s{color} | {color:blue} storage-api in master has 58 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
59s{color} | {color:blue} standalone-metastore/metastore-common in master has
35 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
36s{color} | {color:blue} common in master has 63 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
48s{color} | {color:blue} serde in master has 197 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
15s{color} | {color:blue} ql in master has 1533 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
49s{color} | {color:blue} llap-server in master has 90 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 11m
7s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
31s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 12m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 10m
42s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
13s{color} | {color:red} standalone-metastore/metastore-common: The patch
generated 1 new + 1 unchanged - 0 fixed = 2 total (was 1) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
18s{color} | {color:red} common: The patch generated 4 new + 370 unchanged - 0
fixed = 374 total (was 370) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
18s{color} | {color:red} serde: The patch generated 2 new + 105 unchanged - 3
fixed = 107 total (was 108) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
1s{color} | {color:red} ql: The patch generated 27 new + 1565 unchanged - 11
fixed = 1592 total (was 1576) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
16s{color} | {color:red} llap-server: The patch generated 3 new + 185 unchanged
- 1 fixed = 188 total (was 186) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m
42s{color} | {color:red} root: The patch generated 37 new + 2232 unchanged - 15
fixed = 2269 total (was 2247) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
23s{color} | {color:red} patch/storage-api cannot run setBugDatabaseInfo from
findbugs {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
26s{color} | {color:red} patch/standalone-metastore/metastore-common cannot run
setBugDatabaseInfo from findbugs {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
20s{color} | {color:red} patch/common cannot run setBugDatabaseInfo from
findbugs {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
22s{color} | {color:red} patch/serde
