[jira] [Commented] (HIVE-22315) Support Decimal64 column division with decimal64 scalar
[
https://issues.apache.org/jira/browse/HIVE-22315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16955762#comment-16955762
]
Hive QA commented on HIVE-22315:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12983566/HIVE-22315.6.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17547 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1 (batchId=274)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19081/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19081/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19081/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12983566 - PreCommit-HIVE-Build
> Support Decimal64 column division with decimal64 scalar
> ---
>
> Key: HIVE-22315
> URL: https://issues.apache.org/jira/browse/HIVE-22315
> Project: Hive
> Issue Type: Bug
>Reporter: Ramesh Kumar Thangarajan
>Assignee: Ramesh Kumar Thangarajan
>Priority: Major
> Attachments: HIVE-22315.1.patch, HIVE-22315.2.patch,
> HIVE-22315.3.patch, HIVE-22315.4.patch, HIVE-22315.6.patch
>
>
> Currently division operation is not supported for Decimal64 column. This Jira
> will take care of supporting decimal64 column division with a decimal64
> scalar.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22315) Support Decimal64 column division with decimal64 scalar
[
https://issues.apache.org/jira/browse/HIVE-22315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16955761#comment-16955761
]
Hive QA commented on HIVE-22315:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
50s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
15s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
5s{color} | {color:blue} ql in master has 1547 extant Findbugs warnings.
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
7s{color} | {color:red} branch/itests/hive-jmh cannot run convertXmlToText from
findbugs {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 8m
8s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
25s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 8m
2s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
43s{color} | {color:red} ql: The patch generated 1 new + 386 unchanged - 1
fixed = 387 total (was 387) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m
6s{color} | {color:red} root: The patch generated 5 new + 731 unchanged - 1
fixed = 736 total (was 732) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
10s{color} | {color:red} itests/hive-jmh: The patch generated 4 new + 15
unchanged - 0 fixed = 19 total (was 15) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
8s{color} | {color:red} patch/itests/hive-jmh cannot run convertXmlToText from
findbugs {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 7m
53s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
14s{color} | {color:red} The patch generated 3 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 73m 21s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19081/dev-support/hive-personality.sh
|
| git revision | master / c1e5fa3 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19081/yetus/branch-findbugs-itests_hive-jmh.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19081/yetus/diff-checkstyle-ql.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19081/yetus/diff-checkstyle-root.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19081/yetus/diff-checkstyle-itests_hive-jmh.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19081/yetus/patch-findbugs-itests_hive-jmh.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19081/yetus/patch-asflicense-problems.txt
|
| modules | C: vector-code-gen ql . itests itests/hive-jmh U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19081/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Support Decimal64 column division with decimal64 scalar
>
[jira] [Comment Edited] (HIVE-22374) Upgrade commons-compress version to 1.19
[
https://issues.apache.org/jira/browse/HIVE-22374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16955748#comment-16955748
]
Sumin Byeon edited comment on HIVE-22374 at 10/21/19 5:31 AM:
--
This is my first time submitting a patch to an Apache project, and I apologize
in advance for not strictly adhering to all the rules that I may have
overlooked.
I would like to figure out 'Affects Version/s' and 'Fix Version/s' (and other
required labels), and it would be great if someone could point me to a right
direction. A wiki page, a guideline, or anything like that would be a great
help. Thanks.
was (Author: shortbread):
This is my first time submitting a patch to an Apache project, and I apologize
in advance for not strictly following all the rules that I'm not accustomed to.
I would like to figure out 'Affects Version/s' and 'Fix Version/s' (and other
required labels), and it would be great if someone could point me to a right
direction. A wiki page, a guideline, or anything like that would be a great
help. Thanks.
> Upgrade commons-compress version to 1.19
>
>
> Key: HIVE-22374
> URL: https://issues.apache.org/jira/browse/HIVE-22374
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Sumin Byeon
>Priority: Critical
> Attachments: HIVE-22374.patch
>
>
> As described in
> [CVE-2019-12402|https://nvd.nist.gov/vuln/detail/CVE-2019-12402],
> {{commons-compress:1.18}} has an issue where certain inputs may cause an
> infinite loop which leads to a denial of service attack.
> This patch simply upgrades {{common-compress}} versions from 1.18 to 1.19
> which is the latest minor version at the date of filing this issue ([Maven
> repo|https://mvnrepository.com/artifact/org.apache.commons/commons-compress]).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (HIVE-22374) Upgrade commons-compress version to 1.19
[
https://issues.apache.org/jira/browse/HIVE-22374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16955748#comment-16955748
]
Sumin Byeon edited comment on HIVE-22374 at 10/21/19 4:49 AM:
--
This is my first time submitting a patch to an Apache project, and I apologize
in advance for not strictly following all the rules that I'm not accustomed to.
I would like to figure out 'Affects Version/s' and 'Fix Version/s' (and other
required labels), and it would be great if someone could point me to a right
direction. A wiki page, a guideline, or anything like that would be a great
help. Thanks.
was (Author: shortbread):
This is my first time submitting a patch to an Apache project, and I apologize
in advance for not strictly following all rules that I'm not accustomed to.
I would like to figure out 'Affects Version/s' and 'Fix Version/s' (and other
required labels), and it would be great if someone could point me to a right
direction. A wiki page, a guideline, or anything like that would be a great
help. Thanks.
> Upgrade commons-compress version to 1.19
>
>
> Key: HIVE-22374
> URL: https://issues.apache.org/jira/browse/HIVE-22374
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Sumin Byeon
>Priority: Critical
> Attachments: HIVE-22374.patch
>
>
> As described in
> [CVE-2019-12402|https://nvd.nist.gov/vuln/detail/CVE-2019-12402],
> {{commons-compress:1.18}} has an issue where certain inputs may cause an
> infinite loop which leads to a denial of service attack.
> This patch simply upgrades {{common-compress}} versions from 1.18 to 1.19
> which is the latest minor version at the date of filing this issue ([Maven
> repo|https://mvnrepository.com/artifact/org.apache.commons/commons-compress]).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (HIVE-22374) Upgrade commons-compress version to 1.19
[
https://issues.apache.org/jira/browse/HIVE-22374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16955748#comment-16955748
]
Sumin Byeon edited comment on HIVE-22374 at 10/21/19 4:49 AM:
--
This is my first time submitting a patch to an Apache project, and I apologize
in advance for not strictly following all rules that I'm not accustomed to.
I would like to figure out 'Affects Version/s' and 'Fix Version/s' (and other
required labels), and it would be great if someone could point me to a right
direction. A wiki page, a guideline, or anything like that would be a great
help. Thanks.
was (Author: shortbread):
This is my first time submitting a patch to an Apache project, and I apologize
in advance for not strictly following all rules that I'm not aware of yet.
I would like to figure out 'Affects Version/s' and 'Fix Version/s' (and other
required labels), and it would be great if someone could point me to a right
direction. A wiki page, a guideline, or anything like that would be a great
help. Thanks.
> Upgrade commons-compress version to 1.19
>
>
> Key: HIVE-22374
> URL: https://issues.apache.org/jira/browse/HIVE-22374
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Sumin Byeon
>Priority: Critical
> Attachments: HIVE-22374.patch
>
>
> As described in
> [CVE-2019-12402|https://nvd.nist.gov/vuln/detail/CVE-2019-12402],
> {{commons-compress:1.18}} has an issue where certain inputs may cause an
> infinite loop which leads to a denial of service attack.
> This patch simply upgrades {{common-compress}} versions from 1.18 to 1.19
> which is the latest minor version at the date of filing this issue ([Maven
> repo|https://mvnrepository.com/artifact/org.apache.commons/commons-compress]).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22374) Upgrade commons-compress version to 1.19
[
https://issues.apache.org/jira/browse/HIVE-22374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16955748#comment-16955748
]
Sumin Byeon commented on HIVE-22374:
This is my first time submitting a patch to an Apache project, and I apologize
in advance for not strictly following all rules that I'm not aware of yet.
I would like to figure out 'Affects Version/s' and 'Fix Version/s' (and other
required labels), and it would be great if someone could point me to a right
direction. A wiki page, a guideline, or anything like that would be a great
help. Thanks.
> Upgrade commons-compress version to 1.19
>
>
> Key: HIVE-22374
> URL: https://issues.apache.org/jira/browse/HIVE-22374
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Sumin Byeon
>Priority: Critical
> Attachments: HIVE-22374.patch
>
>
> As described in
> [CVE-2019-12402|https://nvd.nist.gov/vuln/detail/CVE-2019-12402],
> {{commons-compress:1.18}} has an issue where certain inputs may cause an
> infinite loop which leads to a denial of service attack.
> This patch simply upgrades {{common-compress}} versions from 1.18 to 1.19
> which is the latest minor version at the date of filing this issue ([Maven
> repo|https://mvnrepository.com/artifact/org.apache.commons/commons-compress]).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22240) Function percentile_cont fails when array parameter passed
[
https://issues.apache.org/jira/browse/HIVE-22240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22240:
--
Status: Open (was: Patch Available)
> Function percentile_cont fails when array parameter passed
> --
>
> Key: HIVE-22240
> URL: https://issues.apache.org/jira/browse/HIVE-22240
> Project: Hive
> Issue Type: Bug
> Components: UDF
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22240.1.patch, HIVE-22240.2.patch,
> HIVE-22240.3.patch, HIVE-22240.3.patch, HIVE-22240.4.patch,
> HIVE-22240.4.patch, HIVE-22240.4.patch, HIVE-22240.4.patch
>
>
> {code}
> SELECT
> percentile_cont(array(0.2, 0.5, 0.9)) WITHIN GROUP (ORDER BY value)
> FROM t_test;
> {code}
> hive.log:
> {code}
> 2019-09-24T21:00:43,203 ERROR [LocalJobRunner Map Task Executor #0]
> mr.ExecMapper: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
> Error while processing row
> at
> org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:573)
> at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:148)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
> at
> org.apache.hadoop.hive.ql.exec.mr.ExecMapRunner.run(ExecMapRunner.java:37)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:465)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:349)
> at
> org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:271)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.ClassCastException: java.util.ArrayList cannot be cast to
> org.apache.hadoop.hive.serde2.io.HiveDecimalWritable
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.process(GroupByOperator.java:793)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:888)
> at
> org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:95)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:888)
> at
> org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:128)
> at
> org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.forward(MapOperator.java:152)
> at
> org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:552)
> ... 11 more
> Caused by: java.lang.ClassCastException: java.util.ArrayList cannot be cast
> to org.apache.hadoop.hive.serde2.io.HiveDecimalWritable
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDAFPercentileCont$PercentileContEvaluator.iterate(GenericUDAFPercentileCont.java:259)
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator.aggregate(GenericUDAFEvaluator.java:214)
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.updateAggregations(GroupByOperator.java:639)
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.processHashAggr(GroupByOperator.java:814)
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.processKey(GroupByOperator.java:720)
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.process(GroupByOperator.java:788)
> ... 17 more
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22240) Function percentile_cont fails when array parameter passed
[
https://issues.apache.org/jira/browse/HIVE-22240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22240:
--
Attachment: HIVE-22240.4.patch
> Function percentile_cont fails when array parameter passed
> --
>
> Key: HIVE-22240
> URL: https://issues.apache.org/jira/browse/HIVE-22240
> Project: Hive
> Issue Type: Bug
> Components: UDF
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22240.1.patch, HIVE-22240.2.patch,
> HIVE-22240.3.patch, HIVE-22240.3.patch, HIVE-22240.4.patch,
> HIVE-22240.4.patch, HIVE-22240.4.patch, HIVE-22240.4.patch
>
>
> {code}
> SELECT
> percentile_cont(array(0.2, 0.5, 0.9)) WITHIN GROUP (ORDER BY value)
> FROM t_test;
> {code}
> hive.log:
> {code}
> 2019-09-24T21:00:43,203 ERROR [LocalJobRunner Map Task Executor #0]
> mr.ExecMapper: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
> Error while processing row
> at
> org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:573)
> at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:148)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
> at
> org.apache.hadoop.hive.ql.exec.mr.ExecMapRunner.run(ExecMapRunner.java:37)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:465)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:349)
> at
> org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:271)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.ClassCastException: java.util.ArrayList cannot be cast to
> org.apache.hadoop.hive.serde2.io.HiveDecimalWritable
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.process(GroupByOperator.java:793)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:888)
> at
> org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:95)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:888)
> at
> org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:128)
> at
> org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.forward(MapOperator.java:152)
> at
> org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:552)
> ... 11 more
> Caused by: java.lang.ClassCastException: java.util.ArrayList cannot be cast
> to org.apache.hadoop.hive.serde2.io.HiveDecimalWritable
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDAFPercentileCont$PercentileContEvaluator.iterate(GenericUDAFPercentileCont.java:259)
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator.aggregate(GenericUDAFEvaluator.java:214)
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.updateAggregations(GroupByOperator.java:639)
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.processHashAggr(GroupByOperator.java:814)
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.processKey(GroupByOperator.java:720)
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.process(GroupByOperator.java:788)
> ... 17 more
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22240) Function percentile_cont fails when array parameter passed
[
https://issues.apache.org/jira/browse/HIVE-22240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22240:
--
Status: Patch Available (was: Open)
> Function percentile_cont fails when array parameter passed
> --
>
> Key: HIVE-22240
> URL: https://issues.apache.org/jira/browse/HIVE-22240
> Project: Hive
> Issue Type: Bug
> Components: UDF
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22240.1.patch, HIVE-22240.2.patch,
> HIVE-22240.3.patch, HIVE-22240.3.patch, HIVE-22240.4.patch,
> HIVE-22240.4.patch, HIVE-22240.4.patch, HIVE-22240.4.patch
>
>
> {code}
> SELECT
> percentile_cont(array(0.2, 0.5, 0.9)) WITHIN GROUP (ORDER BY value)
> FROM t_test;
> {code}
> hive.log:
> {code}
> 2019-09-24T21:00:43,203 ERROR [LocalJobRunner Map Task Executor #0]
> mr.ExecMapper: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
> Error while processing row
> at
> org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:573)
> at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:148)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
> at
> org.apache.hadoop.hive.ql.exec.mr.ExecMapRunner.run(ExecMapRunner.java:37)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:465)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:349)
> at
> org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:271)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.ClassCastException: java.util.ArrayList cannot be cast to
> org.apache.hadoop.hive.serde2.io.HiveDecimalWritable
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.process(GroupByOperator.java:793)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:888)
> at
> org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:95)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:888)
> at
> org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:128)
> at
> org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.forward(MapOperator.java:152)
> at
> org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:552)
> ... 11 more
> Caused by: java.lang.ClassCastException: java.util.ArrayList cannot be cast
> to org.apache.hadoop.hive.serde2.io.HiveDecimalWritable
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDAFPercentileCont$PercentileContEvaluator.iterate(GenericUDAFPercentileCont.java:259)
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator.aggregate(GenericUDAFEvaluator.java:214)
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.updateAggregations(GroupByOperator.java:639)
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.processHashAggr(GroupByOperator.java:814)
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.processKey(GroupByOperator.java:720)
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.process(GroupByOperator.java:788)
> ... 17 more
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22346) Yetus is failing rat check
[
https://issues.apache.org/jira/browse/HIVE-22346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22346:
--
Status: Open (was: Patch Available)
> Yetus is failing rat check
> --
>
> Key: HIVE-22346
> URL: https://issues.apache.org/jira/browse/HIVE-22346
> Project: Hive
> Issue Type: Bug
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22346.1.patch, HIVE-22346.2.patch,
> HIVE-22346.2.patch, HIVE-22346.2.patch, HIVE-22346.2.patch, HIVE-22346.2.patch
>
>
> {code:java}
> Lines that start with ? in the ASF License report indicate files that do
> not have an Apache license header:
> !?
> /data/hiveptest/working/yetus_PreCommit-HIVE-Build-18996/standalone-metastore/metastore-server/src/test/resources/ldap/ad.example.com.ldif
> !?
> /data/hiveptest/working/yetus_PreCommit-HIVE-Build-18996/standalone-metastore/metastore-server/src/test/resources/ldap/example.com.ldif
> !?
> /data/hiveptest/working/yetus_PreCommit-HIVE-Build-18996/standalone-metastore/metastore-server/src/test/resources/ldap/microsoft.schema.ldif
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22346) Yetus is failing rat check
[
https://issues.apache.org/jira/browse/HIVE-22346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22346:
--
Status: Patch Available (was: Open)
> Yetus is failing rat check
> --
>
> Key: HIVE-22346
> URL: https://issues.apache.org/jira/browse/HIVE-22346
> Project: Hive
> Issue Type: Bug
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22346.1.patch, HIVE-22346.2.patch,
> HIVE-22346.2.patch, HIVE-22346.2.patch, HIVE-22346.2.patch, HIVE-22346.2.patch
>
>
> {code:java}
> Lines that start with ? in the ASF License report indicate files that do
> not have an Apache license header:
> !?
> /data/hiveptest/working/yetus_PreCommit-HIVE-Build-18996/standalone-metastore/metastore-server/src/test/resources/ldap/ad.example.com.ldif
> !?
> /data/hiveptest/working/yetus_PreCommit-HIVE-Build-18996/standalone-metastore/metastore-server/src/test/resources/ldap/example.com.ldif
> !?
> /data/hiveptest/working/yetus_PreCommit-HIVE-Build-18996/standalone-metastore/metastore-server/src/test/resources/ldap/microsoft.schema.ldif
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22374) Upgrade commons-compress version to 1.19
[
https://issues.apache.org/jira/browse/HIVE-22374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sumin Byeon updated HIVE-22374:
---
Attachment: HIVE-22374.patch
> Upgrade commons-compress version to 1.19
>
>
> Key: HIVE-22374
> URL: https://issues.apache.org/jira/browse/HIVE-22374
> Project: Hive
> Issue Type: Bug
>Reporter: Sumin Byeon
>Priority: Critical
> Attachments: HIVE-22374.patch
>
>
> As described in
> [CVE-2019-12402|https://nvd.nist.gov/vuln/detail/CVE-2019-12402],
> {{commons-compress:1.18}} has an issue where certain inputs may cause an
> infinite loop which leads to a denial of service attack.
> This patch simply upgrades {{common-compress}} versions from 1.18 to 1.19
> which is the latest minor version at the date of filing this issue ([Maven
> repo|https://mvnrepository.com/artifact/org.apache.commons/commons-compress]).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22374) Upgrade commons-compress version to 1.19
[
https://issues.apache.org/jira/browse/HIVE-22374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sumin Byeon updated HIVE-22374:
---
Component/s: Hive
> Upgrade commons-compress version to 1.19
>
>
> Key: HIVE-22374
> URL: https://issues.apache.org/jira/browse/HIVE-22374
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Sumin Byeon
>Priority: Critical
> Attachments: HIVE-22374.patch
>
>
> As described in
> [CVE-2019-12402|https://nvd.nist.gov/vuln/detail/CVE-2019-12402],
> {{commons-compress:1.18}} has an issue where certain inputs may cause an
> infinite loop which leads to a denial of service attack.
> This patch simply upgrades {{common-compress}} versions from 1.18 to 1.19
> which is the latest minor version at the date of filing this issue ([Maven
> repo|https://mvnrepository.com/artifact/org.apache.commons/commons-compress]).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22346) Yetus is failing rat check
[
https://issues.apache.org/jira/browse/HIVE-22346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22346:
--
Attachment: HIVE-22346.2.patch
> Yetus is failing rat check
> --
>
> Key: HIVE-22346
> URL: https://issues.apache.org/jira/browse/HIVE-22346
> Project: Hive
> Issue Type: Bug
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22346.1.patch, HIVE-22346.2.patch,
> HIVE-22346.2.patch, HIVE-22346.2.patch, HIVE-22346.2.patch, HIVE-22346.2.patch
>
>
> {code:java}
> Lines that start with ? in the ASF License report indicate files that do
> not have an Apache license header:
> !?
> /data/hiveptest/working/yetus_PreCommit-HIVE-Build-18996/standalone-metastore/metastore-server/src/test/resources/ldap/ad.example.com.ldif
> !?
> /data/hiveptest/working/yetus_PreCommit-HIVE-Build-18996/standalone-metastore/metastore-server/src/test/resources/ldap/example.com.ldif
> !?
> /data/hiveptest/working/yetus_PreCommit-HIVE-Build-18996/standalone-metastore/metastore-server/src/test/resources/ldap/microsoft.schema.ldif
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22315) Support Decimal64 column division with decimal64 scalar
[ https://issues.apache.org/jira/browse/HIVE-22315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ramesh Kumar Thangarajan updated HIVE-22315: Attachment: HIVE-22315.6.patch Status: Patch Available (was: Open) > Support Decimal64 column division with decimal64 scalar > --- > > Key: HIVE-22315 > URL: https://issues.apache.org/jira/browse/HIVE-22315 > Project: Hive > Issue Type: Bug >Reporter: Ramesh Kumar Thangarajan >Assignee: Ramesh Kumar Thangarajan >Priority: Major > Attachments: HIVE-22315.1.patch, HIVE-22315.2.patch, > HIVE-22315.3.patch, HIVE-22315.4.patch, HIVE-22315.6.patch > > > Currently division operation is not supported for Decimal64 column. This Jira > will take care of supporting decimal64 column division with a decimal64 > scalar. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22315) Support Decimal64 column division with decimal64 scalar
[ https://issues.apache.org/jira/browse/HIVE-22315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ramesh Kumar Thangarajan updated HIVE-22315: Status: Open (was: Patch Available) > Support Decimal64 column division with decimal64 scalar > --- > > Key: HIVE-22315 > URL: https://issues.apache.org/jira/browse/HIVE-22315 > Project: Hive > Issue Type: Bug >Reporter: Ramesh Kumar Thangarajan >Assignee: Ramesh Kumar Thangarajan >Priority: Major > Attachments: HIVE-22315.1.patch, HIVE-22315.2.patch, > HIVE-22315.3.patch, HIVE-22315.4.patch > > > Currently division operation is not supported for Decimal64 column. This Jira > will take care of supporting decimal64 column division with a decimal64 > scalar. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-21954) QTest: support for running qtests on various metastore DBs
[
https://issues.apache.org/jira/browse/HIVE-21954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16955645#comment-16955645
]
Hive QA commented on HIVE-21954:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12983554/HIVE-21954.03.patch
{color:green}SUCCESS:{color} +1 due to 11 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 17133 tests
executed
*Failed tests:*
{noformat}
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=299)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=299)
TestMinimrCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=299)
TestTezPerfCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=299)
TestTezPerfConstraintsCliDriver - did not produce a TEST-*.xml file (likely
timed out) (batchId=299)
org.apache.hadoop.hive.cli.TestSparkNegativeCliDriver.org.apache.hadoop.hive.cli.TestSparkNegativeCliDriver
(batchId=301)
org.apache.hadoop.hive.llap.security.TestLlapSignerImpl.testSigning
(batchId=364)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19080/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19080/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19080/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12983554 - PreCommit-HIVE-Build
> QTest: support for running qtests on various metastore DBs
> --
>
> Key: HIVE-21954
> URL: https://issues.apache.org/jira/browse/HIVE-21954
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore, Testing Infrastructure
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-21954.01.patch, HIVE-21954.02.patch,
> HIVE-21954.03.patch
>
>
> In HIVE-21940, a postgres metastore related issue has been fixed, and a local
> reproduction has been provided.
> {code}
> export QTEST_LEAVE_FILES=true
> docker kill metastore-test-postgres-install
> docker rm metastore-test-postgres-install
> cd standalone-metastore
> mvn verify -DskipITests=false -Dit.test=ITestPostgres#install -Dtest=nosuch
> -Dmetastore.itest.no.stop.container=true
> cd ..
> mvn test -Dtest.output.overwrite=true -Pitests,hadoop-2 -pl itests/qtest
> -Dtest=TestCliDriver -Dqfile=partition_params_postgres.q
> -Dhive.metastore.rawstore.impl=org.apache.hadoop.hive.metastore.ObjectStore
> {code}
> The problem with this solution is that data/conf/hive-site.xml has to be
> edited manually. My proposal is to introduce a property
> (-Dmetastore.db=postgres), which can take care of the parameters on the fly.
> 2 supported solutions could be:
> 1. simple parameters: -Dmetastore.db=postgres
> In this case, tests depend on settings from ITestPostgres class (password,
> db, etc.)
> 2. verbose but flexible parameters: [see hive-site.xml HIVE-21940's repro
> patch|https://issues.apache.org/jira/secure/attachment/12973534/HIVE-21940.repro.patch]
>
> In the first implementation, I would not start metastore db automatically
> (which is done be 'mvn verify ...'), but it's still under planning.
> In the long term, we should consider running this kind of tests in precommit
> phase, so maybe -Dmetastore.db=postgres could start metastore db
> automatically. Also we should consider running some qtests on various
> metastores. I would not pick randomly, but choose some "metastore-heavy" ones
> instead.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-21954) QTest: support for running qtests on various metastore DBs
[
https://issues.apache.org/jira/browse/HIVE-21954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16955644#comment-16955644
]
Hive QA commented on HIVE-21954:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
40s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
49s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m
2s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
13s{color} | {color:blue} standalone-metastore/metastore-server in master has
171 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
2s{color} | {color:blue} ql in master has 1547 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
39s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
46s{color} | {color:blue} itests/util in master has 53 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 10m
27s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
24s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 11m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 10m
58s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
23s{color} | {color:red} standalone-metastore: The patch generated 7 new + 50
unchanged - 6 fixed = 57 total (was 56) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
18s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 7 new + 50 unchanged - 6 fixed = 57 total (was 56) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m
0s{color} | {color:red} root: The patch generated 7 new + 104 unchanged - 6
fixed = 111 total (was 110) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
4s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 7m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 10m
25s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
15s{color} | {color:red} The patch generated 3 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 88m 31s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense xml javac javadoc findbugs checkstyle
compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19080/dev-support/hive-personality.sh
|
| git revision | master / c1e5fa3 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19080/yetus/diff-checkstyle-standalone-metastore.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19080/yetus/diff-checkstyle-standalone-metastore_metastore-server.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19080/yetus/diff-checkstyle-root.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19080/yetus/patch-asflicense-problems.txt
|
| modules | C:
[jira] [Updated] (HIVE-21954) QTest: support for running qtests on various metastore DBs
[
https://issues.apache.org/jira/browse/HIVE-21954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
László Bodor updated HIVE-21954:
Attachment: HIVE-21954.03.patch
> QTest: support for running qtests on various metastore DBs
> --
>
> Key: HIVE-21954
> URL: https://issues.apache.org/jira/browse/HIVE-21954
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore, Testing Infrastructure
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-21954.01.patch, HIVE-21954.02.patch,
> HIVE-21954.03.patch
>
>
> In HIVE-21940, a postgres metastore related issue has been fixed, and a local
> reproduction has been provided.
> {code}
> export QTEST_LEAVE_FILES=true
> docker kill metastore-test-postgres-install
> docker rm metastore-test-postgres-install
> cd standalone-metastore
> mvn verify -DskipITests=false -Dit.test=ITestPostgres#install -Dtest=nosuch
> -Dmetastore.itest.no.stop.container=true
> cd ..
> mvn test -Dtest.output.overwrite=true -Pitests,hadoop-2 -pl itests/qtest
> -Dtest=TestCliDriver -Dqfile=partition_params_postgres.q
> -Dhive.metastore.rawstore.impl=org.apache.hadoop.hive.metastore.ObjectStore
> {code}
> The problem with this solution is that data/conf/hive-site.xml has to be
> edited manually. My proposal is to introduce a property
> (-Dmetastore.db=postgres), which can take care of the parameters on the fly.
> 2 supported solutions could be:
> 1. simple parameters: -Dmetastore.db=postgres
> In this case, tests depend on settings from ITestPostgres class (password,
> db, etc.)
> 2. verbose but flexible parameters: [see hive-site.xml HIVE-21940's repro
> patch|https://issues.apache.org/jira/secure/attachment/12973534/HIVE-21940.repro.patch]
>
> In the first implementation, I would not start metastore db automatically
> (which is done be 'mvn verify ...'), but it's still under planning.
> In the long term, we should consider running this kind of tests in precommit
> phase, so maybe -Dmetastore.db=postgres could start metastore db
> automatically. Also we should consider running some qtests on various
> metastores. I would not pick randomly, but choose some "metastore-heavy" ones
> instead.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22311) Propagate min/max column values from statistics to the optimizer for timestamp type
[
https://issues.apache.org/jira/browse/HIVE-22311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16955610#comment-16955610
]
Hive QA commented on HIVE-22311:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12983553/HIVE-22311.01.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 12 failed/errored test(s), 17516 tests
executed
*Failed tests:*
{noformat}
TestMiniLlapLocalCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=184)
[vector_windowing_expressions.q,tez_union_group_by.q,materialized_view_rewrite_no_join_opt_2.q,vector_like_2.q,llap_acid.q,sqlmerge.q,schema_evol_orc_acid_part_update_llap_io.q,vector_windowing_gby.q,rfc5424_parser.q,lateral_view.q,vector_decimal_join.q,optimize_nullscan.q,vectorization_decimal_date.q,schema_evol_orc_nonvec_table_llap_io.q,udaf_all_keyword.q,acid_vectorization_original.q,tez_fsstat.q,vector_fullouter_mapjoin_1_optimized_passthru.q,vector_mapjoin_reduce.q,join_acid_non_acid.q,empty_join.q,auto_join21.q,materialized_view_partition_cluster.q,tez_input_counters.q,skiphf_aggr2.q,vector_groupby_sort_11.q,schema_evol_orc_nonvec_part_all_complex_llap_io.q,orc_ppd_timestamp.q,vector_decimal_1.q,tez_fixed_bucket_pruning.q]
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_orc_nonvec_part_all_primitive]
(batchId=187)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_orc_nonvec_part_all_primitive_llap_io]
(batchId=186)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_orc_vec_part_all_primitive]
(batchId=183)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_orc_vec_part_all_primitive_llap_io]
(batchId=168)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_text_nonvec_part_all_primitive]
(batchId=176)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_text_nonvec_part_all_primitive_llap_io]
(batchId=187)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_text_vec_part_all_primitive]
(batchId=178)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_text_vec_part_all_primitive_llap_io]
(batchId=183)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_text_vecrow_part_all_primitive]
(batchId=179)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_text_vecrow_part_all_primitive_llap_io]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[hybridgrace_hashjoin_2]
(batchId=111)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19079/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19079/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19079/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 12 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12983553 - PreCommit-HIVE-Build
> Propagate min/max column values from statistics to the optimizer for
> timestamp type
> ---
>
> Key: HIVE-22311
> URL: https://issues.apache.org/jira/browse/HIVE-22311
> Project: Hive
> Issue Type: Bug
> Components: Statistics
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22311.01.patch, HIVE-22311.01.patch,
> HIVE-22311.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Currently stats annotation does not consider timestamp type e.g. for
> estimates with range predicates.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22311) Propagate min/max column values from statistics to the optimizer for timestamp type
[
https://issues.apache.org/jira/browse/HIVE-22311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16955607#comment-16955607
]
Hive QA commented on HIVE-22311:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
33s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
12s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
33s{color} | {color:blue} standalone-metastore/metastore-common in master has
37 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
7s{color} | {color:blue} standalone-metastore/metastore-server in master has
171 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
7s{color} | {color:blue} ql in master has 1547 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
20s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
9s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
20s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 25 new + 109 unchanged - 11 fixed = 134 total (was 120) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
41s{color} | {color:red} ql: The patch generated 15 new + 280 unchanged - 2
fixed = 295 total (was 282) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
22s{color} | {color:red} standalone-metastore/metastore-server generated 2 new
+ 171 unchanged - 0 fixed = 173 total (was 171) {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
16s{color} | {color:red} ql generated 1 new + 1547 unchanged - 0 fixed = 1548
total (was 1547) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
20s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
15s{color} | {color:red} The patch generated 3 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 40m 53s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:standalone-metastore/metastore-server |
| | Integral division result cast to double or float in
org.apache.hadoop.hive.metastore.columnstats.aggr.TimestampColumnStatsAggregator.aggregate(List,
List, boolean) At TimestampColumnStatsAggregator.java:double or float in
org.apache.hadoop.hive.metastore.columnstats.aggr.TimestampColumnStatsAggregator.aggregate(List,
List, boolean) At TimestampColumnStatsAggregator.java:[line 104] |
| |
org.apache.hadoop.hive.metastore.columnstats.cache.TimestampColumnStatsDataInspector
doesn't override
org.apache.hadoop.hive.metastore.api.TimestampColumnStatsData.equals(Object)
At TimestampColumnStatsDataInspector.java:At
TimestampColumnStatsDataInspector.java:[line 1] |
| FindBugs | module:ql |
| | Dead store to uniformWithinRange in
org.apache.hadoop.hive.ql.optimizer.stats.annotation.StatsRulesProcFactory$JoinStatsRule.process(Node,
Stack, NodeProcessorCtx, Object[]) At
StatsRulesProcFactory.java:org.apache.hadoop.hive.ql.optimizer.stats.annotation.StatsRulesProcFactory$JoinStatsRule.process(Node,
Stack, NodeProcessorCtx, Object[]) At StatsRulesProcFactory.java:[line 2041] |
\\
\\
[jira] [Commented] (HIVE-22373) File Merge tasks fail when containers are reused
[
https://issues.apache.org/jira/browse/HIVE-22373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16955592#comment-16955592
]
Hive QA commented on HIVE-22373:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12983550/HIVE-22373.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17545 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19078/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19078/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19078/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12983550 - PreCommit-HIVE-Build
> File Merge tasks fail when containers are reused
>
>
> Key: HIVE-22373
> URL: https://issues.apache.org/jira/browse/HIVE-22373
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.2
>Reporter: Toshihiko Uchida
>Assignee: Toshihiko Uchida
>Priority: Major
> Attachments: HIVE-22373.patch
>
>
> h1. Problems
> Setting tez.am.container.reuse.enabled=true allows for containers to be
> reused across multiple tasks.
> When two File Merge tasks run on the same container, the last task fails in
> renaming the output path.
> Below is an error log of the task 01_0 on the container
> container_e87_1570604853053_11564_01_03, where the task 04_0 ran
> before the task 01_0.
> It shows that the task 01_0's output file name is taken from the previous
> task id 04_0 mistakenly.
> {code}
> 2019-10-15 13:00:31,438 [ERROR] [TezChild] |tez.TezProcessor|:
> java.lang.RuntimeException: Hive Runtime Error while closing operators
> at
> org.apache.hadoop.hive.ql.exec.tez.MergeFileRecordProcessor.close(MergeFileRecordProcessor.java:188)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:284)
> at
> org.apache.hadoop.hive.ql.exec.tez.MergeFileTezProcessor.run(MergeFileTezProcessor.java:42)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:108)
> at
> com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:41)
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:77)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Failed to close
> AbstractFileMergeOperator
> at
> org.apache.hadoop.hive.ql.exec.AbstractFileMergeOperator.closeOp(AbstractFileMergeOperator.java:315)
> at
> org.apache.hadoop.hive.ql.exec.OrcFileMergeOperator.closeOp(OrcFileMergeOperator.java:265)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:733)
> at
> org.apache.hadoop.hive.ql.exec.tez.MergeFileRecordProcessor.close(MergeFileRecordProcessor.java:180)
> ... 17 more
> Caused by: java.io.IOException: Unable to rename
> viewfs:///user//.hive-staging_hive_2019-10-15_12-59-32_916_2461818728035733124-15476/_task_tmp.-ext-1/_tmp.04_0
> to
> viewfs:///user//.hive-staging_hive_2019-10-15_12-59-32_916_2461818728035733124-15476/_tmp.-ext-1/04_0
> at
> org.apache.hadoop.hive.ql.exec.AbstractFileMergeOperator.closeOp(AbstractFileMergeOperator.java:254)
> ... 20 more
> {code}
> h1. Causes
> When
[jira] [Updated] (HIVE-22311) Propagate min/max column values from statistics to the optimizer for timestamp type
[ https://issues.apache.org/jira/browse/HIVE-22311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-22311: --- Attachment: HIVE-22311.01.patch > Propagate min/max column values from statistics to the optimizer for > timestamp type > --- > > Key: HIVE-22311 > URL: https://issues.apache.org/jira/browse/HIVE-22311 > Project: Hive > Issue Type: Bug > Components: Statistics >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22311.01.patch, HIVE-22311.01.patch, > HIVE-22311.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Currently stats annotation does not consider timestamp type e.g. for > estimates with range predicates. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22373) File Merge tasks fail when containers are reused
[
https://issues.apache.org/jira/browse/HIVE-22373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1696#comment-1696
]
Hive QA commented on HIVE-22373:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 6m
44s{color} | {color:blue} ql in master has 1547 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
46s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
16s{color} | {color:red} The patch generated 3 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 37m 31s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19078/dev-support/hive-personality.sh
|
| git revision | master / c1e5fa3 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19078/yetus/patch-asflicense-problems.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19078/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> File Merge tasks fail when containers are reused
>
>
> Key: HIVE-22373
> URL: https://issues.apache.org/jira/browse/HIVE-22373
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.2
>Reporter: Toshihiko Uchida
>Assignee: Toshihiko Uchida
>Priority: Major
> Attachments: HIVE-22373.patch
>
>
> h1. Problems
> Setting tez.am.container.reuse.enabled=true allows for containers to be
> reused across multiple tasks.
> When two File Merge tasks run on the same container, the last task fails in
> renaming the output path.
> Below is an error log of the task 01_0 on the container
> container_e87_1570604853053_11564_01_03, where the task 04_0 ran
> before the task 01_0.
> It shows that the task 01_0's output file name is taken from the previous
> task id 04_0 mistakenly.
> {code}
> 2019-10-15 13:00:31,438 [ERROR] [TezChild] |tez.TezProcessor|:
> java.lang.RuntimeException: Hive Runtime Error while closing operators
> at
> org.apache.hadoop.hive.ql.exec.tez.MergeFileRecordProcessor.close(MergeFileRecordProcessor.java:188)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:284)
> at
> org.apache.hadoop.hive.ql.exec.tez.MergeFileTezProcessor.run(MergeFileTezProcessor.java:42)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
> at
>
[jira] [Commented] (HIVE-22343) Fix incorrect spelling of 'artifectId' in pom.xml
[
https://issues.apache.org/jira/browse/HIVE-22343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1695#comment-1695
]
Hive QA commented on HIVE-22343:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
25s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 20m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 16m
32s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 22m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 29m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 29m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
51s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 26m
45s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
25s{color} | {color:red} The patch generated 6 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}129m 14s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc xml compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19077/dev-support/hive-personality.sh
|
| git revision | master / c1e5fa3 |
| Default Java | 1.8.0_111 |
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19077/yetus/patch-asflicense-problems.txt
|
| modules | C: shims/common shims/0.23 shims/scheduler shims/aggregator
standalone-metastore/metastore-common common serde metastore vector-code-gen
standalone-metastore/metastore-server ql service jdbc beeline cli contrib
druid-handler hbase-handler hcatalog/core hcatalog/hcatalog-pig-adapter
hcatalog/server-extensions hcatalog/webhcat/java-client hcatalog/streaming
streaming testutils packaging upgrade-acid/pre-upgrade . itests/hcatalog-unit
itests/hive-minikdc itests/hive-unit itests/hive-unit-hadoop2 itests/qtest
itests/qtest-accumulo itests/qtest-druid itests/qtest-spark itests/test-serde
itests/util testutils/ptest2 U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19077/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Fix incorrect spelling of 'artifectId' in pom.xml
> -
>
> Key: HIVE-22343
> URL: https://issues.apache.org/jira/browse/HIVE-22343
> Project: Hive
> Issue Type: Improvement
>Reporter: ice bai
>Assignee: ice bai
>Priority: Major
> Fix For: 4.0.0, 3.2.0
>
> Attachments: HIVE-22343.patch, HIVE-22343.patch
>
>
> There are some incorrect spelling of 'artifectId' in pom.xml or xxx/pom.xml
> Such as:
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22343) Fix incorrect spelling of 'artifectId' in pom.xml
[
https://issues.apache.org/jira/browse/HIVE-22343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16955544#comment-16955544
]
Hive QA commented on HIVE-22343:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12983547/HIVE-22343.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17545 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19077/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19077/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19077/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12983547 - PreCommit-HIVE-Build
> Fix incorrect spelling of 'artifectId' in pom.xml
> -
>
> Key: HIVE-22343
> URL: https://issues.apache.org/jira/browse/HIVE-22343
> Project: Hive
> Issue Type: Improvement
>Reporter: ice bai
>Assignee: ice bai
>Priority: Major
> Fix For: 4.0.0, 3.2.0
>
> Attachments: HIVE-22343.patch, HIVE-22343.patch
>
>
> There are some incorrect spelling of 'artifectId' in pom.xml or xxx/pom.xml
> Such as:
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22369) Handle HiveTableFunctionScan at return path
[
https://issues.apache.org/jira/browse/HIVE-22369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16955540#comment-16955540
]
Jesus Camacho Rodriguez commented on HIVE-22369:
[~mgergely], I think that makes sense. This seems to have happened because
{{HiveTableFunctionScan}} was added after the return path was initially
developed, and we added support for the translation in the {{ASTConverter}}
path but we missed it in the {{HiveOpConverter}} path.
> Handle HiveTableFunctionScan at return path
> ---
>
> Key: HIVE-22369
> URL: https://issues.apache.org/jira/browse/HIVE-22369
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
>
> The
> [optimizedOptiqPlan|https://github.com/apache/hive/blob/5c91d324f22c2ae47e234e76a9bc5ee1a71e6a70/ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java#L1573]
> at CalcitePlanner.getOptimizedHiveOPDag is ultimately generated by
> CalcitePlanner.internalGenSelectLogicalPlan, which may either provide a
> [HiveProject|https://github.com/apache/hive/blob/5c91d324f22c2ae47e234e76a9bc5ee1a71e6a70/ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java#L4831]
> or a
> [HiveTableFunctionScan|https://github.com/apache/hive/blob/5c91d324f22c2ae47e234e76a9bc5ee1a71e6a70/ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java#L4776].
> When HiveCalciteUtil.getTopLevelSelect is invoked on this it is looking for
> a
> [HiveProject|https://github.com/apache/hive/blob/5c91d324f22c2ae47e234e76a9bc5ee1a71e6a70/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveCalciteUtil.java#L633]
> node in the tree, which if won't find in case of a HiveTableFunctionScan was
> returned. This is why TestNewGetSplitsFormat is failing with return path.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22373) File Merge tasks fail when containers are reused
[
https://issues.apache.org/jira/browse/HIVE-22373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Toshihiko Uchida updated HIVE-22373:
Attachment: HIVE-22373.patch
Status: Patch Available (was: In Progress)
> File Merge tasks fail when containers are reused
>
>
> Key: HIVE-22373
> URL: https://issues.apache.org/jira/browse/HIVE-22373
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.2
>Reporter: Toshihiko Uchida
>Assignee: Toshihiko Uchida
>Priority: Major
> Attachments: HIVE-22373.patch
>
>
> h1. Problems
> Setting tez.am.container.reuse.enabled=true allows for containers to be
> reused across multiple tasks.
> When two File Merge tasks run on the same container, the last task fails in
> renaming the output path.
> Below is an error log of the task 01_0 on the container
> container_e87_1570604853053_11564_01_03, where the task 04_0 ran
> before the task 01_0.
> It shows that the task 01_0's output file name is taken from the previous
> task id 04_0 mistakenly.
> {code}
> 2019-10-15 13:00:31,438 [ERROR] [TezChild] |tez.TezProcessor|:
> java.lang.RuntimeException: Hive Runtime Error while closing operators
> at
> org.apache.hadoop.hive.ql.exec.tez.MergeFileRecordProcessor.close(MergeFileRecordProcessor.java:188)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:284)
> at
> org.apache.hadoop.hive.ql.exec.tez.MergeFileTezProcessor.run(MergeFileTezProcessor.java:42)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:108)
> at
> com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:41)
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:77)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Failed to close
> AbstractFileMergeOperator
> at
> org.apache.hadoop.hive.ql.exec.AbstractFileMergeOperator.closeOp(AbstractFileMergeOperator.java:315)
> at
> org.apache.hadoop.hive.ql.exec.OrcFileMergeOperator.closeOp(OrcFileMergeOperator.java:265)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:733)
> at
> org.apache.hadoop.hive.ql.exec.tez.MergeFileRecordProcessor.close(MergeFileRecordProcessor.java:180)
> ... 17 more
> Caused by: java.io.IOException: Unable to rename
> viewfs:///user//.hive-staging_hive_2019-10-15_12-59-32_916_2461818728035733124-15476/_task_tmp.-ext-1/_tmp.04_0
> to
> viewfs:///user//.hive-staging_hive_2019-10-15_12-59-32_916_2461818728035733124-15476/_tmp.-ext-1/04_0
> at
> org.apache.hadoop.hive.ql.exec.AbstractFileMergeOperator.closeOp(AbstractFileMergeOperator.java:254)
> ... 20 more
> {code}
> h1. Causes
> When AbstractFileMergeOperator is initialized, taskId is updated only for the
> first time.
> - AbstractFileMergeOperator.java
> {code}
> private void updatePaths(Path tp, Path ttp) {
> if (taskId == null) {
> taskId = Utilities.getTaskId(jc);
> }
> {code}
> It leads to the above conflict of the output file names.
> h1. Solutions
> Remove the null-checking conditional, which was introduced in HIVE-14640, and
> update taskId from JobConf whenever the operator is initialized.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-22373) File Merge tasks fail when containers are reused
[
https://issues.apache.org/jira/browse/HIVE-22373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Toshihiko Uchida reassigned HIVE-22373:
---
Assignee: Toshihiko Uchida
> File Merge tasks fail when containers are reused
>
>
> Key: HIVE-22373
> URL: https://issues.apache.org/jira/browse/HIVE-22373
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.2
>Reporter: Toshihiko Uchida
>Assignee: Toshihiko Uchida
>Priority: Major
>
> h1. Problems
> Setting tez.am.container.reuse.enabled=true allows for containers to be
> reused across multiple tasks.
> When two File Merge tasks run on the same container, the last task fails in
> renaming the output path.
> Below is an error log of the task 01_0 on the container
> container_e87_1570604853053_11564_01_03, where the task 04_0 ran
> before the task 01_0.
> It shows that the task 01_0's output file name is taken from the previous
> task id 04_0 mistakenly.
> {code}
> 2019-10-15 13:00:31,438 [ERROR] [TezChild] |tez.TezProcessor|:
> java.lang.RuntimeException: Hive Runtime Error while closing operators
> at
> org.apache.hadoop.hive.ql.exec.tez.MergeFileRecordProcessor.close(MergeFileRecordProcessor.java:188)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:284)
> at
> org.apache.hadoop.hive.ql.exec.tez.MergeFileTezProcessor.run(MergeFileTezProcessor.java:42)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:108)
> at
> com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:41)
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:77)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Failed to close
> AbstractFileMergeOperator
> at
> org.apache.hadoop.hive.ql.exec.AbstractFileMergeOperator.closeOp(AbstractFileMergeOperator.java:315)
> at
> org.apache.hadoop.hive.ql.exec.OrcFileMergeOperator.closeOp(OrcFileMergeOperator.java:265)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:733)
> at
> org.apache.hadoop.hive.ql.exec.tez.MergeFileRecordProcessor.close(MergeFileRecordProcessor.java:180)
> ... 17 more
> Caused by: java.io.IOException: Unable to rename
> viewfs:///user//.hive-staging_hive_2019-10-15_12-59-32_916_2461818728035733124-15476/_task_tmp.-ext-1/_tmp.04_0
> to
> viewfs:///user//.hive-staging_hive_2019-10-15_12-59-32_916_2461818728035733124-15476/_tmp.-ext-1/04_0
> at
> org.apache.hadoop.hive.ql.exec.AbstractFileMergeOperator.closeOp(AbstractFileMergeOperator.java:254)
> ... 20 more
> {code}
> h1. Causes
> When AbstractFileMergeOperator is initialized, taskId is updated only for the
> first time.
> - AbstractFileMergeOperator.java
> {code}
> private void updatePaths(Path tp, Path ttp) {
> if (taskId == null) {
> taskId = Utilities.getTaskId(jc);
> }
> {code}
> It leads to the above conflict of the output file names.
> h1. Solutions
> Remove the null-checking conditional, which was introduced in HIVE-14640, and
> update taskId from JobConf whenever the operator is initialized.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work started] (HIVE-22373) File Merge tasks fail when containers are reused
[
https://issues.apache.org/jira/browse/HIVE-22373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HIVE-22373 started by Toshihiko Uchida.
---
> File Merge tasks fail when containers are reused
>
>
> Key: HIVE-22373
> URL: https://issues.apache.org/jira/browse/HIVE-22373
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.2
>Reporter: Toshihiko Uchida
>Assignee: Toshihiko Uchida
>Priority: Major
>
> h1. Problems
> Setting tez.am.container.reuse.enabled=true allows for containers to be
> reused across multiple tasks.
> When two File Merge tasks run on the same container, the last task fails in
> renaming the output path.
> Below is an error log of the task 01_0 on the container
> container_e87_1570604853053_11564_01_03, where the task 04_0 ran
> before the task 01_0.
> It shows that the task 01_0's output file name is taken from the previous
> task id 04_0 mistakenly.
> {code}
> 2019-10-15 13:00:31,438 [ERROR] [TezChild] |tez.TezProcessor|:
> java.lang.RuntimeException: Hive Runtime Error while closing operators
> at
> org.apache.hadoop.hive.ql.exec.tez.MergeFileRecordProcessor.close(MergeFileRecordProcessor.java:188)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:284)
> at
> org.apache.hadoop.hive.ql.exec.tez.MergeFileTezProcessor.run(MergeFileTezProcessor.java:42)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:108)
> at
> com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:41)
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:77)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Failed to close
> AbstractFileMergeOperator
> at
> org.apache.hadoop.hive.ql.exec.AbstractFileMergeOperator.closeOp(AbstractFileMergeOperator.java:315)
> at
> org.apache.hadoop.hive.ql.exec.OrcFileMergeOperator.closeOp(OrcFileMergeOperator.java:265)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:733)
> at
> org.apache.hadoop.hive.ql.exec.tez.MergeFileRecordProcessor.close(MergeFileRecordProcessor.java:180)
> ... 17 more
> Caused by: java.io.IOException: Unable to rename
> viewfs:///user//.hive-staging_hive_2019-10-15_12-59-32_916_2461818728035733124-15476/_task_tmp.-ext-1/_tmp.04_0
> to
> viewfs:///user//.hive-staging_hive_2019-10-15_12-59-32_916_2461818728035733124-15476/_tmp.-ext-1/04_0
> at
> org.apache.hadoop.hive.ql.exec.AbstractFileMergeOperator.closeOp(AbstractFileMergeOperator.java:254)
> ... 20 more
> {code}
> h1. Causes
> When AbstractFileMergeOperator is initialized, taskId is updated only for the
> first time.
> - AbstractFileMergeOperator.java
> {code}
> private void updatePaths(Path tp, Path ttp) {
> if (taskId == null) {
> taskId = Utilities.getTaskId(jc);
> }
> {code}
> It leads to the above conflict of the output file names.
> h1. Solutions
> Remove the null-checking conditional, which was introduced in HIVE-14640, and
> update taskId from JobConf whenever the operator is initialized.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22343) Fix incorrect spelling of 'artifectId' in pom.xml
[ https://issues.apache.org/jira/browse/HIVE-22343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ice bai updated HIVE-22343: --- Status: Open (was: Patch Available) > Fix incorrect spelling of 'artifectId' in pom.xml > - > > Key: HIVE-22343 > URL: https://issues.apache.org/jira/browse/HIVE-22343 > Project: Hive > Issue Type: Improvement >Reporter: ice bai >Assignee: ice bai >Priority: Major > Fix For: 4.0.0, 3.2.0 > > Attachments: HIVE-22343.patch, HIVE-22343.patch > > > There are some incorrect spelling of 'artifectId' in pom.xml or xxx/pom.xml > Such as: > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22343) Fix incorrect spelling of 'artifectId' in pom.xml
[ https://issues.apache.org/jira/browse/HIVE-22343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ice bai updated HIVE-22343: --- Attachment: HIVE-22343.patch Status: Patch Available (was: Open) > Fix incorrect spelling of 'artifectId' in pom.xml > - > > Key: HIVE-22343 > URL: https://issues.apache.org/jira/browse/HIVE-22343 > Project: Hive > Issue Type: Improvement >Reporter: ice bai >Assignee: ice bai >Priority: Major > Fix For: 4.0.0, 3.2.0 > > Attachments: HIVE-22343.patch, HIVE-22343.patch > > > There are some incorrect spelling of 'artifectId' in pom.xml or xxx/pom.xml > Such as: > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
