[jira] [Commented] (HIVE-22568) Process compaction candidates in parallel by the Initiator
[
https://issues.apache.org/jira/browse/HIVE-22568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16992275#comment-16992275
]
Hive QA commented on HIVE-22568:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
2s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
39s{color} | {color:blue} common in master has 65 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
23s{color} | {color:blue} ql in master has 1532 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
21s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
42s{color} | {color:red} ql: The patch generated 1 new + 17 unchanged - 5 fixed
= 18 total (was 22) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 31m 34s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19847/dev-support/hive-personality.sh
|
| git revision | master / d7a193b |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19847/yetus/diff-checkstyle-ql.txt
|
| modules | C: common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19847/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Process compaction candidates in parallel by the Initiator
> --
>
> Key: HIVE-22568
> URL: https://issues.apache.org/jira/browse/HIVE-22568
> Project: Hive
> Issue Type: Improvement
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-22568.1.patch, HIVE-22568.2.patch,
> HIVE-22568.3.patch, HIVE-22568.4.patch, HIVE-22568.5.patch
>
>
> `checkForCompaction` includes many file metadata checks and may be expensive.
> Therefore, make sense using a thread pool here and running
> `checkForCompactions` in parallel.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22508) KeyWrapperComparator throws exception
[ https://issues.apache.org/jira/browse/HIVE-22508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-22508: -- Status: Patch Available (was: Open) > KeyWrapperComparator throws exception > - > > Key: HIVE-22508 > URL: https://issues.apache.org/jira/browse/HIVE-22508 > Project: Hive > Issue Type: Bug > Components: Physical Optimizer >Reporter: Krisztian Kasa >Assignee: Krisztian Kasa >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22508.1.patch, HIVE-22508.1.patch, > HIVE-22508.1.patch, HIVE-22508.2.patch, HIVE-22508.2.patch, HIVE-22508.2.patch > > > TopNKeyOperator.KeyWrapperComparator throws exception when a new key and a > copied key should be compared. > The current implementation uses the standard object inspectors for all > KeyWrapper instances. However when comparing untouched KeyWrappers the key > object inspector should be used which can be extracted form > Operator.inputObjectInspectors during initialization of the key's > ExprNodeEvaluator. > This can cause a ClassCastException when the comparator is used collections > like TreeSet. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22508) KeyWrapperComparator throws exception
[ https://issues.apache.org/jira/browse/HIVE-22508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-22508: -- Status: Open (was: Patch Available) > KeyWrapperComparator throws exception > - > > Key: HIVE-22508 > URL: https://issues.apache.org/jira/browse/HIVE-22508 > Project: Hive > Issue Type: Bug > Components: Physical Optimizer >Reporter: Krisztian Kasa >Assignee: Krisztian Kasa >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22508.1.patch, HIVE-22508.1.patch, > HIVE-22508.1.patch, HIVE-22508.2.patch, HIVE-22508.2.patch, HIVE-22508.2.patch > > > TopNKeyOperator.KeyWrapperComparator throws exception when a new key and a > copied key should be compared. > The current implementation uses the standard object inspectors for all > KeyWrapper instances. However when comparing untouched KeyWrappers the key > object inspector should be used which can be extracted form > Operator.inputObjectInspectors during initialization of the key's > ExprNodeEvaluator. > This can cause a ClassCastException when the comparator is used collections > like TreeSet. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22508) KeyWrapperComparator throws exception
[ https://issues.apache.org/jira/browse/HIVE-22508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-22508: -- Attachment: HIVE-22508.2.patch > KeyWrapperComparator throws exception > - > > Key: HIVE-22508 > URL: https://issues.apache.org/jira/browse/HIVE-22508 > Project: Hive > Issue Type: Bug > Components: Physical Optimizer >Reporter: Krisztian Kasa >Assignee: Krisztian Kasa >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22508.1.patch, HIVE-22508.1.patch, > HIVE-22508.1.patch, HIVE-22508.2.patch, HIVE-22508.2.patch, HIVE-22508.2.patch > > > TopNKeyOperator.KeyWrapperComparator throws exception when a new key and a > copied key should be compared. > The current implementation uses the standard object inspectors for all > KeyWrapper instances. However when comparing untouched KeyWrappers the key > object inspector should be used which can be extracted form > Operator.inputObjectInspectors during initialization of the key's > ExprNodeEvaluator. > This can cause a ClassCastException when the comparator is used collections > like TreeSet. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-20150) TopNKey pushdown
[ https://issues.apache.org/jira/browse/HIVE-20150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-20150: -- Attachment: HIVE-20150.25.patch > TopNKey pushdown > > > Key: HIVE-20150 > URL: https://issues.apache.org/jira/browse/HIVE-20150 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Affects Versions: 4.0.0 >Reporter: Teddy Choi >Assignee: Krisztian Kasa >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20150.1.patch, HIVE-20150.10.patch, > HIVE-20150.11.patch, HIVE-20150.11.patch, HIVE-20150.14.patch, > HIVE-20150.15.patch, HIVE-20150.16.patch, HIVE-20150.17.patch, > HIVE-20150.17.patch, HIVE-20150.18.patch, HIVE-20150.18.patch, > HIVE-20150.19.patch, HIVE-20150.2.patch, HIVE-20150.20.patch, > HIVE-20150.21.patch, HIVE-20150.22.patch, HIVE-20150.23.patch, > HIVE-20150.24.patch, HIVE-20150.25.patch, HIVE-20150.4.patch, > HIVE-20150.5.patch, HIVE-20150.6.patch, HIVE-20150.7.patch, > HIVE-20150.8.patch, HIVE-20150.9.patch > > > TopNKey operator is implemented in HIVE-17896, but it needs more work in > pushdown implementation. So this issue covers TopNKey pushdown implementation > with proper tests. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-20150) TopNKey pushdown
[ https://issues.apache.org/jira/browse/HIVE-20150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-20150: -- Status: Patch Available (was: Open) > TopNKey pushdown > > > Key: HIVE-20150 > URL: https://issues.apache.org/jira/browse/HIVE-20150 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Affects Versions: 4.0.0 >Reporter: Teddy Choi >Assignee: Krisztian Kasa >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20150.1.patch, HIVE-20150.10.patch, > HIVE-20150.11.patch, HIVE-20150.11.patch, HIVE-20150.14.patch, > HIVE-20150.15.patch, HIVE-20150.16.patch, HIVE-20150.17.patch, > HIVE-20150.17.patch, HIVE-20150.18.patch, HIVE-20150.18.patch, > HIVE-20150.19.patch, HIVE-20150.2.patch, HIVE-20150.20.patch, > HIVE-20150.21.patch, HIVE-20150.22.patch, HIVE-20150.23.patch, > HIVE-20150.24.patch, HIVE-20150.25.patch, HIVE-20150.4.patch, > HIVE-20150.5.patch, HIVE-20150.6.patch, HIVE-20150.7.patch, > HIVE-20150.8.patch, HIVE-20150.9.patch > > > TopNKey operator is implemented in HIVE-17896, but it needs more work in > pushdown implementation. So this issue covers TopNKey pushdown implementation > with proper tests. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-20150) TopNKey pushdown
[ https://issues.apache.org/jira/browse/HIVE-20150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-20150: -- Status: Open (was: Patch Available) > TopNKey pushdown > > > Key: HIVE-20150 > URL: https://issues.apache.org/jira/browse/HIVE-20150 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Affects Versions: 4.0.0 >Reporter: Teddy Choi >Assignee: Krisztian Kasa >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20150.1.patch, HIVE-20150.10.patch, > HIVE-20150.11.patch, HIVE-20150.11.patch, HIVE-20150.14.patch, > HIVE-20150.15.patch, HIVE-20150.16.patch, HIVE-20150.17.patch, > HIVE-20150.17.patch, HIVE-20150.18.patch, HIVE-20150.18.patch, > HIVE-20150.19.patch, HIVE-20150.2.patch, HIVE-20150.20.patch, > HIVE-20150.21.patch, HIVE-20150.22.patch, HIVE-20150.23.patch, > HIVE-20150.24.patch, HIVE-20150.25.patch, HIVE-20150.4.patch, > HIVE-20150.5.patch, HIVE-20150.6.patch, HIVE-20150.7.patch, > HIVE-20150.8.patch, HIVE-20150.9.patch > > > TopNKey operator is implemented in HIVE-17896, but it needs more work in > pushdown implementation. So this issue covers TopNKey pushdown implementation > with proper tests. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22606) AvroSerde logs avro.schema.literal under INFO level
[
https://issues.apache.org/jira/browse/HIVE-22606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16992248#comment-16992248
]
Hive QA commented on HIVE-22606:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12988347/HIVE-22606.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17762 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19846/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19846/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19846/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12988347 - PreCommit-HIVE-Build
> AvroSerde logs avro.schema.literal under INFO level
> ---
>
> Key: HIVE-22606
> URL: https://issues.apache.org/jira/browse/HIVE-22606
> Project: Hive
> Issue Type: Bug
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-22606.1.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22561) Data loss on map join for bucketed, partitioned table
[
https://issues.apache.org/jira/browse/HIVE-22561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aditya Shah updated HIVE-22561:
---
Attachment: HIVE-22561.branch-3.1.patch
Status: Patch Available (was: Open)
Submitting patch with the correct name to run with branch-3.1 profile.
> Data loss on map join for bucketed, partitioned table
> -
>
> Key: HIVE-22561
> URL: https://issues.apache.org/jira/browse/HIVE-22561
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.2
>Reporter: Aditya Shah
>Assignee: Aditya Shah
>Priority: Blocker
> Fix For: 3.1.0, 3.0.0
>
> Attachments: HIVE-22561.branch-3.1.patch, HIVE-22561.patch,
> Screenshot 2019-11-28 at 8.45.17 PM.png, image-2019-11-28-20-46-25-432.png
>
>
> A map join on a column (which is neither involved in bucketing and partition)
> causes data loss.
> Steps to reproduce:
> Env: [hive-dev-box|[https://github.com/kgyrtkirk/hive-dev-box]] hive 3.1.2.
> Create tables:
>
> {code:java}
> CREATE TABLE `testj2`(
> `id` int,
> `bn` string,
> `cn` string,
> `ad` map,
> `mi` array)
> PARTITIONED BY (
> `br` string)
> CLUSTERED BY (
> bn)
> INTO 2 BUCKETS
> ROW FORMAT DELIMITED
> FIELDS TERMINATED BY ','
> STORED AS TEXTFILE
> TBLPROPERTIES (
> 'bucketing_version'='2');
> CREATE TABLE `testj1`(
> `id` int,
> `can` string,
> `cn` string,
> `ad` map,
> `av` boolean,
> `mi` array)
> PARTITIONED BY (
> `brand` string)
> CLUSTERED BY (
> can)
> INTO 2 BUCKETS
> ROW FORMAT DELIMITED
> FIELDS TERMINATED BY ','
> STORED AS TEXTFILE
> TBLPROPERTIES (
> 'bucketing_version'='2');
> {code}
> insert some data in both:
> {code:java}
> insert into testj1 values (100, 'mes_1', 'customer_1', map('city1', 560077),
> false, array(5, 10), 'brand_1'),
> (101, 'mes_2', 'customer_2', map('city2', 560078), true, array(10, 20),
> 'brand_2'),
> (102, 'mes_3', 'customer_3', map('city3', 560079), false, array(15, 30),
> 'brand_3'),
> (103, 'mes_4', 'customer_4', map('city4', 560080), true, array(20, 40),
> 'brand_4'),
> (104, 'mes_5', 'customer_5', map('city5', 560081), false, array(25, 50),
> 'brand_5');
> insert into table testj2 values (100, 'tv_0', 'customer_0', map('city0',
> 560076),array(0, 0, 0), 'tv'),
> (101, 'tv_1', 'customer_1', map('city1', 560077),array(20, 25, 30), 'tv'),
> (102, 'tv_2', 'customer_2', map('city2', 560078),array(40, 50, 60), 'tv'),
> (103, 'tv_3', 'customer_3', map('city3', 560079),array(60, 75, 90), 'tv'),
> (104, 'tv_4', 'customer_4', map('city4', 560080),array(80, 100, 120), 'tv');
> {code}
> Do a join between them:
> {code:java}
> select t1.id, t1.can, t1.cn, t2.bn,t2.ad, t2.br FROM testj1 t1 JOIN testj2 t2
> on (t1.id = t2.id) order by t1.id;
> {code}
> Observed results:
> !image-2019-11-28-20-46-25-432.png|width=524,height=100!
> In the plan, I can see a map join. Disabling it gives the correct result.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22561) Data loss on map join for bucketed, partitioned table
[
https://issues.apache.org/jira/browse/HIVE-22561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aditya Shah updated HIVE-22561:
---
Status: Open (was: Patch Available)
> Data loss on map join for bucketed, partitioned table
> -
>
> Key: HIVE-22561
> URL: https://issues.apache.org/jira/browse/HIVE-22561
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.2
>Reporter: Aditya Shah
>Assignee: Aditya Shah
>Priority: Blocker
> Fix For: 3.1.0, 3.0.0
>
> Attachments: HIVE-22561.patch, Screenshot 2019-11-28 at 8.45.17
> PM.png, image-2019-11-28-20-46-25-432.png
>
>
> A map join on a column (which is neither involved in bucketing and partition)
> causes data loss.
> Steps to reproduce:
> Env: [hive-dev-box|[https://github.com/kgyrtkirk/hive-dev-box]] hive 3.1.2.
> Create tables:
>
> {code:java}
> CREATE TABLE `testj2`(
> `id` int,
> `bn` string,
> `cn` string,
> `ad` map,
> `mi` array)
> PARTITIONED BY (
> `br` string)
> CLUSTERED BY (
> bn)
> INTO 2 BUCKETS
> ROW FORMAT DELIMITED
> FIELDS TERMINATED BY ','
> STORED AS TEXTFILE
> TBLPROPERTIES (
> 'bucketing_version'='2');
> CREATE TABLE `testj1`(
> `id` int,
> `can` string,
> `cn` string,
> `ad` map,
> `av` boolean,
> `mi` array)
> PARTITIONED BY (
> `brand` string)
> CLUSTERED BY (
> can)
> INTO 2 BUCKETS
> ROW FORMAT DELIMITED
> FIELDS TERMINATED BY ','
> STORED AS TEXTFILE
> TBLPROPERTIES (
> 'bucketing_version'='2');
> {code}
> insert some data in both:
> {code:java}
> insert into testj1 values (100, 'mes_1', 'customer_1', map('city1', 560077),
> false, array(5, 10), 'brand_1'),
> (101, 'mes_2', 'customer_2', map('city2', 560078), true, array(10, 20),
> 'brand_2'),
> (102, 'mes_3', 'customer_3', map('city3', 560079), false, array(15, 30),
> 'brand_3'),
> (103, 'mes_4', 'customer_4', map('city4', 560080), true, array(20, 40),
> 'brand_4'),
> (104, 'mes_5', 'customer_5', map('city5', 560081), false, array(25, 50),
> 'brand_5');
> insert into table testj2 values (100, 'tv_0', 'customer_0', map('city0',
> 560076),array(0, 0, 0), 'tv'),
> (101, 'tv_1', 'customer_1', map('city1', 560077),array(20, 25, 30), 'tv'),
> (102, 'tv_2', 'customer_2', map('city2', 560078),array(40, 50, 60), 'tv'),
> (103, 'tv_3', 'customer_3', map('city3', 560079),array(60, 75, 90), 'tv'),
> (104, 'tv_4', 'customer_4', map('city4', 560080),array(80, 100, 120), 'tv');
> {code}
> Do a join between them:
> {code:java}
> select t1.id, t1.can, t1.cn, t2.bn,t2.ad, t2.br FROM testj1 t1 JOIN testj2 t2
> on (t1.id = t2.id) order by t1.id;
> {code}
> Observed results:
> !image-2019-11-28-20-46-25-432.png|width=524,height=100!
> In the plan, I can see a map join. Disabling it gives the correct result.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22606) AvroSerde logs avro.schema.literal under INFO level
[
https://issues.apache.org/jira/browse/HIVE-22606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16992213#comment-16992213
]
Hive QA commented on HIVE-22606:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
45s{color} | {color:blue} serde in master has 198 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 13m 42s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19846/dev-support/hive-personality.sh
|
| git revision | master / d7a193b |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: serde U: serde |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19846/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> AvroSerde logs avro.schema.literal under INFO level
> ---
>
> Key: HIVE-22606
> URL: https://issues.apache.org/jira/browse/HIVE-22606
> Project: Hive
> Issue Type: Bug
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-22606.1.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-17440) TestSparkCliDriver.testCliDriver[vectorized_ptf]
[
https://issues.apache.org/jira/browse/HIVE-17440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16992200#comment-16992200
]
Hive QA commented on HIVE-17440:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12988342/HIVE-17440.01.branch-2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 10756 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_queries]
(batchId=227)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[avro_tableproperty_optimize]
(batchId=22)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[explaindenpendencydiffengs]
(batchId=38)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[table_nonprintable]
(batchId=140)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[join_acid_non_acid]
(batchId=158)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[union_fast_stats]
(batchId=153)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorized_parquet_types]
(batchId=155)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[merge_negative_5]
(batchId=88)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[explaindenpendencydiffengs]
(batchId=115)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vectorization_input_format_excludes]
(batchId=117)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19845/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19845/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19845/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 10 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12988342 - PreCommit-HIVE-Build
> TestSparkCliDriver.testCliDriver[vectorized_ptf]
>
>
> Key: HIVE-17440
> URL: https://issues.apache.org/jira/browse/HIVE-17440
> Project: Hive
> Issue Type: Sub-task
>Reporter: Vihang Karajgaonkar
>Assignee: László Bodor
>Priority: Major
> Attachments: HIVE-17440.01.branch-2.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22604) TestMiniLlapCliDriver[table_nonprintable.q]
[
https://issues.apache.org/jira/browse/HIVE-22604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16992187#comment-16992187
]
Hive QA commented on HIVE-22604:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12988332/HIVE-22604.01.branch-2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 10756 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_queries]
(batchId=227)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[avro_tableproperty_optimize]
(batchId=22)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[explaindenpendencydiffengs]
(batchId=38)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[join_acid_non_acid]
(batchId=158)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[union_fast_stats]
(batchId=153)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorized_parquet_types]
(batchId=155)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[merge_negative_5]
(batchId=88)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[explaindenpendencydiffengs]
(batchId=115)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vectorization_input_format_excludes]
(batchId=117)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vectorized_ptf]
(batchId=125)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19844/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19844/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19844/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 10 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12988332 - PreCommit-HIVE-Build
> TestMiniLlapCliDriver[table_nonprintable.q]
> ---
>
> Key: HIVE-22604
> URL: https://issues.apache.org/jira/browse/HIVE-22604
> Project: Hive
> Issue Type: Sub-task
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
> Attachments: HIVE-22604.01.branch-2.patch,
> table_nonprintable.q.out.diff
>
>
> q.out diff attached: [^table_nonprintable.q.out.diff]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-17441) TestSparkCliDriver.testCliDriver[explaindenpendencydiffengs]
[
https://issues.apache.org/jira/browse/HIVE-17441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16992164#comment-16992164
]
Hive QA commented on HIVE-17441:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12988341/HIVE-17441.01.branch-2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 11 failed/errored test(s), 10756 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_queries]
(batchId=227)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[avro_tableproperty_optimize]
(batchId=22)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[explaindenpendencydiffengs]
(batchId=38)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[table_nonprintable]
(batchId=140)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[join_acid_non_acid]
(batchId=158)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[union_fast_stats]
(batchId=153)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorized_parquet_types]
(batchId=155)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[merge_negative_5]
(batchId=88)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vectorization_input_format_excludes]
(batchId=117)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vectorized_ptf]
(batchId=125)
org.apache.hive.jdbc.TestJdbcDriver2.testYarnATSGuid (batchId=222)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19843/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19843/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19843/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 11 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12988341 - PreCommit-HIVE-Build
> TestSparkCliDriver.testCliDriver[explaindenpendencydiffengs]
>
>
> Key: HIVE-17441
> URL: https://issues.apache.org/jira/browse/HIVE-17441
> Project: Hive
> Issue Type: Sub-task
>Reporter: Vihang Karajgaonkar
>Assignee: László Bodor
>Priority: Major
> Attachments: HIVE-17441.01.branch-2.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22605) NPE in LlapLoadGeneratorService During Tests
[
https://issues.apache.org/jira/browse/HIVE-22605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-22605:
--
Attachment: HIVE-22605.2.patch
> NPE in LlapLoadGeneratorService During Tests
>
>
> Key: HIVE-22605
> URL: https://issues.apache.org/jira/browse/HIVE-22605
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HIVE-22605.1.patch, HIVE-22605.2.patch
>
>
> {code:none}
> java.lang.NullPointerException: null
> at
> org.apache.hadoop.hive.llap.daemon.impl.LlapLoadGeneratorService.serviceStop(LlapLoadGeneratorService.java:103)
> ~[classes/:?]
> at
> org.apache.hadoop.service.AbstractService.stop(AbstractService.java:220)
> ~[hadoop-common-3.1.0.jar:?]
> at
> org.apache.hadoop.service.ServiceOperations.stop(ServiceOperations.java:54)
> ~[hadoop-common-3.1.0.jar:?]
> at
> org.apache.hadoop.service.ServiceOperations.stopQuietly(ServiceOperations.java:102)
> [hadoop-common-3.1.0.jar:?]
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:172)
> [hadoop-common-3.1.0.jar:?]
> at
> org.apache.hadoop.hive.llap.daemon.impl.TestLlapLoadGeneratorService.testLoadGeneratorFails(TestLlapLoadGeneratorService.java:70)
> [test-classes/:?]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[?:1.8.0_102]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22538) RS deduplication does not always enforce hive.optimize.reducededuplication.min.reducer
[
https://issues.apache.org/jira/browse/HIVE-22538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16992156#comment-16992156
]
Jesus Camacho Rodriguez commented on HIVE-22538:
[~pvary], I did not have time to check this further. There is an enum
{{ReducerTraits}} in {{ReduceSinkDesc}} and an {{EnumSet reduceTraits}}. My
initial take would be to try to set that enum correctly whenever we create a RS
to enforce bucketing for ACID tables (we already pass {{writeType}} in the
constructor hence maybe it is not so difficult to set the traits at that time?
Or maybe from SemanticAnalyzer when the RS is created?). [~gopalv], does that
make sense? Other thoughts?
> RS deduplication does not always enforce
> hive.optimize.reducededuplication.min.reducer
> --
>
> Key: HIVE-22538
> URL: https://issues.apache.org/jira/browse/HIVE-22538
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22538.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> For transactional tables, that property might be overriden to 1, which can
> lead to merging final aggregation into a single stage (hence leading to
> performance degradation). For instance, when autogather column stats is
> enabled, this can happen for the following query:
> {code}
> set hive.support.concurrency=true;
> set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
> EXPLAIN
> CREATE TABLE x STORED AS ORC TBLPROPERTIES('transactional'='true') AS
> SELECT * FROM SRC x CLUSTER BY x.key;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22600) TestAccumuloCliDriver[accumulo_queries.q] - join returns invalid rows
[
https://issues.apache.org/jira/browse/HIVE-22600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16992141#comment-16992141
]
Hive QA commented on HIVE-22600:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12988333/HIVE-22600.01.branch-2.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 13 failed/errored test(s), 10743 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[accumulo_queries]
(batchId=81)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[avro_tableproperty_optimize]
(batchId=22)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[explaindenpendencydiffengs]
(batchId=38)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[table_nonprintable]
(batchId=140)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[join_acid_non_acid]
(batchId=158)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[union_fast_stats]
(batchId=153)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorized_parquet_types]
(batchId=155)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[merge_negative_5]
(batchId=88)
org.apache.hadoop.hive.cli.TestSparkCliDriver.org.apache.hadoop.hive.cli.TestSparkCliDriver
(batchId=98)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[explaindenpendencydiffengs]
(batchId=115)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vectorization_input_format_excludes]
(batchId=117)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vectorized_ptf]
(batchId=125)
org.apache.hive.jdbc.TestSSL.testMetastoreWithSSL (batchId=221)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19842/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19842/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19842/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 13 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12988333 - PreCommit-HIVE-Build
> TestAccumuloCliDriver[accumulo_queries.q] - join returns invalid rows
> -
>
> Key: HIVE-22600
> URL: https://issues.apache.org/jira/browse/HIVE-22600
> Project: Hive
> Issue Type: Sub-task
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
> Attachments: HIVE-22600.01.branch-2.patch
>
>
> missing file, probably easy fix
> {code}
> [ERROR] TestAccumuloCliDriver.testCliDriver:59 Unexpected exception
> java.io.FileNotFoundException:
> /Users/lbodor/apache/hive/itests/qtest/target/qfile-results/accumulo-handler/positive/accumulo_queries.q.out
> (No such file or directory)
> at java.io.FileOutputStream.open0(Native Method)
> at java.io.FileOutputStream.open(FileOutputStream.java:270)
> at java.io.FileOutputStream.(FileOutputStream.java:213)
> at java.io.FileOutputStream.(FileOutputStream.java:162)
> at org.apache.hadoop.hive.ql.QTestUtil.cliInit(QTestUtil.java:1158)
> at org.apache.hadoop.hive.ql.QTestUtil.cliInit(QTestUtil.java:1131)
> at
> org.apache.hadoop.hive.cli.control.CoreAccumuloCliDriver.runTest(CoreAccumuloCliDriver.java:88)
> at
> org.apache.hadoop.hive.cli.control.CliAdapter.runTest(CliAdapter.java:104)
> at
> org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver(TestAccumuloCliDriver.java:59)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.apache.hadoop.hive.cli.control.CliAdapter$2$1.evaluate(CliAdapter.java:92)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> at
>
[jira] [Commented] (HIVE-22605) NPE in LlapLoadGeneratorService During Tests
[
https://issues.apache.org/jira/browse/HIVE-22605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16992125#comment-16992125
]
Hive QA commented on HIVE-22605:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12988344/HIVE-22605.1.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 17762 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[alter_table_wrong_db]
(batchId=103)
org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1 (batchId=276)
org.apache.hive.minikdc.TestHs2HooksWithMiniKdc.testHookContexts (batchId=302)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19841/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19841/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19841/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12988344 - PreCommit-HIVE-Build
> NPE in LlapLoadGeneratorService During Tests
>
>
> Key: HIVE-22605
> URL: https://issues.apache.org/jira/browse/HIVE-22605
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HIVE-22605.1.patch
>
>
> {code:none}
> java.lang.NullPointerException: null
> at
> org.apache.hadoop.hive.llap.daemon.impl.LlapLoadGeneratorService.serviceStop(LlapLoadGeneratorService.java:103)
> ~[classes/:?]
> at
> org.apache.hadoop.service.AbstractService.stop(AbstractService.java:220)
> ~[hadoop-common-3.1.0.jar:?]
> at
> org.apache.hadoop.service.ServiceOperations.stop(ServiceOperations.java:54)
> ~[hadoop-common-3.1.0.jar:?]
> at
> org.apache.hadoop.service.ServiceOperations.stopQuietly(ServiceOperations.java:102)
> [hadoop-common-3.1.0.jar:?]
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:172)
> [hadoop-common-3.1.0.jar:?]
> at
> org.apache.hadoop.hive.llap.daemon.impl.TestLlapLoadGeneratorService.testLoadGeneratorFails(TestLlapLoadGeneratorService.java:70)
> [test-classes/:?]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[?:1.8.0_102]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22601) Some columns will be lost when a UDTF has multiple aliases in some cases
[
https://issues.apache.org/jira/browse/HIVE-22601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
okumin updated HIVE-22601:
--
Status: Open (was: Patch Available)
> Some columns will be lost when a UDTF has multiple aliases in some cases
>
>
> Key: HIVE-22601
> URL: https://issues.apache.org/jira/browse/HIVE-22601
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 3.1.2
>Reporter: okumin
>Assignee: okumin
>Priority: Major
> Attachments: HIVE-22601.patch
>
>
> Only one column will be retained when putting UDTFs with multiple aliases and
> a top-level UNION together.
> For example, the result of the following SQL should have three columns, c1,
> c2 and c3.
> {code:java}
> SELECT stack(1, 'a', 'b', 'c') AS (c1, c2, c3)
> UNION ALL
> SELECT stack(1, 'd', 'e', 'f') AS (c1, c2, c3);
> {code}
> However, It's only the c3 column which I can get.
> {code:java}
> +-+
> | _u1.c3 |
> +-+
> | c |
> | f |
> +-+
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22601) Some columns will be lost when a UDTF has multiple aliases in some cases
[
https://issues.apache.org/jira/browse/HIVE-22601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
okumin updated HIVE-22601:
--
Status: Patch Available (was: Open)
> Some columns will be lost when a UDTF has multiple aliases in some cases
>
>
> Key: HIVE-22601
> URL: https://issues.apache.org/jira/browse/HIVE-22601
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 3.1.2
>Reporter: okumin
>Assignee: okumin
>Priority: Major
> Attachments: HIVE-22601.patch
>
>
> Only one column will be retained when putting UDTFs with multiple aliases and
> a top-level UNION together.
> For example, the result of the following SQL should have three columns, c1,
> c2 and c3.
> {code:java}
> SELECT stack(1, 'a', 'b', 'c') AS (c1, c2, c3)
> UNION ALL
> SELECT stack(1, 'd', 'e', 'f') AS (c1, c2, c3);
> {code}
> However, It's only the c3 column which I can get.
> {code:java}
> +-+
> | _u1.c3 |
> +-+
> | c |
> | f |
> +-+
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (HIVE-22561) Data loss on map join for bucketed, partitioned table
[
https://issues.apache.org/jira/browse/HIVE-22561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16992113#comment-16992113
]
Jesus Camacho Rodriguez edited comment on HIVE-22561 at 12/10/19 2:08 AM:
--
[~aditya-shah], can you rebase the patch for branch-3 and branch-3.1? It does
not apply cleanly. Thanks
was (Author: jcamachorodriguez):
[~aditya-shah], can you rebase the patch branch-3 and branch-3.1? It does not
apply cleanly. Thanks
> Data loss on map join for bucketed, partitioned table
> -
>
> Key: HIVE-22561
> URL: https://issues.apache.org/jira/browse/HIVE-22561
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.2
>Reporter: Aditya Shah
>Assignee: Aditya Shah
>Priority: Blocker
> Fix For: 3.0.0, 3.1.0
>
> Attachments: HIVE-22561.patch, Screenshot 2019-11-28 at 8.45.17
> PM.png, image-2019-11-28-20-46-25-432.png
>
>
> A map join on a column (which is neither involved in bucketing and partition)
> causes data loss.
> Steps to reproduce:
> Env: [hive-dev-box|[https://github.com/kgyrtkirk/hive-dev-box]] hive 3.1.2.
> Create tables:
>
> {code:java}
> CREATE TABLE `testj2`(
> `id` int,
> `bn` string,
> `cn` string,
> `ad` map,
> `mi` array)
> PARTITIONED BY (
> `br` string)
> CLUSTERED BY (
> bn)
> INTO 2 BUCKETS
> ROW FORMAT DELIMITED
> FIELDS TERMINATED BY ','
> STORED AS TEXTFILE
> TBLPROPERTIES (
> 'bucketing_version'='2');
> CREATE TABLE `testj1`(
> `id` int,
> `can` string,
> `cn` string,
> `ad` map,
> `av` boolean,
> `mi` array)
> PARTITIONED BY (
> `brand` string)
> CLUSTERED BY (
> can)
> INTO 2 BUCKETS
> ROW FORMAT DELIMITED
> FIELDS TERMINATED BY ','
> STORED AS TEXTFILE
> TBLPROPERTIES (
> 'bucketing_version'='2');
> {code}
> insert some data in both:
> {code:java}
> insert into testj1 values (100, 'mes_1', 'customer_1', map('city1', 560077),
> false, array(5, 10), 'brand_1'),
> (101, 'mes_2', 'customer_2', map('city2', 560078), true, array(10, 20),
> 'brand_2'),
> (102, 'mes_3', 'customer_3', map('city3', 560079), false, array(15, 30),
> 'brand_3'),
> (103, 'mes_4', 'customer_4', map('city4', 560080), true, array(20, 40),
> 'brand_4'),
> (104, 'mes_5', 'customer_5', map('city5', 560081), false, array(25, 50),
> 'brand_5');
> insert into table testj2 values (100, 'tv_0', 'customer_0', map('city0',
> 560076),array(0, 0, 0), 'tv'),
> (101, 'tv_1', 'customer_1', map('city1', 560077),array(20, 25, 30), 'tv'),
> (102, 'tv_2', 'customer_2', map('city2', 560078),array(40, 50, 60), 'tv'),
> (103, 'tv_3', 'customer_3', map('city3', 560079),array(60, 75, 90), 'tv'),
> (104, 'tv_4', 'customer_4', map('city4', 560080),array(80, 100, 120), 'tv');
> {code}
> Do a join between them:
> {code:java}
> select t1.id, t1.can, t1.cn, t2.bn,t2.ad, t2.br FROM testj1 t1 JOIN testj2 t2
> on (t1.id = t2.id) order by t1.id;
> {code}
> Observed results:
> !image-2019-11-28-20-46-25-432.png|width=524,height=100!
> In the plan, I can see a map join. Disabling it gives the correct result.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (HIVE-22561) Data loss on map join for bucketed, partitioned table
[
https://issues.apache.org/jira/browse/HIVE-22561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16992113#comment-16992113
]
Jesus Camacho Rodriguez edited comment on HIVE-22561 at 12/10/19 2:07 AM:
--
[~aditya-shah], can you rebase the patch branch-3 and branch-3.1? It does not
apply cleanly. Thanks
was (Author: jcamachorodriguez):
[~aditya-shah], can you rebase the patch branch-3 and branch-3.1? It does not
apply cleanly. Thanks
> Data loss on map join for bucketed, partitioned table
> -
>
> Key: HIVE-22561
> URL: https://issues.apache.org/jira/browse/HIVE-22561
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.2
>Reporter: Aditya Shah
>Assignee: Aditya Shah
>Priority: Blocker
> Fix For: 3.0.0, 3.1.0
>
> Attachments: HIVE-22561.patch, Screenshot 2019-11-28 at 8.45.17
> PM.png, image-2019-11-28-20-46-25-432.png
>
>
> A map join on a column (which is neither involved in bucketing and partition)
> causes data loss.
> Steps to reproduce:
> Env: [hive-dev-box|[https://github.com/kgyrtkirk/hive-dev-box]] hive 3.1.2.
> Create tables:
>
> {code:java}
> CREATE TABLE `testj2`(
> `id` int,
> `bn` string,
> `cn` string,
> `ad` map,
> `mi` array)
> PARTITIONED BY (
> `br` string)
> CLUSTERED BY (
> bn)
> INTO 2 BUCKETS
> ROW FORMAT DELIMITED
> FIELDS TERMINATED BY ','
> STORED AS TEXTFILE
> TBLPROPERTIES (
> 'bucketing_version'='2');
> CREATE TABLE `testj1`(
> `id` int,
> `can` string,
> `cn` string,
> `ad` map,
> `av` boolean,
> `mi` array)
> PARTITIONED BY (
> `brand` string)
> CLUSTERED BY (
> can)
> INTO 2 BUCKETS
> ROW FORMAT DELIMITED
> FIELDS TERMINATED BY ','
> STORED AS TEXTFILE
> TBLPROPERTIES (
> 'bucketing_version'='2');
> {code}
> insert some data in both:
> {code:java}
> insert into testj1 values (100, 'mes_1', 'customer_1', map('city1', 560077),
> false, array(5, 10), 'brand_1'),
> (101, 'mes_2', 'customer_2', map('city2', 560078), true, array(10, 20),
> 'brand_2'),
> (102, 'mes_3', 'customer_3', map('city3', 560079), false, array(15, 30),
> 'brand_3'),
> (103, 'mes_4', 'customer_4', map('city4', 560080), true, array(20, 40),
> 'brand_4'),
> (104, 'mes_5', 'customer_5', map('city5', 560081), false, array(25, 50),
> 'brand_5');
> insert into table testj2 values (100, 'tv_0', 'customer_0', map('city0',
> 560076),array(0, 0, 0), 'tv'),
> (101, 'tv_1', 'customer_1', map('city1', 560077),array(20, 25, 30), 'tv'),
> (102, 'tv_2', 'customer_2', map('city2', 560078),array(40, 50, 60), 'tv'),
> (103, 'tv_3', 'customer_3', map('city3', 560079),array(60, 75, 90), 'tv'),
> (104, 'tv_4', 'customer_4', map('city4', 560080),array(80, 100, 120), 'tv');
> {code}
> Do a join between them:
> {code:java}
> select t1.id, t1.can, t1.cn, t2.bn,t2.ad, t2.br FROM testj1 t1 JOIN testj2 t2
> on (t1.id = t2.id) order by t1.id;
> {code}
> Observed results:
> !image-2019-11-28-20-46-25-432.png|width=524,height=100!
> In the plan, I can see a map join. Disabling it gives the correct result.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22561) Data loss on map join for bucketed, partitioned table
[
https://issues.apache.org/jira/browse/HIVE-22561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16992113#comment-16992113
]
Jesus Camacho Rodriguez commented on HIVE-22561:
[~aditya-shah], can you rebase the patch branch-3 and branch-3.1? It does not
apply cleanly. Thanks
> Data loss on map join for bucketed, partitioned table
> -
>
> Key: HIVE-22561
> URL: https://issues.apache.org/jira/browse/HIVE-22561
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.2
>Reporter: Aditya Shah
>Assignee: Aditya Shah
>Priority: Blocker
> Fix For: 3.0.0, 3.1.0
>
> Attachments: HIVE-22561.patch, Screenshot 2019-11-28 at 8.45.17
> PM.png, image-2019-11-28-20-46-25-432.png
>
>
> A map join on a column (which is neither involved in bucketing and partition)
> causes data loss.
> Steps to reproduce:
> Env: [hive-dev-box|[https://github.com/kgyrtkirk/hive-dev-box]] hive 3.1.2.
> Create tables:
>
> {code:java}
> CREATE TABLE `testj2`(
> `id` int,
> `bn` string,
> `cn` string,
> `ad` map,
> `mi` array)
> PARTITIONED BY (
> `br` string)
> CLUSTERED BY (
> bn)
> INTO 2 BUCKETS
> ROW FORMAT DELIMITED
> FIELDS TERMINATED BY ','
> STORED AS TEXTFILE
> TBLPROPERTIES (
> 'bucketing_version'='2');
> CREATE TABLE `testj1`(
> `id` int,
> `can` string,
> `cn` string,
> `ad` map,
> `av` boolean,
> `mi` array)
> PARTITIONED BY (
> `brand` string)
> CLUSTERED BY (
> can)
> INTO 2 BUCKETS
> ROW FORMAT DELIMITED
> FIELDS TERMINATED BY ','
> STORED AS TEXTFILE
> TBLPROPERTIES (
> 'bucketing_version'='2');
> {code}
> insert some data in both:
> {code:java}
> insert into testj1 values (100, 'mes_1', 'customer_1', map('city1', 560077),
> false, array(5, 10), 'brand_1'),
> (101, 'mes_2', 'customer_2', map('city2', 560078), true, array(10, 20),
> 'brand_2'),
> (102, 'mes_3', 'customer_3', map('city3', 560079), false, array(15, 30),
> 'brand_3'),
> (103, 'mes_4', 'customer_4', map('city4', 560080), true, array(20, 40),
> 'brand_4'),
> (104, 'mes_5', 'customer_5', map('city5', 560081), false, array(25, 50),
> 'brand_5');
> insert into table testj2 values (100, 'tv_0', 'customer_0', map('city0',
> 560076),array(0, 0, 0), 'tv'),
> (101, 'tv_1', 'customer_1', map('city1', 560077),array(20, 25, 30), 'tv'),
> (102, 'tv_2', 'customer_2', map('city2', 560078),array(40, 50, 60), 'tv'),
> (103, 'tv_3', 'customer_3', map('city3', 560079),array(60, 75, 90), 'tv'),
> (104, 'tv_4', 'customer_4', map('city4', 560080),array(80, 100, 120), 'tv');
> {code}
> Do a join between them:
> {code:java}
> select t1.id, t1.can, t1.cn, t2.bn,t2.ad, t2.br FROM testj1 t1 JOIN testj2 t2
> on (t1.id = t2.id) order by t1.id;
> {code}
> Observed results:
> !image-2019-11-28-20-46-25-432.png|width=524,height=100!
> In the plan, I can see a map join. Disabling it gives the correct result.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22605) NPE in LlapLoadGeneratorService During Tests
[
https://issues.apache.org/jira/browse/HIVE-22605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16992101#comment-16992101
]
Hive QA commented on HIVE-22605:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
57s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
11s{color} | {color:blue} ql in master has 1532 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
45s{color} | {color:blue} llap-server in master has 90 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
19s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 31m 50s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19841/dev-support/hive-personality.sh
|
| git revision | master / d7a193b |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql llap-server U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19841/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> NPE in LlapLoadGeneratorService During Tests
>
>
> Key: HIVE-22605
> URL: https://issues.apache.org/jira/browse/HIVE-22605
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HIVE-22605.1.patch
>
>
> {code:none}
> java.lang.NullPointerException: null
> at
> org.apache.hadoop.hive.llap.daemon.impl.LlapLoadGeneratorService.serviceStop(LlapLoadGeneratorService.java:103)
> ~[classes/:?]
> at
> org.apache.hadoop.service.AbstractService.stop(AbstractService.java:220)
> ~[hadoop-common-3.1.0.jar:?]
> at
> org.apache.hadoop.service.ServiceOperations.stop(ServiceOperations.java:54)
> ~[hadoop-common-3.1.0.jar:?]
> at
> org.apache.hadoop.service.ServiceOperations.stopQuietly(ServiceOperations.java:102)
> [hadoop-common-3.1.0.jar:?]
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:172)
> [hadoop-common-3.1.0.jar:?]
> at
> org.apache.hadoop.hive.llap.daemon.impl.TestLlapLoadGeneratorService.testLoadGeneratorFails(TestLlapLoadGeneratorService.java:70)
>
[jira] [Commented] (HIVE-17442) TestCliDriver.testCliDriver[explaindenpendencydiffengs]
[
https://issues.apache.org/jira/browse/HIVE-17442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16992070#comment-16992070
]
Hive QA commented on HIVE-17442:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12988339/HIVE-17442.01.branch-2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 10756 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_queries]
(batchId=227)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[avro_tableproperty_optimize]
(batchId=22)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[table_nonprintable]
(batchId=140)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[join_acid_non_acid]
(batchId=158)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[union_fast_stats]
(batchId=153)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorized_parquet_types]
(batchId=155)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[merge_negative_5]
(batchId=88)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[explaindenpendencydiffengs]
(batchId=115)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vectorization_input_format_excludes]
(batchId=117)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vectorized_ptf]
(batchId=125)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19840/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19840/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19840/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 10 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12988339 - PreCommit-HIVE-Build
> TestCliDriver.testCliDriver[explaindenpendencydiffengs]
> ---
>
> Key: HIVE-17442
> URL: https://issues.apache.org/jira/browse/HIVE-17442
> Project: Hive
> Issue Type: Sub-task
>Reporter: Vihang Karajgaonkar
>Assignee: László Bodor
>Priority: Major
> Attachments: HIVE-17442.01.branch-2.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22601) Some columns will be lost when a UDTF has multiple aliases in some cases
[
https://issues.apache.org/jira/browse/HIVE-22601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16992045#comment-16992045
]
Hive QA commented on HIVE-22601:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12988340/HIVE-22601.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17764 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1 (batchId=276)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19839/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19839/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19839/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12988340 - PreCommit-HIVE-Build
> Some columns will be lost when a UDTF has multiple aliases in some cases
>
>
> Key: HIVE-22601
> URL: https://issues.apache.org/jira/browse/HIVE-22601
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 3.1.2
>Reporter: okumin
>Assignee: okumin
>Priority: Major
> Attachments: HIVE-22601.patch
>
>
> Only one column will be retained when putting UDTFs with multiple aliases and
> a top-level UNION together.
> For example, the result of the following SQL should have three columns, c1,
> c2 and c3.
> {code:java}
> SELECT stack(1, 'a', 'b', 'c') AS (c1, c2, c3)
> UNION ALL
> SELECT stack(1, 'd', 'e', 'f') AS (c1, c2, c3);
> {code}
> However, It's only the c3 column which I can get.
> {code:java}
> +-+
> | _u1.c3 |
> +-+
> | c |
> | f |
> +-+
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22508) KeyWrapperComparator throws exception
[ https://issues.apache.org/jira/browse/HIVE-22508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16992022#comment-16992022 ] Jesus Camacho Rodriguez commented on HIVE-22508: +1 (pending tests) > KeyWrapperComparator throws exception > - > > Key: HIVE-22508 > URL: https://issues.apache.org/jira/browse/HIVE-22508 > Project: Hive > Issue Type: Bug > Components: Physical Optimizer >Reporter: Krisztian Kasa >Assignee: Krisztian Kasa >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22508.1.patch, HIVE-22508.1.patch, > HIVE-22508.1.patch, HIVE-22508.2.patch, HIVE-22508.2.patch > > > TopNKeyOperator.KeyWrapperComparator throws exception when a new key and a > copied key should be compared. > The current implementation uses the standard object inspectors for all > KeyWrapper instances. However when comparing untouched KeyWrappers the key > object inspector should be used which can be extracted form > Operator.inputObjectInspectors during initialization of the key's > ExprNodeEvaluator. > This can cause a ClassCastException when the comparator is used collections > like TreeSet. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22601) Some columns will be lost when a UDTF has multiple aliases in some cases
[
https://issues.apache.org/jira/browse/HIVE-22601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16992020#comment-16992020
]
Hive QA commented on HIVE-22601:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
24s{color} | {color:blue} ql in master has 1532 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
47s{color} | {color:red} ql: The patch generated 10 new + 278 unchanged - 2
fixed = 288 total (was 280) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 27m 20s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19839/dev-support/hive-personality.sh
|
| git revision | master / d7a193b |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19839/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19839/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Some columns will be lost when a UDTF has multiple aliases in some cases
>
>
> Key: HIVE-22601
> URL: https://issues.apache.org/jira/browse/HIVE-22601
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 3.1.2
>Reporter: okumin
>Assignee: okumin
>Priority: Major
> Attachments: HIVE-22601.patch
>
>
> Only one column will be retained when putting UDTFs with multiple aliases and
> a top-level UNION together.
> For example, the result of the following SQL should have three columns, c1,
> c2 and c3.
> {code:java}
> SELECT stack(1, 'a', 'b', 'c') AS (c1, c2, c3)
> UNION ALL
> SELECT stack(1, 'd', 'e', 'f') AS (c1, c2, c3);
> {code}
> However, It's only the c3 column which I can get.
> {code:java}
> +-+
> | _u1.c3 |
> +-+
> | c |
> | f |
> +-+
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-20150) TopNKey pushdown
[
https://issues.apache.org/jira/browse/HIVE-20150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16992007#comment-16992007
]
Hive QA commented on HIVE-20150:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12988348/HIVE-20150.24.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 6 failed/errored test(s), 17768 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1 (batchId=276)
org.apache.hadoop.hive.ql.optimizer.topnkey.TestCommonKeyPrefix.testmapWhenAllColumnsMatchButNullOrderMismatch
(batchId=333)
org.apache.hadoop.hive.ql.optimizer.topnkey.TestCommonKeyPrefix.testmapWhenAllColumnsMatchButOrderMismatch
(batchId=333)
org.apache.hadoop.hive.ql.optimizer.topnkey.TestCommonKeyPrefix.testmapWhenAllKeysMatch
(batchId=333)
org.apache.hadoop.hive.ql.optimizer.topnkey.TestCommonKeyPrefix.testmapWhenKeyCountsMismatch
(batchId=333)
org.apache.hadoop.hive.ql.optimizer.topnkey.TestCommonKeyPrefix.testmapWhenOnlyFirstKeyMatchFromTwo
(batchId=333)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19838/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19838/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19838/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 6 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12988348 - PreCommit-HIVE-Build
> TopNKey pushdown
>
>
> Key: HIVE-20150
> URL: https://issues.apache.org/jira/browse/HIVE-20150
> Project: Hive
> Issue Type: Improvement
> Components: Physical Optimizer
>Affects Versions: 4.0.0
>Reporter: Teddy Choi
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-20150.1.patch, HIVE-20150.10.patch,
> HIVE-20150.11.patch, HIVE-20150.11.patch, HIVE-20150.14.patch,
> HIVE-20150.15.patch, HIVE-20150.16.patch, HIVE-20150.17.patch,
> HIVE-20150.17.patch, HIVE-20150.18.patch, HIVE-20150.18.patch,
> HIVE-20150.19.patch, HIVE-20150.2.patch, HIVE-20150.20.patch,
> HIVE-20150.21.patch, HIVE-20150.22.patch, HIVE-20150.23.patch,
> HIVE-20150.24.patch, HIVE-20150.4.patch, HIVE-20150.5.patch,
> HIVE-20150.6.patch, HIVE-20150.7.patch, HIVE-20150.8.patch, HIVE-20150.9.patch
>
>
> TopNKey operator is implemented in HIVE-17896, but it needs more work in
> pushdown implementation. So this issue covers TopNKey pushdown implementation
> with proper tests.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-20150) TopNKey pushdown
[
https://issues.apache.org/jira/browse/HIVE-20150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16992001#comment-16992001
]
Hive QA commented on HIVE-20150:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
49s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
21s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
35s{color} | {color:blue} common in master has 65 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
28s{color} | {color:blue} ql in master has 1532 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 9m
8s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 8m
57s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
42s{color} | {color:red} ql: The patch generated 3 new + 36 unchanged - 2 fixed
= 39 total (was 38) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m
19s{color} | {color:red} root: The patch generated 3 new + 406 unchanged - 2
fixed = 409 total (was 408) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
24s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 1m
5s{color} | {color:red} ql generated 2 new + 98 unchanged - 2 fixed = 100 total
(was 100) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 7m
26s{color} | {color:red} root generated 2 new + 335 unchanged - 2 fixed = 337
total (was 337) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 75m 38s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile
xml |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19838/dev-support/hive-personality.sh
|
| git revision | master / d7a193b |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19838/yetus/diff-checkstyle-ql.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19838/yetus/diff-checkstyle-root.txt
|
| javadoc |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19838/yetus/diff-javadoc-javadoc-ql.txt
|
| javadoc |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19838/yetus/diff-javadoc-javadoc-root.txt
|
| modules | C: common ql . U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19838/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> TopNKey pushdown
>
>
> Key: HIVE-20150
> URL: https://issues.apache.org/jira/browse/HIVE-20150
> Project: Hive
> Issue Type: Improvement
>
[jira] [Updated] (HIVE-22508) KeyWrapperComparator throws exception
[ https://issues.apache.org/jira/browse/HIVE-22508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-22508: -- Status: Patch Available (was: Open) > KeyWrapperComparator throws exception > - > > Key: HIVE-22508 > URL: https://issues.apache.org/jira/browse/HIVE-22508 > Project: Hive > Issue Type: Bug > Components: Physical Optimizer >Reporter: Krisztian Kasa >Assignee: Krisztian Kasa >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22508.1.patch, HIVE-22508.1.patch, > HIVE-22508.1.patch, HIVE-22508.2.patch, HIVE-22508.2.patch > > > TopNKeyOperator.KeyWrapperComparator throws exception when a new key and a > copied key should be compared. > The current implementation uses the standard object inspectors for all > KeyWrapper instances. However when comparing untouched KeyWrappers the key > object inspector should be used which can be extracted form > Operator.inputObjectInspectors during initialization of the key's > ExprNodeEvaluator. > This can cause a ClassCastException when the comparator is used collections > like TreeSet. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22508) KeyWrapperComparator throws exception
[ https://issues.apache.org/jira/browse/HIVE-22508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-22508: -- Status: Open (was: Patch Available) > KeyWrapperComparator throws exception > - > > Key: HIVE-22508 > URL: https://issues.apache.org/jira/browse/HIVE-22508 > Project: Hive > Issue Type: Bug > Components: Physical Optimizer >Reporter: Krisztian Kasa >Assignee: Krisztian Kasa >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22508.1.patch, HIVE-22508.1.patch, > HIVE-22508.1.patch, HIVE-22508.2.patch, HIVE-22508.2.patch > > > TopNKeyOperator.KeyWrapperComparator throws exception when a new key and a > copied key should be compared. > The current implementation uses the standard object inspectors for all > KeyWrapper instances. However when comparing untouched KeyWrappers the key > object inspector should be used which can be extracted form > Operator.inputObjectInspectors during initialization of the key's > ExprNodeEvaluator. > This can cause a ClassCastException when the comparator is used collections > like TreeSet. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22508) KeyWrapperComparator throws exception
[ https://issues.apache.org/jira/browse/HIVE-22508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-22508: -- Attachment: HIVE-22508.2.patch > KeyWrapperComparator throws exception > - > > Key: HIVE-22508 > URL: https://issues.apache.org/jira/browse/HIVE-22508 > Project: Hive > Issue Type: Bug > Components: Physical Optimizer >Reporter: Krisztian Kasa >Assignee: Krisztian Kasa >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22508.1.patch, HIVE-22508.1.patch, > HIVE-22508.1.patch, HIVE-22508.2.patch, HIVE-22508.2.patch > > > TopNKeyOperator.KeyWrapperComparator throws exception when a new key and a > copied key should be compared. > The current implementation uses the standard object inspectors for all > KeyWrapper instances. However when comparing untouched KeyWrappers the key > object inspector should be used which can be extracted form > Operator.inputObjectInspectors during initialization of the key's > ExprNodeEvaluator. > This can cause a ClassCastException when the comparator is used collections > like TreeSet. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22553) Expose locks and transactions in sys db
[
https://issues.apache.org/jira/browse/HIVE-22553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16991946#comment-16991946
]
Hive QA commented on HIVE-22553:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12988346/HIVE-22553.7.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17762 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19837/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19837/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19837/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12988346 - PreCommit-HIVE-Build
> Expose locks and transactions in sys db
> ---
>
> Key: HIVE-22553
> URL: https://issues.apache.org/jira/browse/HIVE-22553
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 4.0.0
>Reporter: Zoltan Chovan
>Assignee: Zoltan Chovan
>Priority: Major
> Attachments: HIVE-22553.2.patch, HIVE-22553.3.patch,
> HIVE-22553.3.patch, HIVE-22553.4.patch, HIVE-22553.5.patch,
> HIVE-22553.6.patch, HIVE-22553.7.patch, HIVE-22553.patch
>
>
> Create new sysdb tables/views to access lock and transaction data.
> This allows to provide admins with live data about ongoing locks and
> transacions. Due to this being in the sys db access to this information can
> be restricted to select privileged users.
> Information about locks and compactions can be joined and accessed at the
> same time.
> Compaction related transactions would also be visible.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22578) CBO genOPTree is not failsafe for CTAS and VIEW statements
[ https://issues.apache.org/jira/browse/HIVE-22578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16991943#comment-16991943 ] Zoltan Haindrich commented on HIVE-22578: - +1 > CBO genOPTree is not failsafe for CTAS and VIEW statements > -- > > Key: HIVE-22578 > URL: https://issues.apache.org/jira/browse/HIVE-22578 > Project: Hive > Issue Type: Bug > Components: CBO >Affects Versions: 3.0.0, 3.1.2 >Reporter: Aron Hamvas >Assignee: Aron Hamvas >Priority: Major > Fix For: 3.0.1, 4.0.0, 3.1.3 > > Attachments: HIVE-22578.2.patch, HIVE-22578.patch > > > If CBO fails during OT generation, it is supposed to skip CBO and fall back > to SemanticAnalyzer generating the OT for the original AST. However, for CTAS > and VIEW statements, the original AST is discarded and replaced with the new > AST. > If OT generation for the new AST fails with a SemanticException, the > compilation is destined to fail completely since we cannot fall back to the > original and possibly valid AST. > Also, the SemanticException sometimes gets hidden and misjudged as a missing > column statistics issue due to bad error handling. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22553) Expose locks and transactions in sys db
[
https://issues.apache.org/jira/browse/HIVE-22553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16991883#comment-16991883
]
Hive QA commented on HIVE-22553:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
55s{color} | {color:blue} Maven dependency ordering for branch {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
11s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 3m 5s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19837/dev-support/hive-personality.sh
|
| git revision | master / d7a193b |
| modules | C: metastore ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19837/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Expose locks and transactions in sys db
> ---
>
> Key: HIVE-22553
> URL: https://issues.apache.org/jira/browse/HIVE-22553
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 4.0.0
>Reporter: Zoltan Chovan
>Assignee: Zoltan Chovan
>Priority: Major
> Attachments: HIVE-22553.2.patch, HIVE-22553.3.patch,
> HIVE-22553.3.patch, HIVE-22553.4.patch, HIVE-22553.5.patch,
> HIVE-22553.6.patch, HIVE-22553.7.patch, HIVE-22553.patch
>
>
> Create new sysdb tables/views to access lock and transaction data.
> This allows to provide admins with live data about ongoing locks and
> transacions. Due to this being in the sys db access to this information can
> be restricted to select privileged users.
> Information about locks and compactions can be joined and accessed at the
> same time.
> Compaction related transactions would also be visible.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22597) Include More Context in Database NoSuchObjectException
[
https://issues.apache.org/jira/browse/HIVE-22597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16991882#comment-16991882
]
David Mollitor commented on HIVE-22597:
---
[~abstractdog] Are you able to take a look at this one too? Thanks! :)
> Include More Context in Database NoSuchObjectException
> --
>
> Key: HIVE-22597
> URL: https://issues.apache.org/jira/browse/HIVE-22597
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HIVE-22597.1.patch, HIVE-22597.2.patch
>
>
> {code:none}
> org.apache.hadoop.hive.metastore.api.NoSuchObjectException: default
> at
> org.apache.hadoop.hive.metastore.ObjectStore.getDatabase(ObjectStore.java:717)
> ~[hive-standalone-metastore-server-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at sun.reflect.GeneratedMethodAccessor260.invoke(Unknown Source) ~[?:?]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_102]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_102]
> at
> org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:97)
> ~[hive-standalone-metastore-server-4.0.0-SN
> {code}
> One can decipher that this exception is in regards to a database by looking
> at the stack trace, but it should be specified in the error message itself.
> Also, there is no catalogue information provided, so it could be a bit
> ambiguous.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22597) Include More Context in Database NoSuchObjectException
[
https://issues.apache.org/jira/browse/HIVE-22597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16991878#comment-16991878
]
Hive QA commented on HIVE-22597:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12988330/HIVE-22597.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17762 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19836/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19836/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19836/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12988330 - PreCommit-HIVE-Build
> Include More Context in Database NoSuchObjectException
> --
>
> Key: HIVE-22597
> URL: https://issues.apache.org/jira/browse/HIVE-22597
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HIVE-22597.1.patch, HIVE-22597.2.patch
>
>
> {code:none}
> org.apache.hadoop.hive.metastore.api.NoSuchObjectException: default
> at
> org.apache.hadoop.hive.metastore.ObjectStore.getDatabase(ObjectStore.java:717)
> ~[hive-standalone-metastore-server-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at sun.reflect.GeneratedMethodAccessor260.invoke(Unknown Source) ~[?:?]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_102]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_102]
> at
> org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:97)
> ~[hive-standalone-metastore-server-4.0.0-SN
> {code}
> One can decipher that this exception is in regards to a database by looking
> at the stack trace, but it should be specified in the error message itself.
> Also, there is no catalogue information provided, so it could be a bit
> ambiguous.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22597) Include More Context in Database NoSuchObjectException
[
https://issues.apache.org/jira/browse/HIVE-22597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16991848#comment-16991848
]
Hive QA commented on HIVE-22597:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
58s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
25s{color} | {color:blue} standalone-metastore/metastore-server in master has
179 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
21s{color} | {color:blue} ql in master has 1532 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
28s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
29s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
23s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 33m 16s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19836/dev-support/hive-personality.sh
|
| git revision | master / d7a193b |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: standalone-metastore/metastore-server ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19836/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Include More Context in Database NoSuchObjectException
> --
>
> Key: HIVE-22597
> URL: https://issues.apache.org/jira/browse/HIVE-22597
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HIVE-22597.1.patch, HIVE-22597.2.patch
>
>
> {code:none}
> org.apache.hadoop.hive.metastore.api.NoSuchObjectException: default
> at
> org.apache.hadoop.hive.metastore.ObjectStore.getDatabase(ObjectStore.java:717)
> ~[hive-standalone-metastore-server-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at sun.reflect.GeneratedMethodAccessor260.invoke(Unknown Source) ~[?:?]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_102]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_102]
> at
> org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:97)
> ~[hive-standalone-metastore-server-4.0.0-SN
> {code}
> One can decipher that this exception is in regards to a database by
[jira] [Commented] (HIVE-22569) PartitionPruner use Collections Class
[
https://issues.apache.org/jira/browse/HIVE-22569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16991837#comment-16991837
]
David Mollitor commented on HIVE-22569:
---
[~vgarg] Mind taking a look again? Thanks!
> PartitionPruner use Collections Class
> -
>
> Key: HIVE-22569
> URL: https://issues.apache.org/jira/browse/HIVE-22569
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HIVE-22569.2.patch, HIVE.22569.1.patch
>
>
> Make class {{PrunedPartitionList.java}} more immutable. It kinda is already.
> Immutable classes are easier to reason about.
> Use {{Collections.emptyXXX}} in {{PartitionPruner}} to save an object
> instantiation at runtime.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22569) PartitionPruner use Collections Class
[
https://issues.apache.org/jira/browse/HIVE-22569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-22569:
--
Attachment: (was: HIVE-22569.2.patch)
> PartitionPruner use Collections Class
> -
>
> Key: HIVE-22569
> URL: https://issues.apache.org/jira/browse/HIVE-22569
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HIVE-22569.2.patch, HIVE.22569.1.patch
>
>
> Make class {{PrunedPartitionList.java}} more immutable. It kinda is already.
> Immutable classes are easier to reason about.
> Use {{Collections.emptyXXX}} in {{PartitionPruner}} to save an object
> instantiation at runtime.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22569) PartitionPruner use Collections Class
[
https://issues.apache.org/jira/browse/HIVE-22569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-22569:
--
Attachment: (was: HIVE-22569.2.patch)
> PartitionPruner use Collections Class
> -
>
> Key: HIVE-22569
> URL: https://issues.apache.org/jira/browse/HIVE-22569
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HIVE-22569.2.patch, HIVE.22569.1.patch
>
>
> Make class {{PrunedPartitionList.java}} more immutable. It kinda is already.
> Immutable classes are easier to reason about.
> Use {{Collections.emptyXXX}} in {{PartitionPruner}} to save an object
> instantiation at runtime.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22569) PartitionPruner use Collections Class
[
https://issues.apache.org/jira/browse/HIVE-22569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16991827#comment-16991827
]
Hive QA commented on HIVE-22569:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12988331/HIVE-22569.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17762 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19835/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19835/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19835/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12988331 - PreCommit-HIVE-Build
> PartitionPruner use Collections Class
> -
>
> Key: HIVE-22569
> URL: https://issues.apache.org/jira/browse/HIVE-22569
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HIVE-22569.2.patch, HIVE-22569.2.patch,
> HIVE-22569.2.patch, HIVE.22569.1.patch
>
>
> Make class {{PrunedPartitionList.java}} more immutable. It kinda is already.
> Immutable classes are easier to reason about.
> Use {{Collections.emptyXXX}} in {{PartitionPruner}} to save an object
> instantiation at runtime.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22461) NPE Metastore Transformer
[
https://issues.apache.org/jira/browse/HIVE-22461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yongzhi Chen updated HIVE-22461:
Attachment: HIVE-22461.6.patch
> NPE Metastore Transformer
> -
>
> Key: HIVE-22461
> URL: https://issues.apache.org/jira/browse/HIVE-22461
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.1.2
>Reporter: Yongzhi Chen
>Assignee: Yongzhi Chen
>Priority: Major
> Attachments: HIVE-22461.1.patch, HIVE-22461.5.patch,
> HIVE-22461.6.patch
>
>
> The stack looks as following:
> {noformat}
> 2019-10-08 18:09:12,198 INFO
> org.apache.hadoop.hive.metastore.MetastoreDefaultTransformer:
> [pool-6-thread-328]: Starting translation for processor
> Hiveserver2#3.1.2000.7.0.2.0...@vc0732.halxg.cloudera.com on list 1
> 2019-10-08 18:09:12,198 ERROR
> org.apache.hadoop.hive.metastore.RetryingHMSHandler: [pool-6-thread-328]:
> java.lang.NullPointerException
> at
> org.apache.hadoop.hive.metastore.MetastoreDefaultTransformer.transform(MetastoreDefaultTransformer.java:99)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getTableInternal(HiveMetaStore.java:3391)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table_req(HiveMetaStore.java:3352)
> at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
> at com.sun.proxy.$Proxy28.get_table_req(Unknown Source)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_table_req.getResult(ThriftHiveMetastore.java:16633)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_table_req.getResult(ThriftHiveMetastore.java:16617)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> at
> org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge.java:636)
> at
> org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge.java:631)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
> at
> org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:631)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> 2019-10-08 18:09:12,199 ERROR org.apache.thrift.server.TThreadPoolServer:
> [pool-6-thread-328]: Error occurred during processing of message.
> java.lang.NullPointerException: null
> at
> org.apache.hadoop.hive.metastore.MetastoreDefaultTransformer.transform(MetastoreDefaultTransformer.java:99)
> ~[hive-exec-3.1.2000.7.0.2.0-59.jar:3.1.2000.7.0.2.0-59]
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getTableInternal(HiveMetaStore.java:3391)
> ~[hive-exec-3.1.2000.7.0.2.0-59.jar:3.1.2000.7.0.2.0-59]
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table_req(HiveMetaStore.java:3352)
> ~[hive-exec-3.1.2000.7.0.2.0-59.jar:3.1.2000.7.0.2.0-59]
> at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source) ~[?:?]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_141]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_141]
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
> ~[hive-exec-3.1.2000.7.0.2.0-59.jar:3.1.2000.7.0.2.0-59]
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
> ~[hive-exec-3.1.2000.7.0.2.0-59.jar:3.1.2000.7.0.2.0-59]
> at com.sun.proxy.$Proxy28.get_table_req(Unknown Source) ~[?:?]
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_table_req.getResult(ThriftHiveMetastore.java:16633)
> ~[hive-exec-3.1.2000.7.0.2.0-59.jar:3.1.2000.7.0.2.0-59]
>
[jira] [Updated] (HIVE-22461) NPE Metastore Transformer
[
https://issues.apache.org/jira/browse/HIVE-22461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yongzhi Chen updated HIVE-22461:
Attachment: (was: HIVE-22461.6.patch)
> NPE Metastore Transformer
> -
>
> Key: HIVE-22461
> URL: https://issues.apache.org/jira/browse/HIVE-22461
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.1.2
>Reporter: Yongzhi Chen
>Assignee: Yongzhi Chen
>Priority: Major
> Attachments: HIVE-22461.1.patch, HIVE-22461.5.patch
>
>
> The stack looks as following:
> {noformat}
> 2019-10-08 18:09:12,198 INFO
> org.apache.hadoop.hive.metastore.MetastoreDefaultTransformer:
> [pool-6-thread-328]: Starting translation for processor
> Hiveserver2#3.1.2000.7.0.2.0...@vc0732.halxg.cloudera.com on list 1
> 2019-10-08 18:09:12,198 ERROR
> org.apache.hadoop.hive.metastore.RetryingHMSHandler: [pool-6-thread-328]:
> java.lang.NullPointerException
> at
> org.apache.hadoop.hive.metastore.MetastoreDefaultTransformer.transform(MetastoreDefaultTransformer.java:99)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getTableInternal(HiveMetaStore.java:3391)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table_req(HiveMetaStore.java:3352)
> at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
> at com.sun.proxy.$Proxy28.get_table_req(Unknown Source)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_table_req.getResult(ThriftHiveMetastore.java:16633)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_table_req.getResult(ThriftHiveMetastore.java:16617)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> at
> org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge.java:636)
> at
> org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge.java:631)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
> at
> org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:631)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> 2019-10-08 18:09:12,199 ERROR org.apache.thrift.server.TThreadPoolServer:
> [pool-6-thread-328]: Error occurred during processing of message.
> java.lang.NullPointerException: null
> at
> org.apache.hadoop.hive.metastore.MetastoreDefaultTransformer.transform(MetastoreDefaultTransformer.java:99)
> ~[hive-exec-3.1.2000.7.0.2.0-59.jar:3.1.2000.7.0.2.0-59]
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getTableInternal(HiveMetaStore.java:3391)
> ~[hive-exec-3.1.2000.7.0.2.0-59.jar:3.1.2000.7.0.2.0-59]
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table_req(HiveMetaStore.java:3352)
> ~[hive-exec-3.1.2000.7.0.2.0-59.jar:3.1.2000.7.0.2.0-59]
> at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source) ~[?:?]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_141]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_141]
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
> ~[hive-exec-3.1.2000.7.0.2.0-59.jar:3.1.2000.7.0.2.0-59]
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
> ~[hive-exec-3.1.2000.7.0.2.0-59.jar:3.1.2000.7.0.2.0-59]
> at com.sun.proxy.$Proxy28.get_table_req(Unknown Source) ~[?:?]
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_table_req.getResult(ThriftHiveMetastore.java:16633)
> ~[hive-exec-3.1.2000.7.0.2.0-59.jar:3.1.2000.7.0.2.0-59]
> at
>
[jira] [Commented] (HIVE-22607) Session State Thread Names Are Way Too Long
[
https://issues.apache.org/jira/browse/HIVE-22607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16991809#comment-16991809
]
David Mollitor commented on HIVE-22607:
---
Put up patch that clears that name every time a session is closed so that if a
developer forgets to reset the thread name, it will be reset eventually. Patch
also so clean up, for example, the current code looks like it clears the name
from the front of the list only, but it's a bit ambiguous because it uses
'split' instead of subString to remove the front of the string. Changed it to
make it remove the session ID no matter where it is in the string so that it's
more clear and flexible.
> Session State Thread Names Are Way Too Long
> ---
>
> Key: HIVE-22607
> URL: https://issues.apache.org/jira/browse/HIVE-22607
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-22607.1.patch
>
>
> {code:none}
> 2019-12-08T07:15:34,107 DEBUG [bc661ab9-b44a-4ee5-8ce6-b8b7ae5a0e07
> d480b94c-8773-4817-85fc-faa177308660 73ecdf15-06fd-4358-a249-22623c98d234
> 37e3c8de-9c77-4939-94d0-5aa45501b545 7b152899-d8da-493f-a329-a16f171bb1a3
> 604abf47-22ed-480c-a460-f6d44ac740ec 91fee8a8-aee4-4eee-993f-ad2927784d57
> c496e18f-a681-4db8-b254-6911793e7fb2 c6069d3c-03fb-4e1d-b0be-5dd13882b086
> 3c360edd-9c0f-486b-99f9-dd9f8aac5fcd c2be6f69-3ef5-44e4-b8d5-16b835b0ae2f
> 3bd2c6d2-9cce-4497-aa07-4672b95e6f76 ccb64f47-b1d7-477b-81ed-f8baabae4ee4
> 8b244038-d7a3-4e11-ab2f-b42d117c8e40 8183bded-ef37-4bdc-ab17-a4c4b136401d
> 8b161c72-0fc2-4175-ac9f-9a0c7bf9b387 38fd3a6d-498a-4145-b77e-24e33c31edaf
> 70f2729d-7249-4397-b04f-22694fec391e c65a3e39-009d-4e8a-b16e-1ca0e9de7cef
> 1dd2b274-75bd-4204-80b9-0103fefd0227 0bab6264-40ff-4f05-8d11-0cc9ee3132c0
> 0e7e677b-3f24-408b-ac1e-ddeee54023b8 8fd57ca5-2128-45a1-b7c4-ec611bfdefcd
> 01bf788e-cf98-44e8-abaa-c9e74e782bde 9d27f963-b45b-4991-baa8-2ed2db41857d
> 207da47a-b3b2-4583-b654-61fc551f4eca a10d8b77-8f27-48ca-a3b5-e5cfcbf8a4c0
> 352e77d5-3071-4502-bb0e-a6d0df8d1cac 18697755-6e2b-4907-ae8b-11ee0ff2f057
> d779ca40-0760-4b27-b0b1-c35d985359c1 ca4469aa-7be7-4eaf-9bcd-aeff6441c7c0
> 30ce1107-8e95-4c16-b55a-5a7c4dea0696 ee44b107-07ae-45c9-88a4-0424da8a6bcb
> main] session.SessionState: Updating thread name to
> 749f4123-716a-4534-af7f-b426fdc3ccc9 bc661ab9-b44a-4ee5-8ce6-b8b7ae5a0e07
> d480b94c-8773-4817-85fc-faa177308660 73ecdf15-06fd-4358-a249-22623c98d234
> 37e3c8de-9c77-4939-94d0-5aa45501b545 7b152899-d8da-493f-a329-a16f171bb1a3
> 604abf47-22ed-480c-a460-f6d44ac740ec 91fee8a8-aee4-4eee-993f-ad2927784d57
> c496e18f-a681-4db8-b254-6911793e7fb2 c6069d3c-03fb-4e1d-b0be-5dd13882b086
> 3c360edd-9c0f-486b-99f9-dd9f8aac5fcd c2be6f69-3ef5-44e4-b8d5-16b835b0ae2f
> 3bd2c6d2-9cce-4497-aa07-4672b95e6f76 ccb64f47-b1d7-477b-81ed-f8baabae4ee4
> 8b244038-d7a3-4e11-ab2f-b42d117c8e40 8183bded-ef37-4bdc-ab17-a4c4b136401d
> 8b161c72-0fc2-4175-ac9f-9a0c7bf9b387 38fd3a6d-498a-4145-b77e-24e33c31edaf
> 70f2729d-7249-4397-b04f-22694fec391e c65a3e39-009d-4e8a-b16e-1ca0e9de7cef
> 1dd2b274-75bd-4204-80b9-0103fefd0227 0bab6264-40ff-4f05-8d11-0cc9ee3132c0
> 0e7e677b-3f24-408b-ac1e-ddeee54023b8 8fd57ca5-2128-45a1-b7c4-ec611bfdefcd
> 01bf788e-cf98-44e8-abaa-c9e74e782bde 9d27f963-b45b-4991-baa8-2ed2db41857d
> 207da47a-b3b2-4583-b654-61fc551f4eca a10d8b77-8f27-48ca-a3b5-e5cfcbf8a4c0
> 352e77d5-3071-4502-bb0e-a6d0df8d1cac 18697755-6e2b-4907-ae8b-11ee0ff2f057
> d779ca40-0760-4b27-b0b1-c35d985359c1 ca4469aa-7be7-4eaf-9bcd-aeff6441c7c0
> 30ce1107-8e95-4c16-b55a-5a7c4dea0696 ee44b107-07ae-45c9-88a4-0424da8a6bcb main
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-22607) Session State Thread Names Are Way Too Long
[
https://issues.apache.org/jira/browse/HIVE-22607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor reassigned HIVE-22607:
-
Assignee: David Mollitor
> Session State Thread Names Are Way Too Long
> ---
>
> Key: HIVE-22607
> URL: https://issues.apache.org/jira/browse/HIVE-22607
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-22607.1.patch
>
>
> {code:none}
> 2019-12-08T07:15:34,107 DEBUG [bc661ab9-b44a-4ee5-8ce6-b8b7ae5a0e07
> d480b94c-8773-4817-85fc-faa177308660 73ecdf15-06fd-4358-a249-22623c98d234
> 37e3c8de-9c77-4939-94d0-5aa45501b545 7b152899-d8da-493f-a329-a16f171bb1a3
> 604abf47-22ed-480c-a460-f6d44ac740ec 91fee8a8-aee4-4eee-993f-ad2927784d57
> c496e18f-a681-4db8-b254-6911793e7fb2 c6069d3c-03fb-4e1d-b0be-5dd13882b086
> 3c360edd-9c0f-486b-99f9-dd9f8aac5fcd c2be6f69-3ef5-44e4-b8d5-16b835b0ae2f
> 3bd2c6d2-9cce-4497-aa07-4672b95e6f76 ccb64f47-b1d7-477b-81ed-f8baabae4ee4
> 8b244038-d7a3-4e11-ab2f-b42d117c8e40 8183bded-ef37-4bdc-ab17-a4c4b136401d
> 8b161c72-0fc2-4175-ac9f-9a0c7bf9b387 38fd3a6d-498a-4145-b77e-24e33c31edaf
> 70f2729d-7249-4397-b04f-22694fec391e c65a3e39-009d-4e8a-b16e-1ca0e9de7cef
> 1dd2b274-75bd-4204-80b9-0103fefd0227 0bab6264-40ff-4f05-8d11-0cc9ee3132c0
> 0e7e677b-3f24-408b-ac1e-ddeee54023b8 8fd57ca5-2128-45a1-b7c4-ec611bfdefcd
> 01bf788e-cf98-44e8-abaa-c9e74e782bde 9d27f963-b45b-4991-baa8-2ed2db41857d
> 207da47a-b3b2-4583-b654-61fc551f4eca a10d8b77-8f27-48ca-a3b5-e5cfcbf8a4c0
> 352e77d5-3071-4502-bb0e-a6d0df8d1cac 18697755-6e2b-4907-ae8b-11ee0ff2f057
> d779ca40-0760-4b27-b0b1-c35d985359c1 ca4469aa-7be7-4eaf-9bcd-aeff6441c7c0
> 30ce1107-8e95-4c16-b55a-5a7c4dea0696 ee44b107-07ae-45c9-88a4-0424da8a6bcb
> main] session.SessionState: Updating thread name to
> 749f4123-716a-4534-af7f-b426fdc3ccc9 bc661ab9-b44a-4ee5-8ce6-b8b7ae5a0e07
> d480b94c-8773-4817-85fc-faa177308660 73ecdf15-06fd-4358-a249-22623c98d234
> 37e3c8de-9c77-4939-94d0-5aa45501b545 7b152899-d8da-493f-a329-a16f171bb1a3
> 604abf47-22ed-480c-a460-f6d44ac740ec 91fee8a8-aee4-4eee-993f-ad2927784d57
> c496e18f-a681-4db8-b254-6911793e7fb2 c6069d3c-03fb-4e1d-b0be-5dd13882b086
> 3c360edd-9c0f-486b-99f9-dd9f8aac5fcd c2be6f69-3ef5-44e4-b8d5-16b835b0ae2f
> 3bd2c6d2-9cce-4497-aa07-4672b95e6f76 ccb64f47-b1d7-477b-81ed-f8baabae4ee4
> 8b244038-d7a3-4e11-ab2f-b42d117c8e40 8183bded-ef37-4bdc-ab17-a4c4b136401d
> 8b161c72-0fc2-4175-ac9f-9a0c7bf9b387 38fd3a6d-498a-4145-b77e-24e33c31edaf
> 70f2729d-7249-4397-b04f-22694fec391e c65a3e39-009d-4e8a-b16e-1ca0e9de7cef
> 1dd2b274-75bd-4204-80b9-0103fefd0227 0bab6264-40ff-4f05-8d11-0cc9ee3132c0
> 0e7e677b-3f24-408b-ac1e-ddeee54023b8 8fd57ca5-2128-45a1-b7c4-ec611bfdefcd
> 01bf788e-cf98-44e8-abaa-c9e74e782bde 9d27f963-b45b-4991-baa8-2ed2db41857d
> 207da47a-b3b2-4583-b654-61fc551f4eca a10d8b77-8f27-48ca-a3b5-e5cfcbf8a4c0
> 352e77d5-3071-4502-bb0e-a6d0df8d1cac 18697755-6e2b-4907-ae8b-11ee0ff2f057
> d779ca40-0760-4b27-b0b1-c35d985359c1 ca4469aa-7be7-4eaf-9bcd-aeff6441c7c0
> 30ce1107-8e95-4c16-b55a-5a7c4dea0696 ee44b107-07ae-45c9-88a4-0424da8a6bcb main
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22312) MapJoinCounterHook doesnot work for tez
[ https://issues.apache.org/jira/browse/HIVE-22312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16991807#comment-16991807 ] Pulkit Sharma commented on HIVE-22312: -- [~jcamachorodriguez] Please take a look. > MapJoinCounterHook doesnot work for tez > --- > > Key: HIVE-22312 > URL: https://issues.apache.org/jira/browse/HIVE-22312 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: All Versions >Reporter: Pulkit Sharma >Assignee: Pulkit Sharma >Priority: Major > Attachments: HIVE-22312.01.patch, HIVE-22312.02.patch, > HIVE-22312.03.patch > > > In [HIVE-1792|https://issues.apache.org/jira/browse/HIVE-1792], > MapJoinCounterHook hook was added to track joins that get converted to map > join. This hook gets List of Tasks from hookContext and check Tag associated > with each task. For mr, we create Conditional tasks in case of joins and add > tags for respective join conversions. This does not work in case of tez as we > only create TezTask(no Conditional Task is created) which can handle multiple > joins in contrast to one Conditional Task per join in mr. > The current approach will fail even if we add tag to TezTask as it can have > multiple joins of the same type which will require counter. > One possible solution for tez, is to parse query-plan after query completion > which we get from hookContext to get workGraph. Using workGraph, we can walk > through Operator Tree to find join conversions. > If this approach looks good, I can raise Pull Request > cc [~ashutoshc] [~jcamachorodriguez] [~pxiong] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22607) Session State Thread Names Are Way Too Long
[
https://issues.apache.org/jira/browse/HIVE-22607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-22607:
--
Status: Patch Available (was: Open)
> Session State Thread Names Are Way Too Long
> ---
>
> Key: HIVE-22607
> URL: https://issues.apache.org/jira/browse/HIVE-22607
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-22607.1.patch
>
>
> {code:none}
> 2019-12-08T07:15:34,107 DEBUG [bc661ab9-b44a-4ee5-8ce6-b8b7ae5a0e07
> d480b94c-8773-4817-85fc-faa177308660 73ecdf15-06fd-4358-a249-22623c98d234
> 37e3c8de-9c77-4939-94d0-5aa45501b545 7b152899-d8da-493f-a329-a16f171bb1a3
> 604abf47-22ed-480c-a460-f6d44ac740ec 91fee8a8-aee4-4eee-993f-ad2927784d57
> c496e18f-a681-4db8-b254-6911793e7fb2 c6069d3c-03fb-4e1d-b0be-5dd13882b086
> 3c360edd-9c0f-486b-99f9-dd9f8aac5fcd c2be6f69-3ef5-44e4-b8d5-16b835b0ae2f
> 3bd2c6d2-9cce-4497-aa07-4672b95e6f76 ccb64f47-b1d7-477b-81ed-f8baabae4ee4
> 8b244038-d7a3-4e11-ab2f-b42d117c8e40 8183bded-ef37-4bdc-ab17-a4c4b136401d
> 8b161c72-0fc2-4175-ac9f-9a0c7bf9b387 38fd3a6d-498a-4145-b77e-24e33c31edaf
> 70f2729d-7249-4397-b04f-22694fec391e c65a3e39-009d-4e8a-b16e-1ca0e9de7cef
> 1dd2b274-75bd-4204-80b9-0103fefd0227 0bab6264-40ff-4f05-8d11-0cc9ee3132c0
> 0e7e677b-3f24-408b-ac1e-ddeee54023b8 8fd57ca5-2128-45a1-b7c4-ec611bfdefcd
> 01bf788e-cf98-44e8-abaa-c9e74e782bde 9d27f963-b45b-4991-baa8-2ed2db41857d
> 207da47a-b3b2-4583-b654-61fc551f4eca a10d8b77-8f27-48ca-a3b5-e5cfcbf8a4c0
> 352e77d5-3071-4502-bb0e-a6d0df8d1cac 18697755-6e2b-4907-ae8b-11ee0ff2f057
> d779ca40-0760-4b27-b0b1-c35d985359c1 ca4469aa-7be7-4eaf-9bcd-aeff6441c7c0
> 30ce1107-8e95-4c16-b55a-5a7c4dea0696 ee44b107-07ae-45c9-88a4-0424da8a6bcb
> main] session.SessionState: Updating thread name to
> 749f4123-716a-4534-af7f-b426fdc3ccc9 bc661ab9-b44a-4ee5-8ce6-b8b7ae5a0e07
> d480b94c-8773-4817-85fc-faa177308660 73ecdf15-06fd-4358-a249-22623c98d234
> 37e3c8de-9c77-4939-94d0-5aa45501b545 7b152899-d8da-493f-a329-a16f171bb1a3
> 604abf47-22ed-480c-a460-f6d44ac740ec 91fee8a8-aee4-4eee-993f-ad2927784d57
> c496e18f-a681-4db8-b254-6911793e7fb2 c6069d3c-03fb-4e1d-b0be-5dd13882b086
> 3c360edd-9c0f-486b-99f9-dd9f8aac5fcd c2be6f69-3ef5-44e4-b8d5-16b835b0ae2f
> 3bd2c6d2-9cce-4497-aa07-4672b95e6f76 ccb64f47-b1d7-477b-81ed-f8baabae4ee4
> 8b244038-d7a3-4e11-ab2f-b42d117c8e40 8183bded-ef37-4bdc-ab17-a4c4b136401d
> 8b161c72-0fc2-4175-ac9f-9a0c7bf9b387 38fd3a6d-498a-4145-b77e-24e33c31edaf
> 70f2729d-7249-4397-b04f-22694fec391e c65a3e39-009d-4e8a-b16e-1ca0e9de7cef
> 1dd2b274-75bd-4204-80b9-0103fefd0227 0bab6264-40ff-4f05-8d11-0cc9ee3132c0
> 0e7e677b-3f24-408b-ac1e-ddeee54023b8 8fd57ca5-2128-45a1-b7c4-ec611bfdefcd
> 01bf788e-cf98-44e8-abaa-c9e74e782bde 9d27f963-b45b-4991-baa8-2ed2db41857d
> 207da47a-b3b2-4583-b654-61fc551f4eca a10d8b77-8f27-48ca-a3b5-e5cfcbf8a4c0
> 352e77d5-3071-4502-bb0e-a6d0df8d1cac 18697755-6e2b-4907-ae8b-11ee0ff2f057
> d779ca40-0760-4b27-b0b1-c35d985359c1 ca4469aa-7be7-4eaf-9bcd-aeff6441c7c0
> 30ce1107-8e95-4c16-b55a-5a7c4dea0696 ee44b107-07ae-45c9-88a4-0424da8a6bcb main
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22607) Session State Thread Names Are Way Too Long
[
https://issues.apache.org/jira/browse/HIVE-22607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-22607:
--
Attachment: HIVE-22607.1.patch
> Session State Thread Names Are Way Too Long
> ---
>
> Key: HIVE-22607
> URL: https://issues.apache.org/jira/browse/HIVE-22607
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-22607.1.patch
>
>
> {code:none}
> 2019-12-08T07:15:34,107 DEBUG [bc661ab9-b44a-4ee5-8ce6-b8b7ae5a0e07
> d480b94c-8773-4817-85fc-faa177308660 73ecdf15-06fd-4358-a249-22623c98d234
> 37e3c8de-9c77-4939-94d0-5aa45501b545 7b152899-d8da-493f-a329-a16f171bb1a3
> 604abf47-22ed-480c-a460-f6d44ac740ec 91fee8a8-aee4-4eee-993f-ad2927784d57
> c496e18f-a681-4db8-b254-6911793e7fb2 c6069d3c-03fb-4e1d-b0be-5dd13882b086
> 3c360edd-9c0f-486b-99f9-dd9f8aac5fcd c2be6f69-3ef5-44e4-b8d5-16b835b0ae2f
> 3bd2c6d2-9cce-4497-aa07-4672b95e6f76 ccb64f47-b1d7-477b-81ed-f8baabae4ee4
> 8b244038-d7a3-4e11-ab2f-b42d117c8e40 8183bded-ef37-4bdc-ab17-a4c4b136401d
> 8b161c72-0fc2-4175-ac9f-9a0c7bf9b387 38fd3a6d-498a-4145-b77e-24e33c31edaf
> 70f2729d-7249-4397-b04f-22694fec391e c65a3e39-009d-4e8a-b16e-1ca0e9de7cef
> 1dd2b274-75bd-4204-80b9-0103fefd0227 0bab6264-40ff-4f05-8d11-0cc9ee3132c0
> 0e7e677b-3f24-408b-ac1e-ddeee54023b8 8fd57ca5-2128-45a1-b7c4-ec611bfdefcd
> 01bf788e-cf98-44e8-abaa-c9e74e782bde 9d27f963-b45b-4991-baa8-2ed2db41857d
> 207da47a-b3b2-4583-b654-61fc551f4eca a10d8b77-8f27-48ca-a3b5-e5cfcbf8a4c0
> 352e77d5-3071-4502-bb0e-a6d0df8d1cac 18697755-6e2b-4907-ae8b-11ee0ff2f057
> d779ca40-0760-4b27-b0b1-c35d985359c1 ca4469aa-7be7-4eaf-9bcd-aeff6441c7c0
> 30ce1107-8e95-4c16-b55a-5a7c4dea0696 ee44b107-07ae-45c9-88a4-0424da8a6bcb
> main] session.SessionState: Updating thread name to
> 749f4123-716a-4534-af7f-b426fdc3ccc9 bc661ab9-b44a-4ee5-8ce6-b8b7ae5a0e07
> d480b94c-8773-4817-85fc-faa177308660 73ecdf15-06fd-4358-a249-22623c98d234
> 37e3c8de-9c77-4939-94d0-5aa45501b545 7b152899-d8da-493f-a329-a16f171bb1a3
> 604abf47-22ed-480c-a460-f6d44ac740ec 91fee8a8-aee4-4eee-993f-ad2927784d57
> c496e18f-a681-4db8-b254-6911793e7fb2 c6069d3c-03fb-4e1d-b0be-5dd13882b086
> 3c360edd-9c0f-486b-99f9-dd9f8aac5fcd c2be6f69-3ef5-44e4-b8d5-16b835b0ae2f
> 3bd2c6d2-9cce-4497-aa07-4672b95e6f76 ccb64f47-b1d7-477b-81ed-f8baabae4ee4
> 8b244038-d7a3-4e11-ab2f-b42d117c8e40 8183bded-ef37-4bdc-ab17-a4c4b136401d
> 8b161c72-0fc2-4175-ac9f-9a0c7bf9b387 38fd3a6d-498a-4145-b77e-24e33c31edaf
> 70f2729d-7249-4397-b04f-22694fec391e c65a3e39-009d-4e8a-b16e-1ca0e9de7cef
> 1dd2b274-75bd-4204-80b9-0103fefd0227 0bab6264-40ff-4f05-8d11-0cc9ee3132c0
> 0e7e677b-3f24-408b-ac1e-ddeee54023b8 8fd57ca5-2128-45a1-b7c4-ec611bfdefcd
> 01bf788e-cf98-44e8-abaa-c9e74e782bde 9d27f963-b45b-4991-baa8-2ed2db41857d
> 207da47a-b3b2-4583-b654-61fc551f4eca a10d8b77-8f27-48ca-a3b5-e5cfcbf8a4c0
> 352e77d5-3071-4502-bb0e-a6d0df8d1cac 18697755-6e2b-4907-ae8b-11ee0ff2f057
> d779ca40-0760-4b27-b0b1-c35d985359c1 ca4469aa-7be7-4eaf-9bcd-aeff6441c7c0
> 30ce1107-8e95-4c16-b55a-5a7c4dea0696 ee44b107-07ae-45c9-88a4-0424da8a6bcb main
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22498) Schema tool enhancements to merge catalogs
[ https://issues.apache.org/jira/browse/HIVE-22498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16991792#comment-16991792 ] Sam An commented on HIVE-22498: --- I clicked the ship it on review board. +1 here, too > Schema tool enhancements to merge catalogs > -- > > Key: HIVE-22498 > URL: https://issues.apache.org/jira/browse/HIVE-22498 > Project: Hive > Issue Type: Sub-task >Affects Versions: 4.0.0 >Reporter: Naveen Gangam >Assignee: Naveen Gangam >Priority: Major > Attachments: HIVE-22498.2.patch, HIVE-22498.patch > > > Schema tool currently supports relocation of database from one catalog to > another, one at a time. While having to do this one at a time is painful, it > also lacks support for converting them to external tables during migration, > in lieu of the changes to the translation layer where a MANAGED table is > strictly ACID-only table. > Hence we also need to convert them to external tables during relocation. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22569) PartitionPruner use Collections Class
[
https://issues.apache.org/jira/browse/HIVE-22569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16991790#comment-16991790
]
Hive QA commented on HIVE-22569:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
49s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
28s{color} | {color:blue} ql in master has 1532 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 27m 11s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19835/dev-support/hive-personality.sh
|
| git revision | master / d7a193b |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19835/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> PartitionPruner use Collections Class
> -
>
> Key: HIVE-22569
> URL: https://issues.apache.org/jira/browse/HIVE-22569
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HIVE-22569.2.patch, HIVE-22569.2.patch,
> HIVE-22569.2.patch, HIVE.22569.1.patch
>
>
> Make class {{PrunedPartitionList.java}} more immutable. It kinda is already.
> Immutable classes are easier to reason about.
> Use {{Collections.emptyXXX}} in {{PartitionPruner}} to save an object
> instantiation at runtime.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22594) Reorder metastore thrift structures
[ https://issues.apache.org/jira/browse/HIVE-22594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Sherman updated HIVE-22594: Attachment: HIVE-22594.2.patch > Reorder metastore thrift structures > --- > > Key: HIVE-22594 > URL: https://issues.apache.org/jira/browse/HIVE-22594 > Project: Hive > Issue Type: Improvement > Components: Standalone Metastore, Thrift API >Affects Versions: 4.0.0 >Reporter: John Sherman >Assignee: John Sherman >Priority: Major > Attachments: HIVE-22594.1.patch, HIVE-22594.2.patch > > > Currently hive_metastore.thrift C++ output does not compile due to structures > like ScheduledQueryKey and ColumnStatistics being referenced before being > defined later in the file. We should reorder the structures in the file so > the C++ output is usable. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22508) KeyWrapperComparator throws exception
[
https://issues.apache.org/jira/browse/HIVE-22508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16991768#comment-16991768
]
Hive QA commented on HIVE-22508:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12988314/HIVE-22508.2.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17779 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1 (batchId=276)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19834/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19834/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19834/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12988314 - PreCommit-HIVE-Build
> KeyWrapperComparator throws exception
> -
>
> Key: HIVE-22508
> URL: https://issues.apache.org/jira/browse/HIVE-22508
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22508.1.patch, HIVE-22508.1.patch,
> HIVE-22508.1.patch, HIVE-22508.2.patch
>
>
> TopNKeyOperator.KeyWrapperComparator throws exception when a new key and a
> copied key should be compared.
> The current implementation uses the standard object inspectors for all
> KeyWrapper instances. However when comparing untouched KeyWrappers the key
> object inspector should be used which can be extracted form
> Operator.inputObjectInspectors during initialization of the key's
> ExprNodeEvaluator.
> This can cause a ClassCastException when the comparator is used collections
> like TreeSet.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-20420) Provide a fallback authorizer when no other authorizer is in use
[ https://issues.apache.org/jira/browse/HIVE-20420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16991767#comment-16991767 ] Kalyan Kalvagadda commented on HIVE-20420: -- [~thejas] [~daijy] Can you provide more information on this. What is the issue and what has been done to address it. This change is calling for a change in Sentry. I would like to understand it. > Provide a fallback authorizer when no other authorizer is in use > > > Key: HIVE-20420 > URL: https://issues.apache.org/jira/browse/HIVE-20420 > Project: Hive > Issue Type: New Feature >Reporter: Jianyong Dai >Assignee: Jianyong Dai >Priority: Major > Fix For: 4.0.0, 3.1.1, 2.3.4 > > Attachments: HIVE-20420.1.patch, HIVE-20420.2.patch, > HIVE-20420.3.patch, HIVE-20420.4.patch, HIVE-20420.5.patch, > HIVE-20420.6.patch, HIVE-20420.7.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-20972) Enable TestMiniLlapLocalCliDriver cbo_limit and cbo_rp_limit
[ https://issues.apache.org/jira/browse/HIVE-20972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-20972: Attachment: HIVE-20972.03.patch > Enable TestMiniLlapLocalCliDriver cbo_limit and cbo_rp_limit > > > Key: HIVE-20972 > URL: https://issues.apache.org/jira/browse/HIVE-20972 > Project: Hive > Issue Type: Test >Reporter: Vihang Karajgaonkar >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-20972.01.patch, HIVE-20972.02.patch, > HIVE-20972.02.patch, HIVE-20972.03.patch, HIVE-20972.03.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22608) Reduce the number of public methods in Driver
[ https://issues.apache.org/jira/browse/HIVE-22608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22608: -- Status: Patch Available (was: Open) > Reduce the number of public methods in Driver > - > > Key: HIVE-22608 > URL: https://issues.apache.org/jira/browse/HIVE-22608 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22608.01.patch > > > There are 33 public methods in Driver, some of them either don't belong > there, or should not be public. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22568) Process compaction candidates in parallel by the Initiator
[ https://issues.apache.org/jira/browse/HIVE-22568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denys Kuzmenko updated HIVE-22568: -- Attachment: HIVE-22568.5.patch > Process compaction candidates in parallel by the Initiator > -- > > Key: HIVE-22568 > URL: https://issues.apache.org/jira/browse/HIVE-22568 > Project: Hive > Issue Type: Improvement >Reporter: Denys Kuzmenko >Assignee: Denys Kuzmenko >Priority: Major > Attachments: HIVE-22568.1.patch, HIVE-22568.2.patch, > HIVE-22568.3.patch, HIVE-22568.4.patch, HIVE-22568.5.patch > > > `checkForCompaction` includes many file metadata checks and may be expensive. > Therefore, make sense using a thread pool here and running > `checkForCompactions` in parallel. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22608) Reduce the number of public methods in Driver
[ https://issues.apache.org/jira/browse/HIVE-22608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22608: -- Attachment: HIVE-22608.01.patch > Reduce the number of public methods in Driver > - > > Key: HIVE-22608 > URL: https://issues.apache.org/jira/browse/HIVE-22608 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22608.01.patch > > > There are 33 public methods in Driver, some of them either don't belong > there, or should not be public. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22327) Repl: Ignore read-only transactions in notification log
[ https://issues.apache.org/jira/browse/HIVE-22327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denys Kuzmenko updated HIVE-22327: -- Attachment: HIVE-22327.19.patch > Repl: Ignore read-only transactions in notification log > --- > > Key: HIVE-22327 > URL: https://issues.apache.org/jira/browse/HIVE-22327 > Project: Hive > Issue Type: Improvement > Components: repl >Reporter: Gopal Vijayaraghavan >Assignee: Denys Kuzmenko >Priority: Major > Attachments: HIVE-22327.1.patch, HIVE-22327.10.patch, > HIVE-22327.11.patch, HIVE-22327.12.patch, HIVE-22327.13.patch, > HIVE-22327.14.patch, HIVE-22327.15.patch, HIVE-22327.16.patch, > HIVE-22327.17.patch, HIVE-22327.18.patch, HIVE-22327.19.patch, > HIVE-22327.2.patch, HIVE-22327.3.patch, HIVE-22327.4.patch, > HIVE-22327.5.patch, HIVE-22327.6.patch, HIVE-22327.7.patch, > HIVE-22327.8.patch, HIVE-22327.9.patch > > > Read txns need not be replicated. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22606) AvroSerde logs avro.schema.literal under INFO level
[
https://issues.apache.org/jira/browse/HIVE-22606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16991727#comment-16991727
]
Denys Kuzmenko commented on HIVE-22606:
---
Set log level for AvroSerde to DEBUG.
{code}
java.lang.Thread.State: BLOCKED (on object monitor)
at java.io.PrintStream.write(PrintStream.java:478)
{code}
> AvroSerde logs avro.schema.literal under INFO level
> ---
>
> Key: HIVE-22606
> URL: https://issues.apache.org/jira/browse/HIVE-22606
> Project: Hive
> Issue Type: Bug
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-22606.1.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-22608) Reduce the number of public methods in Driver
[ https://issues.apache.org/jira/browse/HIVE-22608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely reassigned HIVE-22608: - > Reduce the number of public methods in Driver > - > > Key: HIVE-22608 > URL: https://issues.apache.org/jira/browse/HIVE-22608 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 4.0.0 > > > There are 33 public methods in Driver, some of them either don't belong > there, or should not be public. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22596) RawStore used by Initiator is not thread-safe
[ https://issues.apache.org/jira/browse/HIVE-22596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denys Kuzmenko updated HIVE-22596: -- Attachment: HIVE-22596.2.patch > RawStore used by Initiator is not thread-safe > - > > Key: HIVE-22596 > URL: https://issues.apache.org/jira/browse/HIVE-22596 > Project: Hive > Issue Type: Improvement >Reporter: Denys Kuzmenko >Assignee: Denys Kuzmenko >Priority: Major > Attachments: HIVE-22596.1.patch, HIVE-22596.2.patch > > > RawStore used by Initiator is not thread-safe. To avoid synchronization, we > can replace it with ThreadLocal variable. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22508) KeyWrapperComparator throws exception
[
https://issues.apache.org/jira/browse/HIVE-22508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16991728#comment-16991728
]
Hive QA commented on HIVE-22508:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
22s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
41s{color} | {color:blue} serde in master has 198 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
5s{color} | {color:blue} ql in master has 1532 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
17s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
18s{color} | {color:red} serde: The patch generated 1 new + 261 unchanged - 0
fixed = 262 total (was 261) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} ql: The patch generated 0 new + 11 unchanged - 1
fixed = 11 total (was 12) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
48s{color} | {color:green} serde in the patch passed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
8s{color} | {color:green} ql generated 0 new + 1531 unchanged - 1 fixed = 1531
total (was 1532) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 29m 1s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19834/dev-support/hive-personality.sh
|
| git revision | master / d7a193b |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19834/yetus/diff-checkstyle-serde.txt
|
| modules | C: serde ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19834/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> KeyWrapperComparator throws exception
> -
>
> Key: HIVE-22508
> URL: https://issues.apache.org/jira/browse/HIVE-22508
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22508.1.patch, HIVE-22508.1.patch,
> HIVE-22508.1.patch, HIVE-22508.2.patch
>
>
> TopNKeyOperator.KeyWrapperComparator throws exception when a new key and a
> copied key should be compared.
> The current implementation uses the standard object inspectors for all
[jira] [Comment Edited] (HIVE-22570) Review of ExprNodeDesc.java
[ https://issues.apache.org/jira/browse/HIVE-22570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16991720#comment-16991720 ] David Mollitor edited comment on HIVE-22570 at 12/9/19 3:55 PM: [~pvary] Thanks for the review. This change would be for Hive 4.0 so should be OK to make this kind of change to a public interface. Returning a null sucks. https://github.com/google/guava/wiki/UsingAndAvoidingNullExplained Does a 'null' value mean that there are no children? If so, what does it mean if the method returns a list with no elements in it? How are those two things different? It's very ambiguous and hard to reason about. Much better to say "if I have no children, just say so... return a list with no elements". was (Author: belugabehr): [~pvary] Thanks for the review. This change would be for Hive 4.0 so should be OK to make this kind of change. Returning a null sucks. https://github.com/google/guava/wiki/UsingAndAvoidingNullExplained Does a 'null' value mean that there are no children? If so, what does it mean if the method returns a list with no elements in it? How are those two things different? It's very ambiguous and hard to reason about. Much better to say "if I have no children, just say so... return a list with no elements". > Review of ExprNodeDesc.java > --- > > Key: HIVE-22570 > URL: https://issues.apache.org/jira/browse/HIVE-22570 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Attachments: HIVE-22570.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22570) Review of ExprNodeDesc.java
[ https://issues.apache.org/jira/browse/HIVE-22570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16991720#comment-16991720 ] David Mollitor commented on HIVE-22570: --- [~pvary] Thanks for the review. This change would be for Hive 4.0 so should be OK to make this kind of change. Returning a null sucks. https://github.com/google/guava/wiki/UsingAndAvoidingNullExplained Does a 'null' value mean that there are no children? If so, what does it mean if the method returns a list with no elements in it? How are those two things different? It's very ambiguous and hard to reason about. Much better to say "if I have no children, just say so... return a list with no elements". > Review of ExprNodeDesc.java > --- > > Key: HIVE-22570 > URL: https://issues.apache.org/jira/browse/HIVE-22570 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Attachments: HIVE-22570.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22498) Schema tool enhancements to merge catalogs
[ https://issues.apache.org/jira/browse/HIVE-22498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16991719#comment-16991719 ] Naveen Gangam commented on HIVE-22498: -- [~ychena] [~samuelan] Could you please review the new version of this patch? Thank you > Schema tool enhancements to merge catalogs > -- > > Key: HIVE-22498 > URL: https://issues.apache.org/jira/browse/HIVE-22498 > Project: Hive > Issue Type: Sub-task >Affects Versions: 4.0.0 >Reporter: Naveen Gangam >Assignee: Naveen Gangam >Priority: Major > Attachments: HIVE-22498.2.patch, HIVE-22498.patch > > > Schema tool currently supports relocation of database from one catalog to > another, one at a time. While having to do this one at a time is painful, it > also lacks support for converting them to external tables during migration, > in lieu of the changes to the translation layer where a MANAGED table is > strictly ACID-only table. > Hence we also need to convert them to external tables during relocation. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-20150) TopNKey pushdown
[ https://issues.apache.org/jira/browse/HIVE-20150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-20150: -- Attachment: HIVE-20150.24.patch > TopNKey pushdown > > > Key: HIVE-20150 > URL: https://issues.apache.org/jira/browse/HIVE-20150 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Affects Versions: 4.0.0 >Reporter: Teddy Choi >Assignee: Krisztian Kasa >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20150.1.patch, HIVE-20150.10.patch, > HIVE-20150.11.patch, HIVE-20150.11.patch, HIVE-20150.14.patch, > HIVE-20150.15.patch, HIVE-20150.16.patch, HIVE-20150.17.patch, > HIVE-20150.17.patch, HIVE-20150.18.patch, HIVE-20150.18.patch, > HIVE-20150.19.patch, HIVE-20150.2.patch, HIVE-20150.20.patch, > HIVE-20150.21.patch, HIVE-20150.22.patch, HIVE-20150.23.patch, > HIVE-20150.24.patch, HIVE-20150.4.patch, HIVE-20150.5.patch, > HIVE-20150.6.patch, HIVE-20150.7.patch, HIVE-20150.8.patch, HIVE-20150.9.patch > > > TopNKey operator is implemented in HIVE-17896, but it needs more work in > pushdown implementation. So this issue covers TopNKey pushdown implementation > with proper tests. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-20150) TopNKey pushdown
[ https://issues.apache.org/jira/browse/HIVE-20150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-20150: -- Status: Patch Available (was: Open) > TopNKey pushdown > > > Key: HIVE-20150 > URL: https://issues.apache.org/jira/browse/HIVE-20150 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Affects Versions: 4.0.0 >Reporter: Teddy Choi >Assignee: Krisztian Kasa >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20150.1.patch, HIVE-20150.10.patch, > HIVE-20150.11.patch, HIVE-20150.11.patch, HIVE-20150.14.patch, > HIVE-20150.15.patch, HIVE-20150.16.patch, HIVE-20150.17.patch, > HIVE-20150.17.patch, HIVE-20150.18.patch, HIVE-20150.18.patch, > HIVE-20150.19.patch, HIVE-20150.2.patch, HIVE-20150.20.patch, > HIVE-20150.21.patch, HIVE-20150.22.patch, HIVE-20150.23.patch, > HIVE-20150.24.patch, HIVE-20150.4.patch, HIVE-20150.5.patch, > HIVE-20150.6.patch, HIVE-20150.7.patch, HIVE-20150.8.patch, HIVE-20150.9.patch > > > TopNKey operator is implemented in HIVE-17896, but it needs more work in > pushdown implementation. So this issue covers TopNKey pushdown implementation > with proper tests. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-20150) TopNKey pushdown
[ https://issues.apache.org/jira/browse/HIVE-20150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-20150: -- Status: Open (was: Patch Available) > TopNKey pushdown > > > Key: HIVE-20150 > URL: https://issues.apache.org/jira/browse/HIVE-20150 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Affects Versions: 4.0.0 >Reporter: Teddy Choi >Assignee: Krisztian Kasa >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20150.1.patch, HIVE-20150.10.patch, > HIVE-20150.11.patch, HIVE-20150.11.patch, HIVE-20150.14.patch, > HIVE-20150.15.patch, HIVE-20150.16.patch, HIVE-20150.17.patch, > HIVE-20150.17.patch, HIVE-20150.18.patch, HIVE-20150.18.patch, > HIVE-20150.19.patch, HIVE-20150.2.patch, HIVE-20150.20.patch, > HIVE-20150.21.patch, HIVE-20150.22.patch, HIVE-20150.23.patch, > HIVE-20150.4.patch, HIVE-20150.5.patch, HIVE-20150.6.patch, > HIVE-20150.7.patch, HIVE-20150.8.patch, HIVE-20150.9.patch > > > TopNKey operator is implemented in HIVE-17896, but it needs more work in > pushdown implementation. So this issue covers TopNKey pushdown implementation > with proper tests. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22570) Review of ExprNodeDesc.java
[
https://issues.apache.org/jira/browse/HIVE-22570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16991710#comment-16991710
]
Peter Vary commented on HIVE-22570:
---
Hi David,
I am concerned about this kind of changes:
{code}
public List getCols() {
return Collections.emptyList();
}
{code}
Changing the return value from {{null}} to empty list in a public method can
cause different behavior when used...
> Review of ExprNodeDesc.java
> ---
>
> Key: HIVE-22570
> URL: https://issues.apache.org/jira/browse/HIVE-22570
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HIVE-22570.1.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22553) Expose locks and transactions in sys db
[ https://issues.apache.org/jira/browse/HIVE-22553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Chovan updated HIVE-22553: - Attachment: HIVE-22553.7.patch > Expose locks and transactions in sys db > --- > > Key: HIVE-22553 > URL: https://issues.apache.org/jira/browse/HIVE-22553 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0 >Reporter: Zoltan Chovan >Assignee: Zoltan Chovan >Priority: Major > Attachments: HIVE-22553.2.patch, HIVE-22553.3.patch, > HIVE-22553.3.patch, HIVE-22553.4.patch, HIVE-22553.5.patch, > HIVE-22553.6.patch, HIVE-22553.7.patch, HIVE-22553.patch > > > Create new sysdb tables/views to access lock and transaction data. > This allows to provide admins with live data about ongoing locks and > transacions. Due to this being in the sys db access to this information can > be restricted to select privileged users. > Information about locks and compactions can be joined and accessed at the > same time. > Compaction related transactions would also be visible. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22606) AvroSerde logs avro.schema.literal under INFO level
[ https://issues.apache.org/jira/browse/HIVE-22606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denys Kuzmenko updated HIVE-22606: -- Attachment: HIVE-22606.1.patch > AvroSerde logs avro.schema.literal under INFO level > --- > > Key: HIVE-22606 > URL: https://issues.apache.org/jira/browse/HIVE-22606 > Project: Hive > Issue Type: Bug >Reporter: Denys Kuzmenko >Assignee: Denys Kuzmenko >Priority: Major > Attachments: HIVE-22606.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22606) AvroSerde logs avro.schema.literal under INFO level
[ https://issues.apache.org/jira/browse/HIVE-22606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denys Kuzmenko updated HIVE-22606: -- Status: Patch Available (was: Open) > AvroSerde logs avro.schema.literal under INFO level > --- > > Key: HIVE-22606 > URL: https://issues.apache.org/jira/browse/HIVE-22606 > Project: Hive > Issue Type: Bug >Reporter: Denys Kuzmenko >Assignee: Denys Kuzmenko >Priority: Major > Attachments: HIVE-22606.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-20150) TopNKey pushdown
[ https://issues.apache.org/jira/browse/HIVE-20150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-20150: -- Attachment: HIVE-20150.23.patch > TopNKey pushdown > > > Key: HIVE-20150 > URL: https://issues.apache.org/jira/browse/HIVE-20150 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Affects Versions: 4.0.0 >Reporter: Teddy Choi >Assignee: Krisztian Kasa >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20150.1.patch, HIVE-20150.10.patch, > HIVE-20150.11.patch, HIVE-20150.11.patch, HIVE-20150.14.patch, > HIVE-20150.15.patch, HIVE-20150.16.patch, HIVE-20150.17.patch, > HIVE-20150.17.patch, HIVE-20150.18.patch, HIVE-20150.18.patch, > HIVE-20150.19.patch, HIVE-20150.2.patch, HIVE-20150.20.patch, > HIVE-20150.21.patch, HIVE-20150.22.patch, HIVE-20150.23.patch, > HIVE-20150.4.patch, HIVE-20150.5.patch, HIVE-20150.6.patch, > HIVE-20150.7.patch, HIVE-20150.8.patch, HIVE-20150.9.patch > > > TopNKey operator is implemented in HIVE-17896, but it needs more work in > pushdown implementation. So this issue covers TopNKey pushdown implementation > with proper tests. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-20150) TopNKey pushdown
[ https://issues.apache.org/jira/browse/HIVE-20150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-20150: -- Status: Open (was: Patch Available) > TopNKey pushdown > > > Key: HIVE-20150 > URL: https://issues.apache.org/jira/browse/HIVE-20150 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Affects Versions: 4.0.0 >Reporter: Teddy Choi >Assignee: Krisztian Kasa >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20150.1.patch, HIVE-20150.10.patch, > HIVE-20150.11.patch, HIVE-20150.11.patch, HIVE-20150.14.patch, > HIVE-20150.15.patch, HIVE-20150.16.patch, HIVE-20150.17.patch, > HIVE-20150.17.patch, HIVE-20150.18.patch, HIVE-20150.18.patch, > HIVE-20150.19.patch, HIVE-20150.2.patch, HIVE-20150.20.patch, > HIVE-20150.21.patch, HIVE-20150.22.patch, HIVE-20150.23.patch, > HIVE-20150.4.patch, HIVE-20150.5.patch, HIVE-20150.6.patch, > HIVE-20150.7.patch, HIVE-20150.8.patch, HIVE-20150.9.patch > > > TopNKey operator is implemented in HIVE-17896, but it needs more work in > pushdown implementation. So this issue covers TopNKey pushdown implementation > with proper tests. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-20150) TopNKey pushdown
[ https://issues.apache.org/jira/browse/HIVE-20150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-20150: -- Status: Patch Available (was: Open) > TopNKey pushdown > > > Key: HIVE-20150 > URL: https://issues.apache.org/jira/browse/HIVE-20150 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Affects Versions: 4.0.0 >Reporter: Teddy Choi >Assignee: Krisztian Kasa >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20150.1.patch, HIVE-20150.10.patch, > HIVE-20150.11.patch, HIVE-20150.11.patch, HIVE-20150.14.patch, > HIVE-20150.15.patch, HIVE-20150.16.patch, HIVE-20150.17.patch, > HIVE-20150.17.patch, HIVE-20150.18.patch, HIVE-20150.18.patch, > HIVE-20150.19.patch, HIVE-20150.2.patch, HIVE-20150.20.patch, > HIVE-20150.21.patch, HIVE-20150.22.patch, HIVE-20150.23.patch, > HIVE-20150.4.patch, HIVE-20150.5.patch, HIVE-20150.6.patch, > HIVE-20150.7.patch, HIVE-20150.8.patch, HIVE-20150.9.patch > > > TopNKey operator is implemented in HIVE-17896, but it needs more work in > pushdown implementation. So this issue covers TopNKey pushdown implementation > with proper tests. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-22606) AvroSerde logs avro.schema.literal under INFO level
[ https://issues.apache.org/jira/browse/HIVE-22606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denys Kuzmenko reassigned HIVE-22606: - Assignee: Denys Kuzmenko > AvroSerde logs avro.schema.literal under INFO level > --- > > Key: HIVE-22606 > URL: https://issues.apache.org/jira/browse/HIVE-22606 > Project: Hive > Issue Type: Bug >Reporter: Denys Kuzmenko >Assignee: Denys Kuzmenko >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-17446) TestMiniLlapLocalCliDriver.testCliDriver[columnstats_part_coltype]
[
https://issues.apache.org/jira/browse/HIVE-17446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16991703#comment-16991703
]
László Bodor commented on HIVE-17446:
-
haven't found in latest precommit run + passes locally
{code}
mvn test -Dtest.groups= -Dpackaging.minimizeJar=false -T 1C -DskipShade
-Dremoteresources.skip=true -Dmaven.javadoc.skip=true -DfailIfNoTests=true
-Dtest.output.overwrite=true -Pitests,hadoop-2 -Denforcer.skip=true -pl
itests/qtest -pl itests/util -Dtest=TestMiniLlapLocalCliDriver
-Dqfile=columnstats_part_coltype.q
{code}
> TestMiniLlapLocalCliDriver.testCliDriver[columnstats_part_coltype]
> --
>
> Key: HIVE-17446
> URL: https://issues.apache.org/jira/browse/HIVE-17446
> Project: Hive
> Issue Type: Sub-task
>Reporter: Vihang Karajgaonkar
>Priority: Major
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22605) NPE in LlapLoadGeneratorService During Tests
[
https://issues.apache.org/jira/browse/HIVE-22605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-22605:
--
Attachment: HIVE-22605.1.patch
> NPE in LlapLoadGeneratorService During Tests
>
>
> Key: HIVE-22605
> URL: https://issues.apache.org/jira/browse/HIVE-22605
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HIVE-22605.1.patch
>
>
> {code:none}
> java.lang.NullPointerException: null
> at
> org.apache.hadoop.hive.llap.daemon.impl.LlapLoadGeneratorService.serviceStop(LlapLoadGeneratorService.java:103)
> ~[classes/:?]
> at
> org.apache.hadoop.service.AbstractService.stop(AbstractService.java:220)
> ~[hadoop-common-3.1.0.jar:?]
> at
> org.apache.hadoop.service.ServiceOperations.stop(ServiceOperations.java:54)
> ~[hadoop-common-3.1.0.jar:?]
> at
> org.apache.hadoop.service.ServiceOperations.stopQuietly(ServiceOperations.java:102)
> [hadoop-common-3.1.0.jar:?]
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:172)
> [hadoop-common-3.1.0.jar:?]
> at
> org.apache.hadoop.hive.llap.daemon.impl.TestLlapLoadGeneratorService.testLoadGeneratorFails(TestLlapLoadGeneratorService.java:70)
> [test-classes/:?]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[?:1.8.0_102]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22605) NPE in LlapLoadGeneratorService During Tests
[
https://issues.apache.org/jira/browse/HIVE-22605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-22605:
--
Status: Patch Available (was: Open)
> NPE in LlapLoadGeneratorService During Tests
>
>
> Key: HIVE-22605
> URL: https://issues.apache.org/jira/browse/HIVE-22605
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HIVE-22605.1.patch
>
>
> {code:none}
> java.lang.NullPointerException: null
> at
> org.apache.hadoop.hive.llap.daemon.impl.LlapLoadGeneratorService.serviceStop(LlapLoadGeneratorService.java:103)
> ~[classes/:?]
> at
> org.apache.hadoop.service.AbstractService.stop(AbstractService.java:220)
> ~[hadoop-common-3.1.0.jar:?]
> at
> org.apache.hadoop.service.ServiceOperations.stop(ServiceOperations.java:54)
> ~[hadoop-common-3.1.0.jar:?]
> at
> org.apache.hadoop.service.ServiceOperations.stopQuietly(ServiceOperations.java:102)
> [hadoop-common-3.1.0.jar:?]
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:172)
> [hadoop-common-3.1.0.jar:?]
> at
> org.apache.hadoop.hive.llap.daemon.impl.TestLlapLoadGeneratorService.testLoadGeneratorFails(TestLlapLoadGeneratorService.java:70)
> [test-classes/:?]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[?:1.8.0_102]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HIVE-17446) TestMiniLlapLocalCliDriver.testCliDriver[columnstats_part_coltype]
[ https://issues.apache.org/jira/browse/HIVE-17446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor resolved HIVE-17446. - Resolution: Cannot Reproduce > TestMiniLlapLocalCliDriver.testCliDriver[columnstats_part_coltype] > -- > > Key: HIVE-17446 > URL: https://issues.apache.org/jira/browse/HIVE-17446 > Project: Hive > Issue Type: Sub-task >Reporter: Vihang Karajgaonkar >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-22605) NPE in LlapLoadGeneratorService During Tests
[
https://issues.apache.org/jira/browse/HIVE-22605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor reassigned HIVE-22605:
-
> NPE in LlapLoadGeneratorService During Tests
>
>
> Key: HIVE-22605
> URL: https://issues.apache.org/jira/browse/HIVE-22605
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
>
> {code:none}
> java.lang.NullPointerException: null
> at
> org.apache.hadoop.hive.llap.daemon.impl.LlapLoadGeneratorService.serviceStop(LlapLoadGeneratorService.java:103)
> ~[classes/:?]
> at
> org.apache.hadoop.service.AbstractService.stop(AbstractService.java:220)
> ~[hadoop-common-3.1.0.jar:?]
> at
> org.apache.hadoop.service.ServiceOperations.stop(ServiceOperations.java:54)
> ~[hadoop-common-3.1.0.jar:?]
> at
> org.apache.hadoop.service.ServiceOperations.stopQuietly(ServiceOperations.java:102)
> [hadoop-common-3.1.0.jar:?]
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:172)
> [hadoop-common-3.1.0.jar:?]
> at
> org.apache.hadoop.hive.llap.daemon.impl.TestLlapLoadGeneratorService.testLoadGeneratorFails(TestLlapLoadGeneratorService.java:70)
> [test-classes/:?]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[?:1.8.0_102]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-17440) TestSparkCliDriver.testCliDriver[vectorized_ptf]
[ https://issues.apache.org/jira/browse/HIVE-17440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16991701#comment-16991701 ] László Bodor commented on HIVE-17440: - with the patch, vectorized_ptf.q.out will be inline with the corresponding clientpositive q.out > TestSparkCliDriver.testCliDriver[vectorized_ptf] > > > Key: HIVE-17440 > URL: https://issues.apache.org/jira/browse/HIVE-17440 > Project: Hive > Issue Type: Sub-task >Reporter: Vihang Karajgaonkar >Assignee: László Bodor >Priority: Major > Attachments: HIVE-17440.01.branch-2.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-17440) TestSparkCliDriver.testCliDriver[vectorized_ptf]
[ https://issues.apache.org/jira/browse/HIVE-17440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor updated HIVE-17440: Attachment: HIVE-17440.01.branch-2.patch > TestSparkCliDriver.testCliDriver[vectorized_ptf] > > > Key: HIVE-17440 > URL: https://issues.apache.org/jira/browse/HIVE-17440 > Project: Hive > Issue Type: Sub-task >Reporter: Vihang Karajgaonkar >Assignee: László Bodor >Priority: Major > Attachments: HIVE-17440.01.branch-2.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-17440) TestSparkCliDriver.testCliDriver[vectorized_ptf]
[ https://issues.apache.org/jira/browse/HIVE-17440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor reassigned HIVE-17440: --- Assignee: László Bodor > TestSparkCliDriver.testCliDriver[vectorized_ptf] > > > Key: HIVE-17440 > URL: https://issues.apache.org/jira/browse/HIVE-17440 > Project: Hive > Issue Type: Sub-task >Reporter: Vihang Karajgaonkar >Assignee: László Bodor >Priority: Major > Attachments: HIVE-17440.01.branch-2.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-17440) TestSparkCliDriver.testCliDriver[vectorized_ptf]
[ https://issues.apache.org/jira/browse/HIVE-17440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor updated HIVE-17440: Status: Patch Available (was: Open) > TestSparkCliDriver.testCliDriver[vectorized_ptf] > > > Key: HIVE-17440 > URL: https://issues.apache.org/jira/browse/HIVE-17440 > Project: Hive > Issue Type: Sub-task >Reporter: Vihang Karajgaonkar >Assignee: László Bodor >Priority: Major > Attachments: HIVE-17440.01.branch-2.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22561) Data loss on map join for bucketed, partitioned table
[
https://issues.apache.org/jira/browse/HIVE-22561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16991697#comment-16991697
]
Hive QA commented on HIVE-22561:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12988312/HIVE-22561.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19833/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19833/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19833/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2019-12-09 15:20:31.922
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-19833/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2019-12-09 15:20:31.925
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at d7a193b HIVE-22598: Fix
TestCompactor.testDisableCompactionDuringReplLoad flakyness (Peter Vary
reviewed by Zoltan Haindrich)
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at d7a193b HIVE-22598: Fix
TestCompactor.testDisableCompactionDuringReplLoad flakyness (Peter Vary
reviewed by Zoltan Haindrich)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2019-12-09 15:20:33.143
+ rm -rf ../yetus_PreCommit-HIVE-Build-19833
+ mkdir ../yetus_PreCommit-HIVE-Build-19833
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-19833
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-19833/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error:
a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/metainfo/annotation/OpTraitsRulesProcFactory.java:
does not exist in index
error: a/ql/src/test/queries/clientpositive/bucket_map_join_tez2.q: does not
exist in index
error: a/ql/src/test/results/clientpositive/llap/bucket_map_join_tez2.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/llap/limit_pushdown.q.out: does not
exist in index
error:
a/ql/src/test/results/clientpositive/llap/offset_limit_ppd_optimizer.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/llap/tez_smb_main.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/spark/bucket_map_join_tez2.q.out:
does not exist in index
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/optimizer/metainfo/annotation/OpTraitsRulesProcFactory.java:110
Falling back to three-way merge...
Applied patch to
'ql/src/java/org/apache/hadoop/hive/ql/optimizer/metainfo/annotation/OpTraitsRulesProcFactory.java'
cleanly.
error: patch failed:
ql/src/test/queries/clientpositive/bucket_map_join_tez2.q:138
Falling back to three-way merge...
Applied patch to 'ql/src/test/queries/clientpositive/bucket_map_join_tez2.q'
with conflicts.
error: patch failed:
ql/src/test/results/clientpositive/llap/limit_pushdown.q.out:923
Falling back to three-way merge...
Applied patch to 'ql/src/test/results/clientpositive/llap/limit_pushdown.q.out'
with conflicts.
error: patch failed:
ql/src/test/results/clientpositive/llap/offset_limit_ppd_optimizer.q.out:1317
Falling back to three-way merge...
Applied patch to
'ql/src/test/results/clientpositive/llap/offset_limit_ppd_optimizer.q.out' with
conflicts.
error: patch failed:
ql/src/test/results/clientpositive/llap/tez_smb_main.q.out:592
Falling back to three-way merge...
Applied patch to 'ql/src/test/results/clientpositive/llap/tez_smb_main.q.out'
with conflicts.
[jira] [Assigned] (HIVE-17441) TestSparkCliDriver.testCliDriver[explaindenpendencydiffengs]
[ https://issues.apache.org/jira/browse/HIVE-17441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor reassigned HIVE-17441: --- Assignee: László Bodor > TestSparkCliDriver.testCliDriver[explaindenpendencydiffengs] > > > Key: HIVE-17441 > URL: https://issues.apache.org/jira/browse/HIVE-17441 > Project: Hive > Issue Type: Sub-task >Reporter: Vihang Karajgaonkar >Assignee: László Bodor >Priority: Major > Attachments: HIVE-17441.01.branch-2.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-17441) TestSparkCliDriver.testCliDriver[explaindenpendencydiffengs]
[ https://issues.apache.org/jira/browse/HIVE-17441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor updated HIVE-17441: Attachment: HIVE-17441.01.branch-2.patch > TestSparkCliDriver.testCliDriver[explaindenpendencydiffengs] > > > Key: HIVE-17441 > URL: https://issues.apache.org/jira/browse/HIVE-17441 > Project: Hive > Issue Type: Sub-task >Reporter: Vihang Karajgaonkar >Assignee: László Bodor >Priority: Major > Attachments: HIVE-17441.01.branch-2.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-17441) TestSparkCliDriver.testCliDriver[explaindenpendencydiffengs]
[ https://issues.apache.org/jira/browse/HIVE-17441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor updated HIVE-17441: Status: Patch Available (was: Open) > TestSparkCliDriver.testCliDriver[explaindenpendencydiffengs] > > > Key: HIVE-17441 > URL: https://issues.apache.org/jira/browse/HIVE-17441 > Project: Hive > Issue Type: Sub-task >Reporter: Vihang Karajgaonkar >Assignee: László Bodor >Priority: Major > Attachments: HIVE-17441.01.branch-2.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22583) LLAP cache always misses with non-vectorized serde readers such as OpenCSV
[
https://issues.apache.org/jira/browse/HIVE-22583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16991696#comment-16991696
]
Hive QA commented on HIVE-22583:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12988307/HIVE-22583.2.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17763 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19832/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19832/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19832/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12988307 - PreCommit-HIVE-Build
> LLAP cache always misses with non-vectorized serde readers such as OpenCSV
> --
>
> Key: HIVE-22583
> URL: https://issues.apache.org/jira/browse/HIVE-22583
> Project: Hive
> Issue Type: Bug
> Components: llap
>Reporter: Ádám Szita
>Assignee: Ádám Szita
>Priority: Major
> Attachments: HIVE-22583.0.patch, HIVE-22583.1.patch,
> HIVE-22583.2.patch
>
>
> Although after the first read LLAP cache stores data of tables that are not
> using the LazySimple serde, the stored data is then never used in the future
> subsequent queries, causing a full cache miss and re-read each time.
> Problem is rooted in SerdeEncodedDataReader#cacheFileData is not taking care
> of creating an entry for the root/struct column of the table. The only cases
> this is taken care of are when a vectorized reader is used _(e.g.
> LazySimpleSerde's LazySimpleDeserializeRead)_, where
> SerdeEncodedDataReader#processAsyncCacheData takes care of this.
> This can be reproduced by either using a custom serde, like OpenCSV or using
> LazySimpleSerde, but turning off _hive.llap.io.encode.vector.serde.enabled_.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22601) Some columns will be lost when a UDTF has multiple aliases in some cases
[
https://issues.apache.org/jira/browse/HIVE-22601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
okumin updated HIVE-22601:
--
Status: Patch Available (was: In Progress)
> Some columns will be lost when a UDTF has multiple aliases in some cases
>
>
> Key: HIVE-22601
> URL: https://issues.apache.org/jira/browse/HIVE-22601
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 3.1.2
>Reporter: okumin
>Assignee: okumin
>Priority: Major
> Attachments: HIVE-22601.patch
>
>
> Only one column will be retained when putting UDTFs with multiple aliases and
> a top-level UNION together.
> For example, the result of the following SQL should have three columns, c1,
> c2 and c3.
> {code:java}
> SELECT stack(1, 'a', 'b', 'c') AS (c1, c2, c3)
> UNION ALL
> SELECT stack(1, 'd', 'e', 'f') AS (c1, c2, c3);
> {code}
> However, It's only the c3 column which I can get.
> {code:java}
> +-+
> | _u1.c3 |
> +-+
> | c |
> | f |
> +-+
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22601) Some columns will be lost when a UDTF has multiple aliases in some cases
[
https://issues.apache.org/jira/browse/HIVE-22601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
okumin updated HIVE-22601:
--
Attachment: HIVE-22601.patch
> Some columns will be lost when a UDTF has multiple aliases in some cases
>
>
> Key: HIVE-22601
> URL: https://issues.apache.org/jira/browse/HIVE-22601
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 3.1.2
>Reporter: okumin
>Assignee: okumin
>Priority: Major
> Attachments: HIVE-22601.patch
>
>
> Only one column will be retained when putting UDTFs with multiple aliases and
> a top-level UNION together.
> For example, the result of the following SQL should have three columns, c1,
> c2 and c3.
> {code:java}
> SELECT stack(1, 'a', 'b', 'c') AS (c1, c2, c3)
> UNION ALL
> SELECT stack(1, 'd', 'e', 'f') AS (c1, c2, c3);
> {code}
> However, It's only the c3 column which I can get.
> {code:java}
> +-+
> | _u1.c3 |
> +-+
> | c |
> | f |
> +-+
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22601) Some columns will be lost when a UDTF has multiple aliases in some cases
[
https://issues.apache.org/jira/browse/HIVE-22601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
okumin updated HIVE-22601:
--
Attachment: (was: HIVE-22601.patch)
> Some columns will be lost when a UDTF has multiple aliases in some cases
>
>
> Key: HIVE-22601
> URL: https://issues.apache.org/jira/browse/HIVE-22601
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 3.1.2
>Reporter: okumin
>Assignee: okumin
>Priority: Major
> Attachments: HIVE-22601.patch
>
>
> Only one column will be retained when putting UDTFs with multiple aliases and
> a top-level UNION together.
> For example, the result of the following SQL should have three columns, c1,
> c2 and c3.
> {code:java}
> SELECT stack(1, 'a', 'b', 'c') AS (c1, c2, c3)
> UNION ALL
> SELECT stack(1, 'd', 'e', 'f') AS (c1, c2, c3);
> {code}
> However, It's only the c3 column which I can get.
> {code:java}
> +-+
> | _u1.c3 |
> +-+
> | c |
> | f |
> +-+
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-17442) TestCliDriver.testCliDriver[explaindenpendencydiffengs]
[ https://issues.apache.org/jira/browse/HIVE-17442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor updated HIVE-17442: Attachment: HIVE-17442.01.branch-2.patch > TestCliDriver.testCliDriver[explaindenpendencydiffengs] > --- > > Key: HIVE-17442 > URL: https://issues.apache.org/jira/browse/HIVE-17442 > Project: Hive > Issue Type: Sub-task >Reporter: Vihang Karajgaonkar >Assignee: László Bodor >Priority: Major > Attachments: HIVE-17442.01.branch-2.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-17442) TestCliDriver.testCliDriver[explaindenpendencydiffengs]
[ https://issues.apache.org/jira/browse/HIVE-17442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor updated HIVE-17442: Status: Patch Available (was: Open) > TestCliDriver.testCliDriver[explaindenpendencydiffengs] > --- > > Key: HIVE-17442 > URL: https://issues.apache.org/jira/browse/HIVE-17442 > Project: Hive > Issue Type: Sub-task >Reporter: Vihang Karajgaonkar >Assignee: László Bodor >Priority: Major > Attachments: HIVE-17442.01.branch-2.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-17442) TestCliDriver.testCliDriver[explaindenpendencydiffengs]
[ https://issues.apache.org/jira/browse/HIVE-17442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor reassigned HIVE-17442: --- Assignee: László Bodor > TestCliDriver.testCliDriver[explaindenpendencydiffengs] > --- > > Key: HIVE-17442 > URL: https://issues.apache.org/jira/browse/HIVE-17442 > Project: Hive > Issue Type: Sub-task >Reporter: Vihang Karajgaonkar >Assignee: László Bodor >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22579) ACID v1: covered delta-only splits (without base) should be marked as covered (branch-2)
[
https://issues.apache.org/jira/browse/HIVE-22579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16991689#comment-16991689
]
Peter Vary commented on HIVE-22579:
---
+1
> ACID v1: covered delta-only splits (without base) should be marked as covered
> (branch-2)
>
>
> Key: HIVE-22579
> URL: https://issues.apache.org/jira/browse/HIVE-22579
> Project: Hive
> Issue Type: Bug
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
> Attachments: HIVE-22579.01.branch-2.patch,
> HIVE-22579.01.branch-2.patch
>
>
> There is a scenario when different SplitGenerator instances try to cover the
> delta-only buckets (having no base file) more than once, so there could be
> multiple OrcSplit instances generated for the same delta file, causing more
> tasks to read the same delta file more than once, causing duplicate records
> in a simple select star query.
> File structure for a 256 bucket table
> {code}
> drwxrwxrwx - hive hadoop 0 2019-11-29 15:55
> /apps/hive/warehouse/naresh.db/test1/base_013
> -rw-r--r-- 3 hive hadoop353 2019-11-29 15:55
> /apps/hive/warehouse/naresh.db/test1/base_013/bucket_00012
> -rw-r--r-- 3 hive hadoop 1642 2019-11-29 15:55
> /apps/hive/warehouse/naresh.db/test1/base_013/bucket_00140
> drwxrwxrwx - hive hadoop 0 2019-11-29 15:55
> /apps/hive/warehouse/naresh.db/test1/delta_014_014_
> -rwxrwxrwx 3 hive hadoop348 2019-11-29 15:55
> /apps/hive/warehouse/naresh.db/test1/delta_014_014_/bucket_00012
> -rwxrwxrwx 3 hive hadoop 1635 2019-11-29 15:55
> /apps/hive/warehouse/naresh.db/test1/delta_014_014_/bucket_00140
> drwxrwxrwx - hive hadoop 0 2019-11-29 16:04
> /apps/hive/warehouse/naresh.db/test1/delta_015_015_
> -rwxrwxrwx 3 hive hadoop348 2019-11-29 16:04
> /apps/hive/warehouse/naresh.db/test1/delta_015_015_/bucket_00012
> -rwxrwxrwx 3 hive hadoop 1808 2019-11-29 16:04
> /apps/hive/warehouse/naresh.db/test1/delta_015_015_/bucket_00140
> drwxrwxrwx - hive hadoop 0 2019-11-29 16:06
> /apps/hive/warehouse/naresh.db/test1/delta_016_016_
> -rwxrwxrwx 3 hive hadoop348 2019-11-29 16:06
> /apps/hive/warehouse/naresh.db/test1/delta_016_016_/bucket_00043
> -rwxrwxrwx 3 hive hadoop 1633 2019-11-29 16:06
> /apps/hive/warehouse/naresh.db/test1/delta_016_016_/bucket_00171
> {code}
> in this case, when bucket_00171 file has a record, and there is no base file
> for that, a select (*) with ETL split strategy can generate 2 splits for the
> same delta bucket...
> the scenario of the issue:
> 1. ETLSplitStrategy contains a [covered[]
> array|https://github.com/apache/hive/blob/298f749ec7be04abb797fb119f3f0b923c8a1b27/ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java#L763]
> which is [shared between the SplitInfo
> instances|https://github.com/apache/hive/blob/298f749ec7be04abb797fb119f3f0b923c8a1b27/ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java#L824]
> to be created
> 2. a SplitInfo instance is created for [every base file (2 in this
> case)|https://github.com/apache/hive/blob/298f749ec7be04abb797fb119f3f0b923c8a1b27/ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java#L809]
> 3. for every SplitInfo, [a SplitGenerator is
> created|https://github.com/apache/hive/blob/298f749ec7be04abb797fb119f3f0b923c8a1b27/ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java#L925-L926],
> and in the constructor, [parent's getSplit is
> called|https://github.com/apache/hive/blob/298f749ec7be04abb797fb119f3f0b923c8a1b27/ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java#L1251],
> which tries to take care of the deltas
> I'm not sure at the moment what's the intention of this, but this way,
> duplicated delta split can be generated, which can cause duplicated read
> later (note that both tasks read the same delta file: bucket_00171)
> {code}
> 2019-12-01T16:24:53,669 INFO [TezTR-127843_16_30_0_171_0
> (1575040127843_0016_30_00_000171_0)] orc.ReaderImpl: Reading ORC rows from
> hdfs://c3351-node2.squadron.support.hortonworks.com:8020/apps/hive/warehouse/naresh.db/test1/delta_016_016_/bucket_00171
> with {include: [true, true, true, true, true, true, true, true, true, true,
> true, true], offset: 0, length: 9223372036854775807, schema:
> struct}
> 2019-12-01T16:24:53,672 INFO [TezTR-127843_16_30_0_171_0
> (1575040127843_0016_30_00_000171_0)] lib.MRReaderMapred: Processing split:
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat:OrcSplit
>
[jira] [Work started] (HIVE-22601) Some columns will be lost when a UDTF has multiple aliases in some cases
[
https://issues.apache.org/jira/browse/HIVE-22601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HIVE-22601 started by okumin.
-
> Some columns will be lost when a UDTF has multiple aliases in some cases
>
>
> Key: HIVE-22601
> URL: https://issues.apache.org/jira/browse/HIVE-22601
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 3.1.2
>Reporter: okumin
>Assignee: okumin
>Priority: Major
> Attachments: HIVE-22601.patch
>
>
> Only one column will be retained when putting UDTFs with multiple aliases and
> a top-level UNION together.
> For example, the result of the following SQL should have three columns, c1,
> c2 and c3.
> {code:java}
> SELECT stack(1, 'a', 'b', 'c') AS (c1, c2, c3)
> UNION ALL
> SELECT stack(1, 'd', 'e', 'f') AS (c1, c2, c3);
> {code}
> However, It's only the c3 column which I can get.
> {code:java}
> +-+
> | _u1.c3 |
> +-+
> | c |
> | f |
> +-+
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
