[jira] [Commented] (HIVE-22145) Avoid optimizations for analyze compute statistics
[ https://issues.apache.org/jira/browse/HIVE-22145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16935213#comment-16935213 ] Ashutosh Chauhan commented on HIVE-22145: - +1 > Avoid optimizations for analyze compute statistics > -- > > Key: HIVE-22145 > URL: https://issues.apache.org/jira/browse/HIVE-22145 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Affects Versions: 4.0.0 >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-22145.1.patch, HIVE-22145.2.patch, > HIVE-22145.3.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22227) Tez bucket pruning produces wrong result with shared work optimization
[
https://issues.apache.org/jira/browse/HIVE-7?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16935200#comment-16935200
]
Hive QA commented on HIVE-7:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980950/HIVE-7.1.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 16833 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18680/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18680/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18680/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980950 - PreCommit-HIVE-Build
> Tez bucket pruning produces wrong result with shared work optimization
> --
>
> Key: HIVE-7
> URL: https://issues.apache.org/jira/browse/HIVE-7
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Affects Versions: 4.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-7.1.patch
>
>
> *Reproducer*
> {code:sql}
> set hive.tez.bucket.pruning=true;
> set hive.optimize.shared.work=true;
> CREATE TABLE srcbucket_mapjoin_n16(key int, value string) partitioned by (ds
> string) CLUSTERED BY (key) INTO 2 BUCKETS STORED AS TEXTFILE;
> CREATE TABLE tab_part_n10 (key int, value string) PARTITIONED BY(ds STRING)
> CLUSTERED BY (key) SORTED BY (key) INTO 4 BUCKETS STORED AS ORCFILE;
> CREATE TABLE srcbucket_mapjoin_part_n17 (key int, value string) partitioned
> by (ds string) CLUSTERED BY (key) INTO 4 BUCKETS STORED AS TEXTFILE;
> load data local inpath '$HIVE_SRC/data/files/bmj/00_0' INTO TABLE
> srcbucket_mapjoin_n16 partition(ds='2008-04-08');
> load data local inpath '.$HIVE_SRC/data/files/bmj1/01_0' INTO TABLE
> srcbucket_mapjoin_n16 partition(ds='2008-04-08');
> load data local inpath '$HIVE_SRC/data/files/bmj/00_0' INTO TABLE
> srcbucket_mapjoin_part_n17 partition(ds='2008-04-08');
> load data local inpath '$HIVE_SRC/data/files/bmj/01_0' INTO TABLE
> srcbucket_mapjoin_part_n17 partition(ds='2008-04-08');
> load data local inpath '$HIVE_SRC/data/files/bmj/02_0' INTO TABLE
> srcbucket_mapjoin_part_n17 partition(ds='2008-04-08');
> set hive.optimize.bucketingsorting=false;
> insert overwrite table tab_part_n10 partition (ds='2008-04-08')
> select key,value from srcbucket_mapjoin_part_n17;
> CREATE TABLE tab_n9(key int, value string) PARTITIONED BY(ds STRING)
> CLUSTERED BY (key) SORTED BY (key) INTO 2 BUCKETS STORED AS ORCFILE;
> insert overwrite table tab_n9 partition (ds='2008-04-08')
> select key,value from srcbucket_mapjoin_n16;
> select * from
> (select * from tab_n9 where tab_n9.key = 0)a
> join
> (select * from tab_part_n10 where tab_part_n10.key = 98)b full outer join
> tab_part_n10 c on a.key = b.key and b.key = c.key
> order by 1,2,3,4,5,6,7,8,9;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22227) Tez bucket pruning produces wrong result with shared work optimization
[
https://issues.apache.org/jira/browse/HIVE-7?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16935193#comment-16935193
]
Hive QA commented on HIVE-7:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
55s{color} | {color:blue} ql in master has 1570 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
37s{color} | {color:red} ql: The patch generated 1 new + 27 unchanged - 0 fixed
= 28 total (was 27) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 42s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18680/dev-support/hive-personality.sh
|
| git revision | master / 25f0fb4 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18680/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18680/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Tez bucket pruning produces wrong result with shared work optimization
> --
>
> Key: HIVE-7
> URL: https://issues.apache.org/jira/browse/HIVE-7
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Affects Versions: 4.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-7.1.patch
>
>
> *Reproducer*
> {code:sql}
> set hive.tez.bucket.pruning=true;
> set hive.optimize.shared.work=true;
> CREATE TABLE srcbucket_mapjoin_n16(key int, value string) partitioned by (ds
> string) CLUSTERED BY (key) INTO 2 BUCKETS STORED AS TEXTFILE;
> CREATE TABLE tab_part_n10 (key int, value string) PARTITIONED BY(ds STRING)
> CLUSTERED BY (key) SORTED BY (key) INTO 4 BUCKETS STORED AS ORCFILE;
> CREATE TABLE srcbucket_mapjoin_part_n17 (key int, value string) partitioned
> by (ds string) CLUSTERED BY (key) INTO 4 BUCKETS STORED AS TEXTFILE;
> load data local inpath '$HIVE_SRC/data/files/bmj/00_0' INTO TABLE
> srcbucket_mapjoin_n16 partition(ds='2008-04-08');
> load data local inpath '.$HIVE_SRC/data/files/bmj1/01_0' INTO TABLE
> srcbucket_mapjoin_n16 partition(ds='2008-04-08');
> load data local inpath '$HIVE_SRC/data/files/bmj/00_0' INTO TABLE
> srcbucket_mapjoin_part_n17 partition(ds='2008-04-08');
> load data local inpath '$HIVE_SRC/data/files/bmj/01_0' INTO TABLE
> srcbucket_mapjoin_part_n17
[jira] [Work logged] (HIVE-22079) Post order walker for iterating over expression tree
[
https://issues.apache.org/jira/browse/HIVE-22079?focusedWorklogId=316202=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-316202
]
ASF GitHub Bot logged work on HIVE-22079:
-
Author: ASF GitHub Bot
Created on: 22/Sep/19 02:41
Start Date: 22/Sep/19 02:41
Worklog Time Spent: 10m
Work Description: vineetgarg02 commented on pull request #780:
HIVE-22079: Post order walker for iterating over expression tree
URL: https://github.com/apache/hive/pull/780
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 316202)
Remaining Estimate: 0h
Time Spent: 10m
> Post order walker for iterating over expression tree
>
>
> Key: HIVE-22079
> URL: https://issues.apache.org/jira/browse/HIVE-22079
> Project: Hive
> Issue Type: Improvement
> Components: Logical Optimizer, Physical Optimizer
>Affects Versions: 4.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22079.1.patch, HIVE-22079.2.patch,
> HIVE-22079.3.patch, HIVE-22079.4.patch, HIVE-22079.5.patch,
> HIVE-22079.6.patch, HIVE-22079.7.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Current {{DefaultGraphWalker}} is used to iterate over an expression tree.
> This walker uses hash map to keep track of visited/processed nodes. If an
> expression tree is large this adds significant overhead due to map lookup.
> For an expression trees we can instead use post order traversal and avoid
> using map.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22079) Post order walker for iterating over expression tree
[
https://issues.apache.org/jira/browse/HIVE-22079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-22079:
--
Labels: pull-request-available (was: )
> Post order walker for iterating over expression tree
>
>
> Key: HIVE-22079
> URL: https://issues.apache.org/jira/browse/HIVE-22079
> Project: Hive
> Issue Type: Improvement
> Components: Logical Optimizer, Physical Optimizer
>Affects Versions: 4.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22079.1.patch, HIVE-22079.2.patch,
> HIVE-22079.3.patch, HIVE-22079.4.patch, HIVE-22079.5.patch,
> HIVE-22079.6.patch, HIVE-22079.7.patch
>
>
> Current {{DefaultGraphWalker}} is used to iterate over an expression tree.
> This walker uses hash map to keep track of visited/processed nodes. If an
> expression tree is large this adds significant overhead due to map lookup.
> For an expression trees we can instead use post order traversal and avoid
> using map.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22079) Post order walker for iterating over expression tree
[
https://issues.apache.org/jira/browse/HIVE-22079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16935190#comment-16935190
]
Vineet Garg commented on HIVE-22079:
[~jcamachorodriguez] Would you mind taking a look at
https://github.com/apache/hive/pull/780?
> Post order walker for iterating over expression tree
>
>
> Key: HIVE-22079
> URL: https://issues.apache.org/jira/browse/HIVE-22079
> Project: Hive
> Issue Type: Improvement
> Components: Logical Optimizer, Physical Optimizer
>Affects Versions: 4.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22079.1.patch, HIVE-22079.2.patch,
> HIVE-22079.3.patch, HIVE-22079.4.patch, HIVE-22079.5.patch,
> HIVE-22079.6.patch, HIVE-22079.7.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Current {{DefaultGraphWalker}} is used to iterate over an expression tree.
> This walker uses hash map to keep track of visited/processed nodes. If an
> expression tree is large this adds significant overhead due to map lookup.
> For an expression trees we can instead use post order traversal and avoid
> using map.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-20113) Shuffle avoidance: Disable 1-1 edges for sorted shuffle
[ https://issues.apache.org/jira/browse/HIVE-20113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-20113: --- Fix Version/s: 4.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to master. Thanks [~gopalv], [~jcamachorodriguez] > Shuffle avoidance: Disable 1-1 edges for sorted shuffle > > > Key: HIVE-20113 > URL: https://issues.apache.org/jira/browse/HIVE-20113 > Project: Hive > Issue Type: Bug > Components: Tez >Reporter: Gopal V >Assignee: Vineet Garg >Priority: Major > Labels: Branch3Candidate > Fix For: 4.0.0 > > Attachments: HIVE-20113.1.patch, HIVE-20113.10.patch, > HIVE-20113.10.patch, HIVE-20113.2.patch, HIVE-20113.3.patch, > HIVE-20113.4.patch, HIVE-20113.4.patch, HIVE-20113.5.patch, > HIVE-20113.6.patch, HIVE-20113.7.patch, HIVE-20113.8.patch, HIVE-20113.9.patch > > > The sorted shuffle avoidance can have some issues when the shuffle data gets > broken up into multiple chunks on disk. > The 1-1 edge cannot skip the tez final merge - there's no reason for 1-1 to > have a final merge at all, it should open a single compressed file and write > a single index entry. > Until the shuffle issue is resolved & a lot more testing, it is prudent to > disable the optimization for sorted shuffle edges and stop rewriting the > RS(sorted) = = = RS(sorted) into RS(sorted) = = = RS(FORWARD). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22145) Avoid optimizations for analyze compute statistics
[ https://issues.apache.org/jira/browse/HIVE-22145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16935179#comment-16935179 ] Vineet Garg commented on HIVE-22145: [~jcamachorodriguez] [~ashutoshc] Can you take a look? > Avoid optimizations for analyze compute statistics > -- > > Key: HIVE-22145 > URL: https://issues.apache.org/jira/browse/HIVE-22145 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Affects Versions: 4.0.0 >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-22145.1.patch, HIVE-22145.2.patch, > HIVE-22145.3.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22219) Bringing a node manager down blocks restart of LLAP service
[ https://issues.apache.org/jira/browse/HIVE-22219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-22219: --- Fix Version/s: 4.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to master, thanks for reviewing [~bslim]. > Bringing a node manager down blocks restart of LLAP service > --- > > Key: HIVE-22219 > URL: https://issues.apache.org/jira/browse/HIVE-22219 > Project: Hive > Issue Type: Bug > Components: llap >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22219.patch, HIVE-22219.patch > > > For YARN service, when number of running instances != number of desired > instances, the service state may be in STARTED or FLEX (instead of STABLE). > For Hive LLAP side, there is a config to control the threshold of service > health check. The Hive LLAP code misses checking these states, which can > result in the service not coming up even if the threshold is met. > https://github.com/apache/hive/blob/master/llap-server/src/java/org/apache/hadoop/hive/llap/cli/status/LlapStatusServiceDriver.java#L382 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22219) Bringing a node manager down blocks restart of LLAP service
[
https://issues.apache.org/jira/browse/HIVE-22219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16935025#comment-16935025
]
Hive QA commented on HIVE-22219:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980947/HIVE-22219.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 16833 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18679/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18679/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18679/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980947 - PreCommit-HIVE-Build
> Bringing a node manager down blocks restart of LLAP service
> ---
>
> Key: HIVE-22219
> URL: https://issues.apache.org/jira/browse/HIVE-22219
> Project: Hive
> Issue Type: Bug
> Components: llap
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-22219.patch, HIVE-22219.patch
>
>
> For YARN service, when number of running instances != number of desired
> instances, the service state may be in STARTED or FLEX (instead of STABLE).
> For Hive LLAP side, there is a config to control the threshold of service
> health check. The Hive LLAP code misses checking these states, which can
> result in the service not coming up even if the threshold is met.
> https://github.com/apache/hive/blob/master/llap-server/src/java/org/apache/hadoop/hive/llap/cli/status/LlapStatusServiceDriver.java#L382
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22219) Bringing a node manager down blocks restart of LLAP service
[
https://issues.apache.org/jira/browse/HIVE-22219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16935000#comment-16935000
]
Hive QA commented on HIVE-22219:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
43s{color} | {color:blue} llap-server in master has 90 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 12m 42s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18679/dev-support/hive-personality.sh
|
| git revision | master / 8b571ea |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: llap-server U: llap-server |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18679/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Bringing a node manager down blocks restart of LLAP service
> ---
>
> Key: HIVE-22219
> URL: https://issues.apache.org/jira/browse/HIVE-22219
> Project: Hive
> Issue Type: Bug
> Components: llap
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-22219.patch, HIVE-22219.patch
>
>
> For YARN service, when number of running instances != number of desired
> instances, the service state may be in STARTED or FLEX (instead of STABLE).
> For Hive LLAP side, there is a config to control the threshold of service
> health check. The Hive LLAP code misses checking these states, which can
> result in the service not coming up even if the threshold is met.
> https://github.com/apache/hive/blob/master/llap-server/src/java/org/apache/hadoop/hive/llap/cli/status/LlapStatusServiceDriver.java#L382
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22145) Avoid optimizations for analyze compute statistics
[
https://issues.apache.org/jira/browse/HIVE-22145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16934994#comment-16934994
]
Hive QA commented on HIVE-22145:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980946/HIVE-22145.3.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 16833 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18678/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18678/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18678/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980946 - PreCommit-HIVE-Build
> Avoid optimizations for analyze compute statistics
> --
>
> Key: HIVE-22145
> URL: https://issues.apache.org/jira/browse/HIVE-22145
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Affects Versions: 4.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-22145.1.patch, HIVE-22145.2.patch,
> HIVE-22145.3.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22217) Better Logging for Hive JAR Reload
[ https://issues.apache.org/jira/browse/HIVE-22217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laszlo Bodor updated HIVE-22217: Resolution: Fixed Status: Resolved (was: Patch Available) > Better Logging for Hive JAR Reload > -- > > Key: HIVE-22217 > URL: https://issues.apache.org/jira/browse/HIVE-22217 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Affects Versions: 3.2.0, 2.3.6 >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Fix For: 4.0.0 > > Attachments: HIVE-22217.1.patch > > > Troubleshooting Hive Reloadable Auxiliary JARs has always been difficult. > Add logging to at least confirm which JAR files are being loaded. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22217) Better Logging for Hive JAR Reload
[ https://issues.apache.org/jira/browse/HIVE-22217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laszlo Bodor updated HIVE-22217: Fix Version/s: 4.0.0 > Better Logging for Hive JAR Reload > -- > > Key: HIVE-22217 > URL: https://issues.apache.org/jira/browse/HIVE-22217 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Affects Versions: 3.2.0, 2.3.6 >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Fix For: 4.0.0 > > Attachments: HIVE-22217.1.patch > > > Troubleshooting Hive Reloadable Auxiliary JARs has always been difficult. > Add logging to at least confirm which JAR files are being loaded. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22217) Better Logging for Hive JAR Reload
[ https://issues.apache.org/jira/browse/HIVE-22217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16934990#comment-16934990 ] Laszlo Bodor commented on HIVE-22217: - pushed to master, thanks [~belugabehr] for the patch and [~ngangam] for the review! > Better Logging for Hive JAR Reload > -- > > Key: HIVE-22217 > URL: https://issues.apache.org/jira/browse/HIVE-22217 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Affects Versions: 3.2.0, 2.3.6 >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Attachments: HIVE-22217.1.patch > > > Troubleshooting Hive Reloadable Auxiliary JARs has always been difficult. > Add logging to at least confirm which JAR files are being loaded. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22145) Avoid optimizations for analyze compute statistics
[
https://issues.apache.org/jira/browse/HIVE-22145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16934985#comment-16934985
]
Hive QA commented on HIVE-22145:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
0s{color} | {color:blue} ql in master has 1571 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 6s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18678/dev-support/hive-personality.sh
|
| git revision | master / 1278bf1 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18678/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Avoid optimizations for analyze compute statistics
> --
>
> Key: HIVE-22145
> URL: https://issues.apache.org/jira/browse/HIVE-22145
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Affects Versions: 4.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-22145.1.patch, HIVE-22145.2.patch,
> HIVE-22145.3.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-20113) Shuffle avoidance: Disable 1-1 edges for sorted shuffle
[
https://issues.apache.org/jira/browse/HIVE-20113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16934981#comment-16934981
]
Hive QA commented on HIVE-20113:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980945/HIVE-20113.10.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 16833 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18677/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18677/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18677/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980945 - PreCommit-HIVE-Build
> Shuffle avoidance: Disable 1-1 edges for sorted shuffle
>
>
> Key: HIVE-20113
> URL: https://issues.apache.org/jira/browse/HIVE-20113
> Project: Hive
> Issue Type: Bug
> Components: Tez
>Reporter: Gopal V
>Assignee: Vineet Garg
>Priority: Major
> Labels: Branch3Candidate
> Attachments: HIVE-20113.1.patch, HIVE-20113.10.patch,
> HIVE-20113.10.patch, HIVE-20113.2.patch, HIVE-20113.3.patch,
> HIVE-20113.4.patch, HIVE-20113.4.patch, HIVE-20113.5.patch,
> HIVE-20113.6.patch, HIVE-20113.7.patch, HIVE-20113.8.patch, HIVE-20113.9.patch
>
>
> The sorted shuffle avoidance can have some issues when the shuffle data gets
> broken up into multiple chunks on disk.
> The 1-1 edge cannot skip the tez final merge - there's no reason for 1-1 to
> have a final merge at all, it should open a single compressed file and write
> a single index entry.
> Until the shuffle issue is resolved & a lot more testing, it is prudent to
> disable the optimization for sorted shuffle edges and stop rewriting the
> RS(sorted) = = = RS(sorted) into RS(sorted) = = = RS(FORWARD).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-20113) Shuffle avoidance: Disable 1-1 edges for sorted shuffle
[
https://issues.apache.org/jira/browse/HIVE-20113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16934977#comment-16934977
]
Hive QA commented on HIVE-20113:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
55s{color} | {color:blue} ql in master has 1571 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
41s{color} | {color:red} ql: The patch generated 1 new + 347 unchanged - 1
fixed = 348 total (was 348) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 12s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18677/dev-support/hive-personality.sh
|
| git revision | master / 1278bf1 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18677/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18677/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Shuffle avoidance: Disable 1-1 edges for sorted shuffle
>
>
> Key: HIVE-20113
> URL: https://issues.apache.org/jira/browse/HIVE-20113
> Project: Hive
> Issue Type: Bug
> Components: Tez
>Reporter: Gopal V
>Assignee: Vineet Garg
>Priority: Major
> Labels: Branch3Candidate
> Attachments: HIVE-20113.1.patch, HIVE-20113.10.patch,
> HIVE-20113.10.patch, HIVE-20113.2.patch, HIVE-20113.3.patch,
> HIVE-20113.4.patch, HIVE-20113.4.patch, HIVE-20113.5.patch,
> HIVE-20113.6.patch, HIVE-20113.7.patch, HIVE-20113.8.patch, HIVE-20113.9.patch
>
>
> The sorted shuffle avoidance can have some issues when the shuffle data gets
> broken up into multiple chunks on disk.
> The 1-1 edge cannot skip the tez final merge - there's no reason for 1-1 to
> have a final merge at all, it should open a single compressed file and write
> a single index entry.
> Until the shuffle issue is resolved & a lot more testing, it is prudent to
> disable the optimization for sorted shuffle edges and stop rewriting the
> RS(sorted) = = = RS(sorted) into RS(sorted) = = = RS(FORWARD).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22079) Post order walker for iterating over expression tree
[
https://issues.apache.org/jira/browse/HIVE-22079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16934974#comment-16934974
]
Hive QA commented on HIVE-22079:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980938/HIVE-22079.7.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 16829 tests
executed
*Failed tests:*
{noformat}
TestStatsReplicationScenariosACIDNoAutogather - did not produce a TEST-*.xml
file (likely timed out) (batchId=254)
org.apache.hadoop.hive.llap.cache.TestBuddyAllocator.testMTT[2] (batchId=361)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18676/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18676/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18676/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980938 - PreCommit-HIVE-Build
> Post order walker for iterating over expression tree
>
>
> Key: HIVE-22079
> URL: https://issues.apache.org/jira/browse/HIVE-22079
> Project: Hive
> Issue Type: Improvement
> Components: Logical Optimizer, Physical Optimizer
>Affects Versions: 4.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-22079.1.patch, HIVE-22079.2.patch,
> HIVE-22079.3.patch, HIVE-22079.4.patch, HIVE-22079.5.patch,
> HIVE-22079.6.patch, HIVE-22079.7.patch
>
>
> Current {{DefaultGraphWalker}} is used to iterate over an expression tree.
> This walker uses hash map to keep track of visited/processed nodes. If an
> expression tree is large this adds significant overhead due to map lookup.
> For an expression trees we can instead use post order traversal and avoid
> using map.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-22222) Clean up the error handling in Driver - get rid of global variables
[ https://issues.apache.org/jira/browse/HIVE-2?focusedWorklogId=316086=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-316086 ] ASF GitHub Bot logged work on HIVE-2: - Author: ASF GitHub Bot Created on: 21/Sep/19 08:12 Start Date: 21/Sep/19 08:12 Worklog Time Spent: 10m Work Description: miklosgergely commented on pull request #779: HIVE-2 Clean up the error handling in Driver - get rid of global variables URL: https://github.com/apache/hive/pull/779 The error handling in Hive is done with some global variables for no apparent reason, as all the data that is gathered to described an exception are produced and used at the point where the exception occurred. Thus having global variables is misleading. Also imported the heavily used org.apache.hadoop.util.StringUtils instead of the only one time used org.apache.commons.lang.StringUtils, thus the former shouldn't be referenced everywhere with it's fully qualified class name. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 316086) Remaining Estimate: 0h Time Spent: 10m > Clean up the error handling in Driver - get rid of global variables > --- > > Key: HIVE-2 > URL: https://issues.apache.org/jira/browse/HIVE-2 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-2.01.patch, HIVE-2.02.patch > > Time Spent: 10m > Remaining Estimate: 0h > > The error handling in Hive is done with some global variables for no apparent > reason, as all the data that is gathered to described an exception are > produced and used at the point where the exception occurred. Thus having > global variables is misleading. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22222) Clean up the error handling in Driver - get rid of global variables
[ https://issues.apache.org/jira/browse/HIVE-2?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-2: -- Labels: pull-request-available (was: ) > Clean up the error handling in Driver - get rid of global variables > --- > > Key: HIVE-2 > URL: https://issues.apache.org/jira/browse/HIVE-2 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-2.01.patch, HIVE-2.02.patch > > > The error handling in Hive is done with some global variables for no apparent > reason, as all the data that is gathered to described an exception are > produced and used at the point where the exception occurred. Thus having > global variables is misleading. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22222) Clean up the error handling in Driver - get rid of global variables
[ https://issues.apache.org/jira/browse/HIVE-2?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-2: -- Attachment: HIVE-2.02.patch > Clean up the error handling in Driver - get rid of global variables > --- > > Key: HIVE-2 > URL: https://issues.apache.org/jira/browse/HIVE-2 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-2.01.patch, HIVE-2.02.patch > > > The error handling in Hive is done with some global variables for no apparent > reason, as all the data that is gathered to described an exception are > produced and used at the point where the exception occurred. Thus having > global variables is misleading. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-21164) ACID: explore how we can avoid a move step during inserts/compaction
[ https://issues.apache.org/jira/browse/HIVE-21164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laszlo Bodor updated HIVE-21164: Attachment: HIVE-21164.01.patch > ACID: explore how we can avoid a move step during inserts/compaction > > > Key: HIVE-21164 > URL: https://issues.apache.org/jira/browse/HIVE-21164 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 3.1.1 >Reporter: Vaibhav Gumashta >Assignee: Laszlo Bodor >Priority: Major > Attachments: HIVE-21164.1.patch, HIVE-21164.10.patch, > HIVE-21164.11.patch, HIVE-21164.2.patch, HIVE-21164.3.patch, > HIVE-21164.4.patch, HIVE-21164.5.patch, HIVE-21164.6.patch, > HIVE-21164.7.patch, HIVE-21164.8.patch, HIVE-21164.9.patch > > > Currently, we write compacted data to a temporary location and then move the > files to a final location, which is an expensive operation on some cloud file > systems. Since HIVE-20823 is already in, it can control the visibility of > compacted data for the readers. Therefore, we can perhaps avoid writing data > to a temporary location and directly write compacted data to the intended > final path. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-21164) ACID: explore how we can avoid a move step during inserts/compaction
[ https://issues.apache.org/jira/browse/HIVE-21164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laszlo Bodor updated HIVE-21164: Attachment: (was: HIVE-21164.01.patch) > ACID: explore how we can avoid a move step during inserts/compaction > > > Key: HIVE-21164 > URL: https://issues.apache.org/jira/browse/HIVE-21164 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 3.1.1 >Reporter: Vaibhav Gumashta >Assignee: Laszlo Bodor >Priority: Major > Attachments: HIVE-21164.1.patch, HIVE-21164.10.patch, > HIVE-21164.11.patch, HIVE-21164.2.patch, HIVE-21164.3.patch, > HIVE-21164.4.patch, HIVE-21164.5.patch, HIVE-21164.6.patch, > HIVE-21164.7.patch, HIVE-21164.8.patch, HIVE-21164.9.patch > > > Currently, we write compacted data to a temporary location and then move the > files to a final location, which is an expensive operation on some cloud file > systems. Since HIVE-20823 is already in, it can control the visibility of > compacted data for the readers. Therefore, we can perhaps avoid writing data > to a temporary location and directly write compacted data to the intended > final path. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-21164) ACID: explore how we can avoid a move step during inserts/compaction
[ https://issues.apache.org/jira/browse/HIVE-21164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laszlo Bodor updated HIVE-21164: Attachment: HIVE-21164.11.patch > ACID: explore how we can avoid a move step during inserts/compaction > > > Key: HIVE-21164 > URL: https://issues.apache.org/jira/browse/HIVE-21164 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 3.1.1 >Reporter: Vaibhav Gumashta >Assignee: Laszlo Bodor >Priority: Major > Attachments: HIVE-21164.1.patch, HIVE-21164.10.patch, > HIVE-21164.11.patch, HIVE-21164.2.patch, HIVE-21164.3.patch, > HIVE-21164.4.patch, HIVE-21164.5.patch, HIVE-21164.6.patch, > HIVE-21164.7.patch, HIVE-21164.8.patch, HIVE-21164.9.patch > > > Currently, we write compacted data to a temporary location and then move the > files to a final location, which is an expensive operation on some cloud file > systems. Since HIVE-20823 is already in, it can control the visibility of > compacted data for the readers. Therefore, we can perhaps avoid writing data > to a temporary location and directly write compacted data to the intended > final path. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22079) Post order walker for iterating over expression tree
[
https://issues.apache.org/jira/browse/HIVE-22079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16934966#comment-16934966
]
Hive QA commented on HIVE-22079:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
5s{color} | {color:blue} ql in master has 1571 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
36s{color} | {color:red} ql: The patch generated 4 new + 104 unchanged - 1
fixed = 108 total (was 105) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
8s{color} | {color:red} ql generated 1 new + 1571 unchanged - 0 fixed = 1572
total (was 1571) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 1s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Should org.apache.hadoop.hive.ql.lib.ExpressionWalker$NodeLabeled be a
_static_ inner class? At ExpressionWalker.java:inner class? At
ExpressionWalker.java:[lines 47-61] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18676/dev-support/hive-personality.sh
|
| git revision | master / 1278bf1 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18676/yetus/diff-checkstyle-ql.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18676/yetus/new-findbugs-ql.html
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18676/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Post order walker for iterating over expression tree
>
>
> Key: HIVE-22079
> URL: https://issues.apache.org/jira/browse/HIVE-22079
> Project: Hive
> Issue Type: Improvement
> Components: Logical Optimizer, Physical Optimizer
>Affects Versions: 4.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-22079.1.patch, HIVE-22079.2.patch,
> HIVE-22079.3.patch, HIVE-22079.4.patch, HIVE-22079.5.patch,
> HIVE-22079.6.patch, HIVE-22079.7.patch
>
>
> Current {{DefaultGraphWalker}} is used to iterate over an expression tree.
> This walker uses hash map to keep track of visited/processed nodes. If an
> expression tree is large this adds significant overhead due to map lookup.
> For an expression trees we can instead use post order traversal and avoid
> using map.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22225) add owner name for create database if missing
[
https://issues.apache.org/jira/browse/HIVE-5?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16934958#comment-16934958
]
Hive QA commented on HIVE-5:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980937/HIVE-5.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 16829 tests
executed
*Failed tests:*
{noformat}
TestStatsReplicationScenariosACIDNoAutogather - did not produce a TEST-*.xml
file (likely timed out) (batchId=254)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18675/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18675/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18675/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980937 - PreCommit-HIVE-Build
> add owner name for create database if missing
> -
>
> Key: HIVE-5
> URL: https://issues.apache.org/jira/browse/HIVE-5
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Sam An
>Assignee: Sam An
>Priority: Minor
> Attachments: HIVE-5.1.patch
>
>
> When Spark connects to HMS, the database owner name is not filled. This could
> happen to other clients as well. We shall fill this in HMS. Ownership info is
> useful for authorizer component in Ranger, etc.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
