[jira] [Updated] (HIVE-22481) Expose null sort order at default level
[ https://issues.apache.org/jira/browse/HIVE-22481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-22481: -- Attachment: HIVE-22481.3.patch > Expose null sort order at default level > --- > > Key: HIVE-22481 > URL: https://issues.apache.org/jira/browse/HIVE-22481 > Project: Hive > Issue Type: Improvement >Reporter: Krisztian Kasa >Assignee: Krisztian Kasa >Priority: Minor > Attachments: HIVE-22481.1.patch, HIVE-22481.1.patch, > HIVE-22481.2.patch, HIVE-22481.3.patch > > > ReduceSinkDesc > https://github.com/apache/hive/blob/91f9ff5648854c487860a79b82450538fbcf6c48/ql/src/java/org/apache/hadoop/hive/ql/plan/ReduceSinkDesc.java#L410 > and TopNKeyDesc -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22481) Expose null sort order at default level
[ https://issues.apache.org/jira/browse/HIVE-22481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-22481: -- Status: Patch Available (was: Open) > Expose null sort order at default level > --- > > Key: HIVE-22481 > URL: https://issues.apache.org/jira/browse/HIVE-22481 > Project: Hive > Issue Type: Improvement >Reporter: Krisztian Kasa >Assignee: Krisztian Kasa >Priority: Minor > Attachments: HIVE-22481.1.patch, HIVE-22481.1.patch, > HIVE-22481.2.patch, HIVE-22481.3.patch > > > ReduceSinkDesc > https://github.com/apache/hive/blob/91f9ff5648854c487860a79b82450538fbcf6c48/ql/src/java/org/apache/hadoop/hive/ql/plan/ReduceSinkDesc.java#L410 > and TopNKeyDesc -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22481) Expose null sort order at default level
[ https://issues.apache.org/jira/browse/HIVE-22481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-22481: -- Status: Open (was: Patch Available) > Expose null sort order at default level > --- > > Key: HIVE-22481 > URL: https://issues.apache.org/jira/browse/HIVE-22481 > Project: Hive > Issue Type: Improvement >Reporter: Krisztian Kasa >Assignee: Krisztian Kasa >Priority: Minor > Attachments: HIVE-22481.1.patch, HIVE-22481.1.patch, > HIVE-22481.2.patch, HIVE-22481.3.patch > > > ReduceSinkDesc > https://github.com/apache/hive/blob/91f9ff5648854c487860a79b82450538fbcf6c48/ql/src/java/org/apache/hadoop/hive/ql/plan/ReduceSinkDesc.java#L410 > and TopNKeyDesc -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22512) Use direct SQL to fetch column privileges in refreshPrivileges
[
https://issues.apache.org/jira/browse/HIVE-22512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16981348#comment-16981348
]
Hive QA commented on HIVE-22512:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12986630/HIVE-22512.03.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17717 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.schq.TestScheduledQueryIntegration.testScheduledQueryExecutionImpersonation
(batchId=279)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19591/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19591/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19591/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12986630 - PreCommit-HIVE-Build
> Use direct SQL to fetch column privileges in refreshPrivileges
> --
>
> Key: HIVE-22512
> URL: https://issues.apache.org/jira/browse/HIVE-22512
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
>Affects Versions: 4.0.0
>Reporter: Ashutosh Bapat
>Assignee: Ashutosh Bapat
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22512.01.patch, HIVE-22512.02.patch,
> HIVE-22512.03.patch
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> refreshPrivileges() calls listTableAllColumnGrants() to fetch the column
> level privileges. The later function retrieves the individual column objects
> by firing one query per column privilege object, thus causing the backend db
> to be swamped by these queries when PrivilegeSynchronizer is run.
> PrivilegeSynchronizer synchronizes privileges of all the databases, tables
> and columns and thus the backend db can get swamped really bad when there are
> thousands of tables with hundreds of columns.
> The output of listTableAllColumnGrants() is not used completely so all the
> columns the PM has tried to retrieves anyway goes waste.
> Fix this by using direct SQL to fetch column privileges.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22523) The error handler in LlapRecordReader might block if its queue is full
[ https://issues.apache.org/jira/browse/HIVE-22523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Magyar updated HIVE-22523: - Status: Patch Available (was: Open) > The error handler in LlapRecordReader might block if its queue is full > -- > > Key: HIVE-22523 > URL: https://issues.apache.org/jira/browse/HIVE-22523 > Project: Hive > Issue Type: Bug >Reporter: Attila Magyar >Assignee: Attila Magyar >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22523.1.patch, HIVE-22523.2.patch > > > In setError() we set the value of an atomic reference (pendingError) and we > also put the error in a queue. The latter seems not just unnecessary but it > might block the caller of the handler if the queue is full. Also closing of > the reader is might not properly handled as some of the flags are not > volatile. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22523) The error handler in LlapRecordReader might block if its queue is full
[ https://issues.apache.org/jira/browse/HIVE-22523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Magyar updated HIVE-22523: - Status: Open (was: Patch Available) > The error handler in LlapRecordReader might block if its queue is full > -- > > Key: HIVE-22523 > URL: https://issues.apache.org/jira/browse/HIVE-22523 > Project: Hive > Issue Type: Bug >Reporter: Attila Magyar >Assignee: Attila Magyar >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22523.1.patch, HIVE-22523.2.patch > > > In setError() we set the value of an atomic reference (pendingError) and we > also put the error in a queue. The latter seems not just unnecessary but it > might block the caller of the handler if the queue is full. Also closing of > the reader is might not properly handled as some of the flags are not > volatile. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22523) The error handler in LlapRecordReader might block if its queue is full
[ https://issues.apache.org/jira/browse/HIVE-22523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Magyar updated HIVE-22523: - Attachment: HIVE-22523.2.patch > The error handler in LlapRecordReader might block if its queue is full > -- > > Key: HIVE-22523 > URL: https://issues.apache.org/jira/browse/HIVE-22523 > Project: Hive > Issue Type: Bug >Reporter: Attila Magyar >Assignee: Attila Magyar >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22523.1.patch, HIVE-22523.2.patch > > > In setError() we set the value of an atomic reference (pendingError) and we > also put the error in a queue. The latter seems not just unnecessary but it > might block the caller of the handler if the queue is full. Also closing of > the reader is might not properly handled as some of the flags are not > volatile. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22512) Use direct SQL to fetch column privileges in refreshPrivileges
[
https://issues.apache.org/jira/browse/HIVE-22512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16981327#comment-16981327
]
Hive QA commented on HIVE-22512:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
25s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
18s{color} | {color:blue} standalone-metastore/metastore-server in master has
178 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
23s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
25s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 7 new + 615 unchanged - 6 fixed = 622 total (was 621) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 17m 13s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19591/dev-support/hive-personality.sh
|
| git revision | master / a70408a |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19591/yetus/diff-checkstyle-standalone-metastore_metastore-server.txt
|
| modules | C: standalone-metastore/metastore-server U:
standalone-metastore/metastore-server |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19591/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Use direct SQL to fetch column privileges in refreshPrivileges
> --
>
> Key: HIVE-22512
> URL: https://issues.apache.org/jira/browse/HIVE-22512
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
>Affects Versions: 4.0.0
>Reporter: Ashutosh Bapat
>Assignee: Ashutosh Bapat
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22512.01.patch, HIVE-22512.02.patch,
> HIVE-22512.03.patch
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> refreshPrivileges() calls listTableAllColumnGrants() to fetch the column
> level privileges. The later function retrieves the individual column objects
> by firing one query per column privilege object, thus causing the backend db
> to be swamped by these queries when PrivilegeSynchronizer is run.
> PrivilegeSynchronizer synchronizes privileges of all the databases, tables
> and columns and thus the backend db can get swamped really bad when there are
> thousands of tables with hundreds of columns.
> The output of listTableAllColumnGrants() is not used completely so all the
> columns the PM has tried to retrieves anyway goes waste.
> Fix this by using direct SQL to fetch column privileges.
--
This message was
[jira] [Commented] (HIVE-22532) PTFPPD may push limit incorrectly through Rank/DenseRank function
[
https://issues.apache.org/jira/browse/HIVE-22532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16981318#comment-16981318
]
Hive QA commented on HIVE-22532:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12986625/HIVE-22532.03.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17718 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19590/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19590/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19590/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12986625 - PreCommit-HIVE-Build
> PTFPPD may push limit incorrectly through Rank/DenseRank function
> -
>
> Key: HIVE-22532
> URL: https://issues.apache.org/jira/browse/HIVE-22532
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22532.01.patch, HIVE-22532.02.patch,
> HIVE-22532.03.patch, HIVE-22532.03.patch, HIVE-22532.03.patch,
> HIVE-22532.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22521) Both Driver and SessionState has a userName
[ https://issues.apache.org/jira/browse/HIVE-22521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-22521: Attachment: HIVE-22521.01.patch > Both Driver and SessionState has a userName > --- > > Key: HIVE-22521 > URL: https://issues.apache.org/jira/browse/HIVE-22521 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-22521.01.patch, HIVE-22521.01.patch, > HIVE-22521.01.patch, HIVE-22521.01.patch > > > This caused some confusing behaviour to me...especially when the 2 values > were different. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22517) Sysdb related qtests also output the sysdb sql commands to q.out
[ https://issues.apache.org/jira/browse/HIVE-22517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-22517: Attachment: HIVE-22517.02.patch > Sysdb related qtests also output the sysdb sql commands to q.out > > > Key: HIVE-22517 > URL: https://issues.apache.org/jira/browse/HIVE-22517 > Project: Hive > Issue Type: Improvement > Components: Test >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-22517.01.patch, HIVE-22517.02.patch, > HIVE-22517.02.patch, HIVE-22517.02.patch, HIVE-22517.02.patch > > > it would be better to not have it on the outputs -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22512) Use direct SQL to fetch column privileges in refreshPrivileges
[ https://issues.apache.org/jira/browse/HIVE-22512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Bapat updated HIVE-22512: -- Attachment: HIVE-22512.03.patch Status: Patch Available (was: Open) > Use direct SQL to fetch column privileges in refreshPrivileges > -- > > Key: HIVE-22512 > URL: https://issues.apache.org/jira/browse/HIVE-22512 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 4.0.0 >Reporter: Ashutosh Bapat >Assignee: Ashutosh Bapat >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22512.01.patch, HIVE-22512.02.patch, > HIVE-22512.03.patch > > Time Spent: 1.5h > Remaining Estimate: 0h > > refreshPrivileges() calls listTableAllColumnGrants() to fetch the column > level privileges. The later function retrieves the individual column objects > by firing one query per column privilege object, thus causing the backend db > to be swamped by these queries when PrivilegeSynchronizer is run. > PrivilegeSynchronizer synchronizes privileges of all the databases, tables > and columns and thus the backend db can get swamped really bad when there are > thousands of tables with hundreds of columns. > The output of listTableAllColumnGrants() is not used completely so all the > columns the PM has tried to retrieves anyway goes waste. > Fix this by using direct SQL to fetch column privileges. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22512) Use direct SQL to fetch column privileges in refreshPrivileges
[ https://issues.apache.org/jira/browse/HIVE-22512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Bapat updated HIVE-22512: -- Status: Open (was: Patch Available) > Use direct SQL to fetch column privileges in refreshPrivileges > -- > > Key: HIVE-22512 > URL: https://issues.apache.org/jira/browse/HIVE-22512 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 4.0.0 >Reporter: Ashutosh Bapat >Assignee: Ashutosh Bapat >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22512.01.patch, HIVE-22512.02.patch > > Time Spent: 1.5h > Remaining Estimate: 0h > > refreshPrivileges() calls listTableAllColumnGrants() to fetch the column > level privileges. The later function retrieves the individual column objects > by firing one query per column privilege object, thus causing the backend db > to be swamped by these queries when PrivilegeSynchronizer is run. > PrivilegeSynchronizer synchronizes privileges of all the databases, tables > and columns and thus the backend db can get swamped really bad when there are > thousands of tables with hundreds of columns. > The output of listTableAllColumnGrants() is not used completely so all the > columns the PM has tried to retrieves anyway goes waste. > Fix this by using direct SQL to fetch column privileges. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22512) Use direct SQL to fetch column privileges in refreshPrivileges
[ https://issues.apache.org/jira/browse/HIVE-22512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Bapat updated HIVE-22512: -- Attachment: (was: HIVE-22512.03.patch) > Use direct SQL to fetch column privileges in refreshPrivileges > -- > > Key: HIVE-22512 > URL: https://issues.apache.org/jira/browse/HIVE-22512 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 4.0.0 >Reporter: Ashutosh Bapat >Assignee: Ashutosh Bapat >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22512.01.patch, HIVE-22512.02.patch > > Time Spent: 1.5h > Remaining Estimate: 0h > > refreshPrivileges() calls listTableAllColumnGrants() to fetch the column > level privileges. The later function retrieves the individual column objects > by firing one query per column privilege object, thus causing the backend db > to be swamped by these queries when PrivilegeSynchronizer is run. > PrivilegeSynchronizer synchronizes privileges of all the databases, tables > and columns and thus the backend db can get swamped really bad when there are > thousands of tables with hundreds of columns. > The output of listTableAllColumnGrants() is not used completely so all the > columns the PM has tried to retrieves anyway goes waste. > Fix this by using direct SQL to fetch column privileges. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22532) PTFPPD may push limit incorrectly through Rank/DenseRank function
[
https://issues.apache.org/jira/browse/HIVE-22532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16981311#comment-16981311
]
Hive QA commented on HIVE-22532:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
13s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
31s{color} | {color:blue} ql in master has 1539 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
30s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 29m 2s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19590/dev-support/hive-personality.sh
|
| git revision | master / a70408a |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql itests U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19590/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> PTFPPD may push limit incorrectly through Rank/DenseRank function
> -
>
> Key: HIVE-22532
> URL: https://issues.apache.org/jira/browse/HIVE-22532
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22532.01.patch, HIVE-22532.02.patch,
> HIVE-22532.03.patch, HIVE-22532.03.patch, HIVE-22532.03.patch,
> HIVE-22532.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22526) Extract Compiler from Driver
[
https://issues.apache.org/jira/browse/HIVE-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16981296#comment-16981296
]
Hive QA commented on HIVE-22526:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12986621/HIVE-22526.05.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17717 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19589/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19589/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19589/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12986621 - PreCommit-HIVE-Build
> Extract Compiler from Driver
>
>
> Key: HIVE-22526
> URL: https://issues.apache.org/jira/browse/HIVE-22526
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22526.01.patch, HIVE-22526.02.patch,
> HIVE-22526.03.patch, HIVE-22526.04.patch, HIVE-22526.05.patch
>
>
> The Driver class contains ~600 lines of code responsible for compiling the
> command. That means that from the command String a Plan needs to be created,
> and also a transaction needs to be started (in most of the cases). This is a
> thing done by the compile function, which has a lot of sub functions to help
> this task, while itself is also really big. All these codes should be put
> into a separate class, where it can do it's job without getting mixed with
> the other codes in the Driver.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22428) Remove superfluous "Failed to get database" WARN Logging in ObjectStore
[
https://issues.apache.org/jira/browse/HIVE-22428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16981295#comment-16981295
]
Ashutosh Bapat commented on HIVE-22428:
---
[~belugabehr],
That's true, however, that DEBUG is printed on the previous line and not on the
line where "Thread Stack Trace ..." is printed. Since outputs from different
threads can be intermingled, usually the two consecutive lines are not
associated with each other.
> Remove superfluous "Failed to get database" WARN Logging in ObjectStore
> ---
>
> Key: HIVE-22428
> URL: https://issues.apache.org/jira/browse/HIVE-22428
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Affects Versions: 3.2.0
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22428.1.patch
>
>
> In my testing, I get lots of logs like this:
> {code:none}
> Line 26319: 2019-10-28T21:09:52,134 WARN [pool-6-thread-5]
> metastore.ObjectStore: Failed to get database hive.compdb, returning
> NoSuchObjectException
> Line 26327: 2019-10-28T21:09:52,135 WARN [pool-6-thread-5]
> metastore.ObjectStore: Failed to get database hive.compdb, returning
> NoSuchObjectException
> Line 26504: 2019-10-28T21:09:52,600 WARN [pool-6-thread-5]
> metastore.ObjectStore: Failed to get database hive.tstatsfast, returning
> NoSuchObjectException
> Line 26519: 2019-10-28T21:09:52,606 WARN [pool-6-thread-5]
> metastore.ObjectStore: Failed to get database hive.tstatsfast, returning
> NoSuchObjectException
> Line 26695: 2019-10-28T21:09:52,922 WARN [pool-6-thread-5]
> metastore.ObjectStore: Failed to get database hive.createDb, returning
> NoSuchObjectException
> Line 26703: 2019-10-28T21:09:52,923 WARN [pool-6-thread-5]
> metastore.ObjectStore: Failed to get database hive.createDb, returning
> NoSuchObjectException
> Line 26763: 2019-10-28T21:09:52,936 WARN [pool-6-thread-5]
> metastore.ObjectStore: Failed to get database hive.compdb, returning
> NoSuchObjectException
> Line 26778: 2019-10-28T21:09:52,939 WARN [pool-6-thread-5]
> metastore.ObjectStore: Failed to get database hive.compdb, returning
> NoSuchObjectException
> Line 26963: 2019-10-28T21:09:53,273 WARN [pool-6-thread-5]
> metastore.ObjectStore: Failed to get database hive.db1, returning
> NoSuchObjectException
> Line 26978: 2019-10-28T21:09:53,276 WARN [pool-6-thread-5]
> metastore.ObjectStore: Failed to get database hive.db2, returning
> NoSuchObjectException
> Line 26986: 2019-10-28T21:09:53,277 WARN [pool-6-thread-5]
> metastore.ObjectStore: Failed to get database hive.db1, returning
> NoSuchObjectException
> Line 27018: 2019-10-28T21:09:53,300 WARN [pool-6-thread-5]
> metastore.ObjectStore: Failed to get database hive.db2, returning
> NoSuchObjectException
> {code}
> This is a superfluous log message. It might be pretty common for a database
> to not exists if, for example, a user fat-fingers the name of the database.
> The code also has the bad habit of log-and-throw. Just log or throw, not
> both.
> Since I'm looking at this class, touch up some of the other logging as well.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (HIVE-22522) llap doesn't work using complex join operation
[ https://issues.apache.org/jira/browse/HIVE-22522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16979092#comment-16979092 ] lv haiyang edited comment on HIVE-22522 at 11/25/19 4:17 AM: - Oops! when i try again,The error occurred： failed due to:INIT_FAILURE, Fail to create InputInitializerManager, org.apache.tez.dag.api.TezReflectionException: Unable to instantiate class with 1 arguments: org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator @ was (Author: oceanhaiyang): Oops! when i try again,The error occurred： failed due to:INIT_FAILURE, Fail to create InputInitializerManager, org.apache.tez.dag.api.TezReflectionException: Unable to instantiate class with 1 arguments: org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator > llap doesn't work using complex join operation > -- > > Key: HIVE-22522 > URL: https://issues.apache.org/jira/browse/HIVE-22522 > Project: Hive > Issue Type: Bug > Components: llap >Affects Versions: 3.1.1 >Reporter: lv haiyang >Priority: Major > > ERROR : FAILED: Execution Error, return code 2 from > org.apache.hadoop.hive.ql.exec.tez.TezTask. > Dag received [DAG_TERMINATE, SERVICE_PLUGIN_ERROR] in RUNNING state. > Error reported by TaskScheduler [[2:LLAP]][SERVICE_UNAVAILABLE] > No LLAP Daemons are runningVertex killed, vertexName=Reducer 3, > vertexId=vertex_1574126686177_0029_47_08, > diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not > succeed due to DAG_TERMINATED, > failedTasks:0 killedTasks:1, Vertex vertex_1574126686177_0029_47_08 [Reducer > 3] killed/failed due to: > DAG_TERMINATED]Vertex killed, vertexName=Map 1, > vertexId=vertex_1574126686177_0029_47_05, > diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not > succeed due to DAG_TERMINATED, > failedTasks:0 killedTasks:23, Vertex vertex_1574126686177_0029_47_05 [Map 1] > killed/failed due to: > DAG_TERMINATED]Vertex killed, vertexName=Reducer 2, > vertexId=vertex_1574126686177_0029_47_07, > diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not > succeed due to DAG_TERMINATED, > failedTasks:0 killedTasks:68, Vertex vertex_1574126686177_0029_47_07 > [Reducer 2] killed/failed due to: > DAG_TERMINATED]Vertex killed, vertexName=Reducer 4, > vertexId=vertex_1574126686177_0029_47_06, > diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not > succeed due to DAG_TERMINATED, > failedTasks:0 killedTasks:72, Vertex vertex_1574126686177_0029_47_06 > [Reducer 4] killed/failed due to: > DAG_TERMINATED]DAG did not succeed due to SERVICE_PLUGIN_ERROR. > failedVertices:0 killedVertices:4 > INFO : Completed executing > command(queryId=hive_20191120101841_c7d177d8-28bb-48f8-a14f-eb65fc3b); > Time taken: 557.077 seconds > Error: Error while processing statement: FAILED: Execution Error, > return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. > Dag received [DAG_TERMINATE, SERVICE_PLUGIN_ERROR] in RUNNING state. > Error reported by TaskScheduler [[2:LLAP]][SERVICE_UNAVAILABLE] > No LLAP Daemons are runningVertex killed, vertexName=Reducer 3, > vertexId=vertex_1574126686177_0029_47_08, > diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not > succeed due to DAG_TERMINATED, > failedTasks:0 killedTasks:1, Vertex vertex_1574126686177_0029_47_08 [Reducer > 3] killed/failed due to: > DAG_TERMINATED]Vertex killed, vertexName=Map 1, > vertexId=vertex_1574126686177_0029_47_05, > diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not > succeed due to DAG_TERMINATED, > failedTasks:0 killedTasks:23, Vertex vertex_1574126686177_0029_47_05 [Map 1] > killed/failed due to: > DAG_TERMINATED]Vertex killed, vertexName=Reducer 2, > vertexId=vertex_1574126686177_0029_47_07, > diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not > succeed due to DAG_TERMINATED, > failedTasks:0 killedTasks:68, Vertex vertex_1574126686177_0029_47_07 > [Reducer 2] killed/failed due to: > DAG_TERMINATED]Vertex killed, vertexName=Reducer 4, > vertexId=vertex_1574126686177_0029_47_06, > diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not > succeed due to DAG_TERMINATED, > failedTasks:0 killedTasks:72, Vertex vertex_1574126686177_0029_47_06 > [Reducer 4] killed/failed due to: > DAG_TERMINATED]DAG did not succeed due to SERVICE_PLUGIN_ERROR. > failedVertices:0 killedVertices: > 4 (state=08S01,code=2) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (HIVE-22522) llap doesn't work using complex join operation
[ https://issues.apache.org/jira/browse/HIVE-22522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16979092#comment-16979092 ] lv haiyang edited comment on HIVE-22522 at 11/25/19 4:17 AM: - Oops! when i try again,The error occurred： failed due to:INIT_FAILURE, Fail to create InputInitializerManager, org.apache.tez.dag.api.TezReflectionException: Unable to instantiate class with 1 arguments: org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator [~gopalv] was (Author: oceanhaiyang): Oops! when i try again,The error occurred： failed due to:INIT_FAILURE, Fail to create InputInitializerManager, org.apache.tez.dag.api.TezReflectionException: Unable to instantiate class with 1 arguments: org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator @ > llap doesn't work using complex join operation > -- > > Key: HIVE-22522 > URL: https://issues.apache.org/jira/browse/HIVE-22522 > Project: Hive > Issue Type: Bug > Components: llap >Affects Versions: 3.1.1 >Reporter: lv haiyang >Priority: Major > > ERROR : FAILED: Execution Error, return code 2 from > org.apache.hadoop.hive.ql.exec.tez.TezTask. > Dag received [DAG_TERMINATE, SERVICE_PLUGIN_ERROR] in RUNNING state. > Error reported by TaskScheduler [[2:LLAP]][SERVICE_UNAVAILABLE] > No LLAP Daemons are runningVertex killed, vertexName=Reducer 3, > vertexId=vertex_1574126686177_0029_47_08, > diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not > succeed due to DAG_TERMINATED, > failedTasks:0 killedTasks:1, Vertex vertex_1574126686177_0029_47_08 [Reducer > 3] killed/failed due to: > DAG_TERMINATED]Vertex killed, vertexName=Map 1, > vertexId=vertex_1574126686177_0029_47_05, > diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not > succeed due to DAG_TERMINATED, > failedTasks:0 killedTasks:23, Vertex vertex_1574126686177_0029_47_05 [Map 1] > killed/failed due to: > DAG_TERMINATED]Vertex killed, vertexName=Reducer 2, > vertexId=vertex_1574126686177_0029_47_07, > diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not > succeed due to DAG_TERMINATED, > failedTasks:0 killedTasks:68, Vertex vertex_1574126686177_0029_47_07 > [Reducer 2] killed/failed due to: > DAG_TERMINATED]Vertex killed, vertexName=Reducer 4, > vertexId=vertex_1574126686177_0029_47_06, > diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not > succeed due to DAG_TERMINATED, > failedTasks:0 killedTasks:72, Vertex vertex_1574126686177_0029_47_06 > [Reducer 4] killed/failed due to: > DAG_TERMINATED]DAG did not succeed due to SERVICE_PLUGIN_ERROR. > failedVertices:0 killedVertices:4 > INFO : Completed executing > command(queryId=hive_20191120101841_c7d177d8-28bb-48f8-a14f-eb65fc3b); > Time taken: 557.077 seconds > Error: Error while processing statement: FAILED: Execution Error, > return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. > Dag received [DAG_TERMINATE, SERVICE_PLUGIN_ERROR] in RUNNING state. > Error reported by TaskScheduler [[2:LLAP]][SERVICE_UNAVAILABLE] > No LLAP Daemons are runningVertex killed, vertexName=Reducer 3, > vertexId=vertex_1574126686177_0029_47_08, > diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not > succeed due to DAG_TERMINATED, > failedTasks:0 killedTasks:1, Vertex vertex_1574126686177_0029_47_08 [Reducer > 3] killed/failed due to: > DAG_TERMINATED]Vertex killed, vertexName=Map 1, > vertexId=vertex_1574126686177_0029_47_05, > diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not > succeed due to DAG_TERMINATED, > failedTasks:0 killedTasks:23, Vertex vertex_1574126686177_0029_47_05 [Map 1] > killed/failed due to: > DAG_TERMINATED]Vertex killed, vertexName=Reducer 2, > vertexId=vertex_1574126686177_0029_47_07, > diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not > succeed due to DAG_TERMINATED, > failedTasks:0 killedTasks:68, Vertex vertex_1574126686177_0029_47_07 > [Reducer 2] killed/failed due to: > DAG_TERMINATED]Vertex killed, vertexName=Reducer 4, > vertexId=vertex_1574126686177_0029_47_06, > diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not > succeed due to DAG_TERMINATED, > failedTasks:0 killedTasks:72, Vertex vertex_1574126686177_0029_47_06 > [Reducer 4] killed/failed due to: > DAG_TERMINATED]DAG did not succeed due to SERVICE_PLUGIN_ERROR. > failedVertices:0 killedVertices: > 4 (state=08S01,code=2) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22526) Extract Compiler from Driver
[
https://issues.apache.org/jira/browse/HIVE-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16981283#comment-16981283
]
Hive QA commented on HIVE-22526:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
26s{color} | {color:blue} ql in master has 1539 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} ql: The patch generated 0 new + 33 unchanged - 5
fixed = 33 total (was 38) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
46s{color} | {color:red} ql generated 2 new + 1536 unchanged - 3 fixed = 1538
total (was 1539) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 28m 7s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Found reliance on default encoding in
org.apache.hadoop.hive.ql.Compiler.getExplainOutput(BaseSemanticAnalyzer,
QueryPlan, ASTNode):in
org.apache.hadoop.hive.ql.Compiler.getExplainOutput(BaseSemanticAnalyzer,
QueryPlan, ASTNode): java.io.ByteArrayOutputStream.toString() At
Compiler.java:[line 521] |
| | Found reliance on default encoding in
org.apache.hadoop.hive.ql.Compiler.getExplainOutput(BaseSemanticAnalyzer,
QueryPlan, ASTNode):in
org.apache.hadoop.hive.ql.Compiler.getExplainOutput(BaseSemanticAnalyzer,
QueryPlan, ASTNode): new java.io.PrintStream(OutputStream) At
Compiler.java:[line 504] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19589/dev-support/hive-personality.sh
|
| git revision | master / a70408a |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19589/yetus/new-findbugs-ql.html
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19589/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Extract Compiler from Driver
>
>
> Key: HIVE-22526
> URL: https://issues.apache.org/jira/browse/HIVE-22526
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22526.01.patch, HIVE-22526.02.patch,
> HIVE-22526.03.patch, HIVE-22526.04.patch, HIVE-22526.05.patch
>
>
> The Driver class contains ~600 lines of code responsible for compiling the
> command. That means that from the command String a Plan needs to be created,
> and also a transaction needs to be started (in most of the cases). This is a
> thing done by the compile function, which has a lot of sub functions to help
[jira] [Updated] (HIVE-22532) PTFPPD may push limit incorrectly through Rank/DenseRank function
[ https://issues.apache.org/jira/browse/HIVE-22532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-22532: --- Attachment: HIVE-22532.03.patch > PTFPPD may push limit incorrectly through Rank/DenseRank function > - > > Key: HIVE-22532 > URL: https://issues.apache.org/jira/browse/HIVE-22532 > Project: Hive > Issue Type: Bug > Components: Physical Optimizer >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22532.01.patch, HIVE-22532.02.patch, > HIVE-22532.03.patch, HIVE-22532.03.patch, HIVE-22532.03.patch, > HIVE-22532.patch > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22532) PTFPPD may push limit incorrectly through Rank/DenseRank function
[
https://issues.apache.org/jira/browse/HIVE-22532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16981274#comment-16981274
]
Hive QA commented on HIVE-22532:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12986620/HIVE-22532.03.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17718 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[stats_part] (batchId=53)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19588/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19588/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19588/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12986620 - PreCommit-HIVE-Build
> PTFPPD may push limit incorrectly through Rank/DenseRank function
> -
>
> Key: HIVE-22532
> URL: https://issues.apache.org/jira/browse/HIVE-22532
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22532.01.patch, HIVE-22532.02.patch,
> HIVE-22532.03.patch, HIVE-22532.03.patch, HIVE-22532.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22532) PTFPPD may push limit incorrectly through Rank/DenseRank function
[
https://issues.apache.org/jira/browse/HIVE-22532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16981261#comment-16981261
]
Hive QA commented on HIVE-22532:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
15s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
49s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
28s{color} | {color:blue} ql in master has 1539 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
29s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 28m 53s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19588/dev-support/hive-personality.sh
|
| git revision | master / a70408a |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql itests U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19588/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> PTFPPD may push limit incorrectly through Rank/DenseRank function
> -
>
> Key: HIVE-22532
> URL: https://issues.apache.org/jira/browse/HIVE-22532
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22532.01.patch, HIVE-22532.02.patch,
> HIVE-22532.03.patch, HIVE-22532.03.patch, HIVE-22532.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-21917) COMPLETED_TXN_COMPONENTS table is never cleaned up unless Compactor runs
[
https://issues.apache.org/jira/browse/HIVE-21917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16981246#comment-16981246
]
Hive QA commented on HIVE-21917:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12986619/HIVE-21917.6.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17717 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19587/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19587/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19587/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12986619 - PreCommit-HIVE-Build
> COMPLETED_TXN_COMPONENTS table is never cleaned up unless Compactor runs
>
>
> Key: HIVE-21917
> URL: https://issues.apache.org/jira/browse/HIVE-21917
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 3.1.0, 3.1.1
>Reporter: Craig Condit
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-21917.1.patch, HIVE-21917.2.patch,
> HIVE-21917.3.patch, HIVE-21917.4.patch, HIVE-21917.5.patch, HIVE-21917.6.patch
>
>
> The Initiator thread in the metastore repeatedly loops over entries in the
> COMPLETED_TXN_COMPONENTS table to determine which partitions / tables might
> need to be compacted. However, entries are never removed from this table
> except by a completed Compactor run.
> In a cluster where most tables / partitions are write-once read-many, this
> results in stale entries in this table never being cleaned up. In a small
> test cluster, we have observed approximately 45k entries in this table
> (virtually equal to the number of partitions in the cluster) while < 100 of
> these tables have delta files at all. Since most of the tables will never get
> enough writes to trigger a compaction (and in fact have only ever been
> written to once), the initiator thread keeps trying to evaluate them on every
> loop.
> On this test cluster, it takes approximately 10 minutes to loop through all
> the entries and results in severe performance degradation on metastore
> operations. With the default run timing of 5 minutes, the initiator basically
> never stops running.
> On a production cluster with 2M partitions, this would be a non-starter.
> The initiator thread should proactively remove entries from
> COMPLETED_TXN_COMPONENTS when it determines that a compaction is not needed,
> so that they are not evaluated again on the next loop.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-21917) COMPLETED_TXN_COMPONENTS table is never cleaned up unless Compactor runs
[
https://issues.apache.org/jira/browse/HIVE-21917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16981238#comment-16981238
]
Hive QA commented on HIVE-21917:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
26s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
13s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
25s{color} | {color:blue} standalone-metastore/metastore-server in master has
178 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
33s{color} | {color:blue} ql in master has 1539 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
28s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
30s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
43s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
29s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 31 new + 627 unchanged - 1 fixed = 658 total (was 628) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
30s{color} | {color:red} standalone-metastore/metastore-server generated 1 new
+ 178 unchanged - 0 fixed = 179 total (was 178) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 35m 39s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:standalone-metastore/metastore-server |
| |
org.apache.hadoop.hive.metastore.txn.CompactionTxnHandler.findPotentialCompactions(int,
long) passes a nonconstant String to an execute or addBatch method on an SQL
statement At CompactionTxnHandler.java:String to an execute or addBatch method
on an SQL statement At CompactionTxnHandler.java:[line 94] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19587/dev-support/hive-personality.sh
|
| git revision | master / a70408a |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19587/yetus/diff-checkstyle-standalone-metastore_metastore-server.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19587/yetus/new-findbugs-standalone-metastore_metastore-server.html
|
| modules | C: standalone-metastore/metastore-server ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19587/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> COMPLETED_TXN_COMPONENTS table is never cleaned up unless Compactor runs
>
>
> Key: HIVE-21917
> URL: https://issues.apache.org/jira/browse/HIVE-21917
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>
[jira] [Commented] (HIVE-21737) Upgrade Avro to version 1.9.1
[
https://issues.apache.org/jira/browse/HIVE-21737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16981224#comment-16981224
]
Hive QA commented on HIVE-21737:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12986618/0001-HIVE-21737-Bump-Apache-Avro-to-1.9.1.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19586/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19586/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19586/
Messages:
{noformat}
This message was trimmed, see log for full details
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/eclipse/jetty/jetty-runner/9.3.27.v20190418/jetty-runner-9.3.27.v20190418.jar(javax/servlet/DispatcherType.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/eclipse/jetty/jetty-runner/9.3.27.v20190418/jetty-runner-9.3.27.v20190418.jar(javax/servlet/Filter.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/eclipse/jetty/jetty-runner/9.3.27.v20190418/jetty-runner-9.3.27.v20190418.jar(javax/servlet/FilterChain.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/eclipse/jetty/jetty-runner/9.3.27.v20190418/jetty-runner-9.3.27.v20190418.jar(javax/servlet/FilterConfig.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/eclipse/jetty/jetty-runner/9.3.27.v20190418/jetty-runner-9.3.27.v20190418.jar(javax/servlet/ServletException.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/eclipse/jetty/jetty-runner/9.3.27.v20190418/jetty-runner-9.3.27.v20190418.jar(javax/servlet/ServletRequest.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/eclipse/jetty/jetty-runner/9.3.27.v20190418/jetty-runner-9.3.27.v20190418.jar(javax/servlet/ServletResponse.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/eclipse/jetty/jetty-runner/9.3.27.v20190418/jetty-runner-9.3.27.v20190418.jar(javax/servlet/annotation/WebFilter.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/eclipse/jetty/jetty-runner/9.3.27.v20190418/jetty-runner-9.3.27.v20190418.jar(javax/servlet/http/HttpServletRequest.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/eclipse/jetty/jetty-runner/9.3.27.v20190418/jetty-runner-9.3.27.v20190418.jar(javax/servlet/http/HttpServletResponse.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/apache-github-source-source/classification/target/hive-classification-4.0.0-SNAPSHOT.jar(org/apache/hadoop/hive/common/classification/InterfaceAudience$LimitedPrivate.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/apache-github-source-source/classification/target/hive-classification-4.0.0-SNAPSHOT.jar(org/apache/hadoop/hive/common/classification/InterfaceStability$Unstable.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/io/ByteArrayOutputStream.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/io/OutputStream.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/io/Closeable.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/lang/AutoCloseable.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/io/Flushable.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(javax/xml/bind/annotation/XmlRootElement.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/apache/commons/commons-exec/1.1/commons-exec-1.1.jar(org/apache/commons/exec/ExecuteException.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/security/PrivilegedExceptionAction.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/util/concurrent/ExecutionException.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/util/concurrent/TimeoutException.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/apache/hadoop/hadoop-common/3.1.0/hadoop-common-3.1.0.jar(org/apache/hadoop/fs/FileSystem.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/apache-github-source-source/shims/common/target/hive-shims-common-4.0.0-SNAPSHOT.jar(org/apache/hadoop/hive/shims/HadoopShimsSecure.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/apache-github-source-source/shims/common/target/hive-shims-common-4.0.0-SNAPSHOT.jar(org/apache/hadoop/hive/shims/ShimLoader.class)]]
[loading
[jira] [Commented] (HIVE-22506) Read-only transactions feature flag
[
https://issues.apache.org/jira/browse/HIVE-22506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16981221#comment-16981221
]
Hive QA commented on HIVE-22506:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12986617/HIVE-22506.5.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17736 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19585/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19585/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19585/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12986617 - PreCommit-HIVE-Build
> Read-only transactions feature flag
> ---
>
> Key: HIVE-22506
> URL: https://issues.apache.org/jira/browse/HIVE-22506
> Project: Hive
> Issue Type: Improvement
> Components: Transactions
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22506.1.patch, HIVE-22506.2.patch,
> HIVE-22506.3.patch, HIVE-22506.4.patch, HIVE-22506.5.patch
>
>
> Introduce a feature flag, so that read-only transaction functionality could
> be conditionally turned on/off.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22506) Read-only transactions feature flag
[
https://issues.apache.org/jira/browse/HIVE-22506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16981216#comment-16981216
]
Hive QA commented on HIVE-22506:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
18s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
42s{color} | {color:blue} common in master has 65 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
48s{color} | {color:blue} ql in master has 1539 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
24s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
34s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 34m 2s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19585/dev-support/hive-personality.sh
|
| git revision | master / a70408a |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19585/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Read-only transactions feature flag
> ---
>
> Key: HIVE-22506
> URL: https://issues.apache.org/jira/browse/HIVE-22506
> Project: Hive
> Issue Type: Improvement
> Components: Transactions
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22506.1.patch, HIVE-22506.2.patch,
> HIVE-22506.3.patch, HIVE-22506.4.patch, HIVE-22506.5.patch
>
>
> Introduce a feature flag, so that read-only transaction functionality could
> be conditionally turned on/off.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22526) Extract Compiler from Driver
[ https://issues.apache.org/jira/browse/HIVE-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22526: -- Attachment: HIVE-22526.05.patch > Extract Compiler from Driver > > > Key: HIVE-22526 > URL: https://issues.apache.org/jira/browse/HIVE-22526 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22526.01.patch, HIVE-22526.02.patch, > HIVE-22526.03.patch, HIVE-22526.04.patch, HIVE-22526.05.patch > > > The Driver class contains ~600 lines of code responsible for compiling the > command. That means that from the command String a Plan needs to be created, > and also a transaction needs to be started (in most of the cases). This is a > thing done by the compile function, which has a lot of sub functions to help > this task, while itself is also really big. All these codes should be put > into a separate class, where it can do it's job without getting mixed with > the other codes in the Driver. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22526) Extract Compiler from Driver
[ https://issues.apache.org/jira/browse/HIVE-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22526: -- Attachment: (was: HIVE-22526.05.patch) > Extract Compiler from Driver > > > Key: HIVE-22526 > URL: https://issues.apache.org/jira/browse/HIVE-22526 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22526.01.patch, HIVE-22526.02.patch, > HIVE-22526.03.patch, HIVE-22526.04.patch, HIVE-22526.05.patch > > > The Driver class contains ~600 lines of code responsible for compiling the > command. That means that from the command String a Plan needs to be created, > and also a transaction needs to be started (in most of the cases). This is a > thing done by the compile function, which has a lot of sub functions to help > this task, while itself is also really big. All these codes should be put > into a separate class, where it can do it's job without getting mixed with > the other codes in the Driver. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22526) Extract Compiler from Driver
[
https://issues.apache.org/jira/browse/HIVE-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16981203#comment-16981203
]
Hive QA commented on HIVE-22526:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12986616/HIVE-22526.05.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 17717 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[uber_reduce]
(batchId=194)
org.apache.hive.service.server.TestHS2HttpServer.testApiServletActiveSessions
(batchId=240)
org.apache.hive.service.server.TestHS2HttpServer.testApiServletHistoricalQueries
(batchId=240)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19584/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19584/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19584/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12986616 - PreCommit-HIVE-Build
> Extract Compiler from Driver
>
>
> Key: HIVE-22526
> URL: https://issues.apache.org/jira/browse/HIVE-22526
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22526.01.patch, HIVE-22526.02.patch,
> HIVE-22526.03.patch, HIVE-22526.04.patch, HIVE-22526.05.patch
>
>
> The Driver class contains ~600 lines of code responsible for compiling the
> command. That means that from the command String a Plan needs to be created,
> and also a transaction needs to be started (in most of the cases). This is a
> thing done by the compile function, which has a lot of sub functions to help
> this task, while itself is also really big. All these codes should be put
> into a separate class, where it can do it's job without getting mixed with
> the other codes in the Driver.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22532) PTFPPD may push limit incorrectly through Rank/DenseRank function
[ https://issues.apache.org/jira/browse/HIVE-22532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-22532: --- Attachment: HIVE-22532.03.patch > PTFPPD may push limit incorrectly through Rank/DenseRank function > - > > Key: HIVE-22532 > URL: https://issues.apache.org/jira/browse/HIVE-22532 > Project: Hive > Issue Type: Bug > Components: Physical Optimizer >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22532.01.patch, HIVE-22532.02.patch, > HIVE-22532.03.patch, HIVE-22532.03.patch, HIVE-22532.patch > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-21917) COMPLETED_TXN_COMPONENTS table is never cleaned up unless Compactor runs
[ https://issues.apache.org/jira/browse/HIVE-21917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denys Kuzmenko updated HIVE-21917: -- Attachment: HIVE-21917.6.patch > COMPLETED_TXN_COMPONENTS table is never cleaned up unless Compactor runs > > > Key: HIVE-21917 > URL: https://issues.apache.org/jira/browse/HIVE-21917 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 3.1.0, 3.1.1 >Reporter: Craig Condit >Assignee: Denys Kuzmenko >Priority: Major > Attachments: HIVE-21917.1.patch, HIVE-21917.2.patch, > HIVE-21917.3.patch, HIVE-21917.4.patch, HIVE-21917.5.patch, HIVE-21917.6.patch > > > The Initiator thread in the metastore repeatedly loops over entries in the > COMPLETED_TXN_COMPONENTS table to determine which partitions / tables might > need to be compacted. However, entries are never removed from this table > except by a completed Compactor run. > In a cluster where most tables / partitions are write-once read-many, this > results in stale entries in this table never being cleaned up. In a small > test cluster, we have observed approximately 45k entries in this table > (virtually equal to the number of partitions in the cluster) while < 100 of > these tables have delta files at all. Since most of the tables will never get > enough writes to trigger a compaction (and in fact have only ever been > written to once), the initiator thread keeps trying to evaluate them on every > loop. > On this test cluster, it takes approximately 10 minutes to loop through all > the entries and results in severe performance degradation on metastore > operations. With the default run timing of 5 minutes, the initiator basically > never stops running. > On a production cluster with 2M partitions, this would be a non-starter. > The initiator thread should proactively remove entries from > COMPLETED_TXN_COMPONENTS when it determines that a compaction is not needed, > so that they are not evaluated again on the next loop. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22526) Extract Compiler from Driver
[
https://issues.apache.org/jira/browse/HIVE-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16981196#comment-16981196
]
Hive QA commented on HIVE-22526:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
46s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
51s{color} | {color:blue} ql in master has 1539 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
10s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s{color} | {color:green} ql: The patch generated 0 new + 33 unchanged - 5
fixed = 33 total (was 38) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 5m
2s{color} | {color:red} ql generated 2 new + 1536 unchanged - 3 fixed = 1538
total (was 1539) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 29m 3s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Found reliance on default encoding in
org.apache.hadoop.hive.ql.Compiler.getExplainOutput(BaseSemanticAnalyzer,
QueryPlan, ASTNode):in
org.apache.hadoop.hive.ql.Compiler.getExplainOutput(BaseSemanticAnalyzer,
QueryPlan, ASTNode): java.io.ByteArrayOutputStream.toString() At
Compiler.java:[line 521] |
| | Found reliance on default encoding in
org.apache.hadoop.hive.ql.Compiler.getExplainOutput(BaseSemanticAnalyzer,
QueryPlan, ASTNode):in
org.apache.hadoop.hive.ql.Compiler.getExplainOutput(BaseSemanticAnalyzer,
QueryPlan, ASTNode): new java.io.PrintStream(OutputStream) At
Compiler.java:[line 504] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19584/dev-support/hive-personality.sh
|
| git revision | master / a70408a |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19584/yetus/new-findbugs-ql.html
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19584/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Extract Compiler from Driver
>
>
> Key: HIVE-22526
> URL: https://issues.apache.org/jira/browse/HIVE-22526
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22526.01.patch, HIVE-22526.02.patch,
> HIVE-22526.03.patch, HIVE-22526.04.patch, HIVE-22526.05.patch
>
>
> The Driver class contains ~600 lines of code responsible for compiling the
> command. That means that from the command String a Plan needs to be created,
> and also a transaction needs to be started (in most of the cases). This is a
> thing done by the compile function, which has a lot of sub functions to help
[jira] [Updated] (HIVE-21737) Upgrade Avro to version 1.9.1
[ https://issues.apache.org/jira/browse/HIVE-21737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated HIVE-21737: Attachment: 0001-HIVE-21737-Bump-Apache-Avro-to-1.9.1.patch Status: Patch Available (was: Open) > Upgrade Avro to version 1.9.1 > - > > Key: HIVE-21737 > URL: https://issues.apache.org/jira/browse/HIVE-21737 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Ismaël Mejía >Assignee: Fokko Driesprong >Priority: Major > Labels: pull-request-available > Attachments: 0001-HIVE-21737-Bump-Apache-Avro-to-1.9.1.patch > > Time Spent: 1h > Remaining Estimate: 0h > > Avro 1.9.0 was released recently. It brings a lot of fixes including a leaner > version of Avro without Jackson in the public API. Worth the update. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-21737) Upgrade Avro to version 1.9.1
[ https://issues.apache.org/jira/browse/HIVE-21737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated HIVE-21737: Status: Open (was: Patch Available) > Upgrade Avro to version 1.9.1 > - > > Key: HIVE-21737 > URL: https://issues.apache.org/jira/browse/HIVE-21737 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Ismaël Mejía >Assignee: Fokko Driesprong >Priority: Major > Labels: pull-request-available > Time Spent: 1h > Remaining Estimate: 0h > > Avro 1.9.0 was released recently. It brings a lot of fixes including a leaner > version of Avro without Jackson in the public API. Worth the update. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-21737) Upgrade Avro to version 1.9.1
[ https://issues.apache.org/jira/browse/HIVE-21737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated HIVE-21737: Attachment: (was: 0001-HIVE-21737-Bump-Apache-Avro-to-1.9.1.patch) > Upgrade Avro to version 1.9.1 > - > > Key: HIVE-21737 > URL: https://issues.apache.org/jira/browse/HIVE-21737 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Ismaël Mejía >Assignee: Fokko Driesprong >Priority: Major > Labels: pull-request-available > Time Spent: 1h > Remaining Estimate: 0h > > Avro 1.9.0 was released recently. It brings a lot of fixes including a leaner > version of Avro without Jackson in the public API. Worth the update. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22506) Read-only transactions feature flag
[ https://issues.apache.org/jira/browse/HIVE-22506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denys Kuzmenko updated HIVE-22506: -- Attachment: HIVE-22506.5.patch > Read-only transactions feature flag > --- > > Key: HIVE-22506 > URL: https://issues.apache.org/jira/browse/HIVE-22506 > Project: Hive > Issue Type: Improvement > Components: Transactions >Reporter: Denys Kuzmenko >Assignee: Denys Kuzmenko >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22506.1.patch, HIVE-22506.2.patch, > HIVE-22506.3.patch, HIVE-22506.4.patch, HIVE-22506.5.patch > > > Introduce a feature flag, so that read-only transaction functionality could > be conditionally turned on/off. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22526) Extract Compiler from Driver
[ https://issues.apache.org/jira/browse/HIVE-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22526: -- Attachment: HIVE-22526.05.patch > Extract Compiler from Driver > > > Key: HIVE-22526 > URL: https://issues.apache.org/jira/browse/HIVE-22526 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22526.01.patch, HIVE-22526.02.patch, > HIVE-22526.03.patch, HIVE-22526.04.patch, HIVE-22526.05.patch > > > The Driver class contains ~600 lines of code responsible for compiling the > command. That means that from the command String a Plan needs to be created, > and also a transaction needs to be started (in most of the cases). This is a > thing done by the compile function, which has a lot of sub functions to help > this task, while itself is also really big. All these codes should be put > into a separate class, where it can do it's job without getting mixed with > the other codes in the Driver. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22532) PTFPPD may push limit incorrectly through Rank/DenseRank function
[
https://issues.apache.org/jira/browse/HIVE-22532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16981180#comment-16981180
]
Hive QA commented on HIVE-22532:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12986609/HIVE-22532.03.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17718 tests
executed
*Failed tests:*
{noformat}
org.apache.hive.service.server.TestHS2HttpServerPamConfiguration.testPamCorrectConfiguration
(batchId=240)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19583/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19583/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19583/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12986609 - PreCommit-HIVE-Build
> PTFPPD may push limit incorrectly through Rank/DenseRank function
> -
>
> Key: HIVE-22532
> URL: https://issues.apache.org/jira/browse/HIVE-22532
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22532.01.patch, HIVE-22532.02.patch,
> HIVE-22532.03.patch, HIVE-22532.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22532) PTFPPD may push limit incorrectly through Rank/DenseRank function
[
https://issues.apache.org/jira/browse/HIVE-22532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16981165#comment-16981165
]
Hive QA commented on HIVE-22532:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
32s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
24s{color} | {color:blue} ql in master has 1539 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
6s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
30s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 27m 42s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19583/dev-support/hive-personality.sh
|
| git revision | master / a70408a |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql itests U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19583/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> PTFPPD may push limit incorrectly through Rank/DenseRank function
> -
>
> Key: HIVE-22532
> URL: https://issues.apache.org/jira/browse/HIVE-22532
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22532.01.patch, HIVE-22532.02.patch,
> HIVE-22532.03.patch, HIVE-22532.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22532) PTFPPD may push limit incorrectly through Rank/DenseRank function
[ https://issues.apache.org/jira/browse/HIVE-22532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-22532: --- Attachment: HIVE-22532.03.patch > PTFPPD may push limit incorrectly through Rank/DenseRank function > - > > Key: HIVE-22532 > URL: https://issues.apache.org/jira/browse/HIVE-22532 > Project: Hive > Issue Type: Bug > Components: Physical Optimizer >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22532.01.patch, HIVE-22532.02.patch, > HIVE-22532.03.patch, HIVE-22532.patch > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22532) PTFPPD may push limit incorrectly through Rank/DenseRank function
[
https://issues.apache.org/jira/browse/HIVE-22532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16981147#comment-16981147
]
Hive QA commented on HIVE-22532:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12986607/HIVE-22532.02.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 17718 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[stats_part] (batchId=53)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb_schq]
(batchId=177)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query44]
(batchId=302)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19582/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19582/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19582/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12986607 - PreCommit-HIVE-Build
> PTFPPD may push limit incorrectly through Rank/DenseRank function
> -
>
> Key: HIVE-22532
> URL: https://issues.apache.org/jira/browse/HIVE-22532
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22532.01.patch, HIVE-22532.02.patch,
> HIVE-22532.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22532) PTFPPD may push limit incorrectly through Rank/DenseRank function
[
https://issues.apache.org/jira/browse/HIVE-22532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16981143#comment-16981143
]
Hive QA commented on HIVE-22532:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
23s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
31s{color} | {color:blue} ql in master has 1539 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
6s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
29s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 28m 0s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19582/dev-support/hive-personality.sh
|
| git revision | master / a70408a |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql itests U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19582/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> PTFPPD may push limit incorrectly through Rank/DenseRank function
> -
>
> Key: HIVE-22532
> URL: https://issues.apache.org/jira/browse/HIVE-22532
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22532.01.patch, HIVE-22532.02.patch,
> HIVE-22532.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22066) Upgrade Apache parent POM to version 21 CLEAR LIBRARY CACHE
[
https://issues.apache.org/jira/browse/HIVE-22066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16981132#comment-16981132
]
David Mollitor commented on HIVE-22066:
---
{code:none}
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-resources-plugin:3.1.0:resources
(default-resources) on project hive-llap-server:
/home/hiveptest/35.184.204.95-hiveptest-0/apache-github-source-source/llap-server/target/classes/hive-webapps/llap/css/bootstrap.min.css
-> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please
read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
{code}
> Upgrade Apache parent POM to version 21 CLEAR LIBRARY CACHE
> ---
>
> Key: HIVE-22066
> URL: https://issues.apache.org/jira/browse/HIVE-22066
> Project: Hive
> Issue Type: Improvement
>Reporter: Jesus Camacho Rodriguez
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-22066.2.patch, HIVE-22066.3.patch,
> HIVE-22066.4.patch, HIVE-22066.5.patch, HIVE-22066.6.patch,
> HIVE-22066.6.patch, HIVE-22066.patch, HIVE-22066.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22532) PTFPPD may push limit incorrectly through Rank/DenseRank function
[ https://issues.apache.org/jira/browse/HIVE-22532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-22532: --- Attachment: HIVE-22532.02.patch > PTFPPD may push limit incorrectly through Rank/DenseRank function > - > > Key: HIVE-22532 > URL: https://issues.apache.org/jira/browse/HIVE-22532 > Project: Hive > Issue Type: Bug > Components: Physical Optimizer >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22532.01.patch, HIVE-22532.02.patch, > HIVE-22532.patch > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22066) Upgrade Apache parent POM to version 21 CLEAR LIBRARY CACHE
[
https://issues.apache.org/jira/browse/HIVE-22066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16981131#comment-16981131
]
David Mollitor commented on HIVE-22066:
---
{code:none}
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.7.0:compile (default-compile)
on project kafka-handler: Compilation failure
[ERROR] cannot access
org.apache.hadoop.hive.common.classification.InterfaceAudience.Public
[ERROR] class file for
org.apache.hadoop.hive.common.classification.InterfaceAudience$Public not found
[ERROR] -> [Help 1]
{code}
> Upgrade Apache parent POM to version 21 CLEAR LIBRARY CACHE
> ---
>
> Key: HIVE-22066
> URL: https://issues.apache.org/jira/browse/HIVE-22066
> Project: Hive
> Issue Type: Improvement
>Reporter: Jesus Camacho Rodriguez
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-22066.2.patch, HIVE-22066.3.patch,
> HIVE-22066.4.patch, HIVE-22066.5.patch, HIVE-22066.6.patch,
> HIVE-22066.6.patch, HIVE-22066.patch, HIVE-22066.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22066) Upgrade Apache parent POM to version 21 CLEAR LIBRARY CACHE
[
https://issues.apache.org/jira/browse/HIVE-22066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16981130#comment-16981130
]
David Mollitor commented on HIVE-22066:
---
{code:none}
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.7.0:testCompile
(default-testCompile) on project hive-hbase-handler: Compilation failure
[ERROR] cannot access
org.apache.hadoop.hive.common.classification.InterfaceAudience
[ERROR] class file for
org.apache.hadoop.hive.common.classification.InterfaceAudience not found
[ERROR] -> [Help 1]
[ERROR]
{code}
> Upgrade Apache parent POM to version 21 CLEAR LIBRARY CACHE
> ---
>
> Key: HIVE-22066
> URL: https://issues.apache.org/jira/browse/HIVE-22066
> Project: Hive
> Issue Type: Improvement
>Reporter: Jesus Camacho Rodriguez
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-22066.2.patch, HIVE-22066.3.patch,
> HIVE-22066.4.patch, HIVE-22066.5.patch, HIVE-22066.6.patch,
> HIVE-22066.6.patch, HIVE-22066.patch, HIVE-22066.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22066) Upgrade Apache parent POM to version 21 CLEAR LIBRARY CACHE
[
https://issues.apache.org/jira/browse/HIVE-22066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16981129#comment-16981129
]
David Mollitor commented on HIVE-22066:
---
{code:none}
[ERROR] symbol: class AbstractSerDe
[ERROR] location: class org.apache.hadoop.hive.llap.LlapRowRecordReader
[ERROR]
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19566/llap-client/src/java/org/apache/hadoop/hive/llap/LlapRowRecordReader.java:[143,46]
cannot find symbol
[ERROR] symbol: class PrimitiveObjectInspector
[ERROR] location: class org.apache.hadoop.hive.llap.LlapRowRecordReader
[ERROR]
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19566/llap-client/src/java/org/apache/hadoop/hive/llap/LlapRowRecordReader.java:[157,42]
cannot find symbol
[ERROR] symbol: class ObjectInspector
[ERROR] location: class org.apache.hadoop.hive.llap.LlapRowRecordReader
{code}
> Upgrade Apache parent POM to version 21 CLEAR LIBRARY CACHE
> ---
>
> Key: HIVE-22066
> URL: https://issues.apache.org/jira/browse/HIVE-22066
> Project: Hive
> Issue Type: Improvement
>Reporter: Jesus Camacho Rodriguez
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-22066.2.patch, HIVE-22066.3.patch,
> HIVE-22066.4.patch, HIVE-22066.5.patch, HIVE-22066.6.patch,
> HIVE-22066.6.patch, HIVE-22066.patch, HIVE-22066.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22486) Send only accessed columns for masking policies request
[ https://issues.apache.org/jira/browse/HIVE-22486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-22486: --- Fix Version/s: 4.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to master. > Send only accessed columns for masking policies request > --- > > Key: HIVE-22486 > URL: https://issues.apache.org/jira/browse/HIVE-22486 > Project: Hive > Issue Type: Improvement > Components: CBO >Affects Versions: 4.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-22486.01.patch, HIVE-22486.02.patch, > HIVE-22486.03.patch, HIVE-22486.05.patch, HIVE-22486.06.patch, > HIVE-22486.patch > > Time Spent: 1h > Remaining Estimate: 0h > > Currently, we send all columns for masking request, even if they are not > accessed by the given query. We could send only those columns for which the > masking policy will be necessary. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-22486) Send only accessed columns for masking policies request
[ https://issues.apache.org/jira/browse/HIVE-22486?focusedWorklogId=348756=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-348756 ] ASF GitHub Bot logged work on HIVE-22486: - Author: ASF GitHub Bot Created on: 24/Nov/19 16:09 Start Date: 24/Nov/19 16:09 Worklog Time Spent: 10m Work Description: asfgit commented on pull request #848: HIVE-22486 URL: https://github.com/apache/hive/pull/848 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 348756) Time Spent: 1h (was: 50m) > Send only accessed columns for masking policies request > --- > > Key: HIVE-22486 > URL: https://issues.apache.org/jira/browse/HIVE-22486 > Project: Hive > Issue Type: Improvement > Components: CBO >Affects Versions: 4.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22486.01.patch, HIVE-22486.02.patch, > HIVE-22486.03.patch, HIVE-22486.05.patch, HIVE-22486.06.patch, > HIVE-22486.patch > > Time Spent: 1h > Remaining Estimate: 0h > > Currently, we send all columns for masking request, even if they are not > accessed by the given query. We could send only those columns for which the > masking policy will be necessary. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22066) Upgrade Apache parent POM to version 21 CLEAR LIBRARY CACHE
[
https://issues.apache.org/jira/browse/HIVE-22066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16981114#comment-16981114
]
David Mollitor commented on HIVE-22066:
---
{code:none}
main:
[INFO] Executed tasks
[INFO]
[INFO] --- maven-compiler-plugin:3.7.0:compile (default-compile) @ hive-hplsql
---
[INFO] Compiling 37 source files to
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19566/hplsql/target/classes
[INFO] -
[ERROR] COMPILATION ERROR :
[INFO] -
[ERROR] cannot access
org.apache.hadoop.hive.common.classification.InterfaceAudience
class file for org.apache.hadoop.hive.common.classification.InterfaceAudience
not found
{code}
> Upgrade Apache parent POM to version 21 CLEAR LIBRARY CACHE
> ---
>
> Key: HIVE-22066
> URL: https://issues.apache.org/jira/browse/HIVE-22066
> Project: Hive
> Issue Type: Improvement
>Reporter: Jesus Camacho Rodriguez
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-22066.2.patch, HIVE-22066.3.patch,
> HIVE-22066.4.patch, HIVE-22066.5.patch, HIVE-22066.6.patch,
> HIVE-22066.6.patch, HIVE-22066.patch, HIVE-22066.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22526) Extract Compiler from Driver
[
https://issues.apache.org/jira/browse/HIVE-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16981085#comment-16981085
]
Hive QA commented on HIVE-22526:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12986601/HIVE-22526.04.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 6 failed/errored test(s), 17717 tests
executed
*Failed tests:*
{noformat}
org.apache.hive.service.cli.operation.TestOperationLoggingAPIWithMr.testFetchResultsOfLogWithExecutionMode
(batchId=283)
org.apache.hive.service.cli.operation.TestOperationLoggingAPIWithMr.testFetchResultsOfLogWithPerformanceMode
(batchId=283)
org.apache.hive.service.cli.operation.TestOperationLoggingAPIWithMr.testFetchResultsOfLogWithVerboseMode
(batchId=283)
org.apache.hive.service.cli.operation.TestOperationLoggingAPIWithTez.testFetchResultsOfLogWithExecutionMode
(batchId=282)
org.apache.hive.service.cli.operation.TestOperationLoggingAPIWithTez.testFetchResultsOfLogWithPerformanceMode
(batchId=282)
org.apache.hive.service.cli.operation.TestOperationLoggingAPIWithTez.testFetchResultsOfLogWithVerboseMode
(batchId=282)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19581/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19581/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19581/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 6 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12986601 - PreCommit-HIVE-Build
> Extract Compiler from Driver
>
>
> Key: HIVE-22526
> URL: https://issues.apache.org/jira/browse/HIVE-22526
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22526.01.patch, HIVE-22526.02.patch,
> HIVE-22526.03.patch, HIVE-22526.04.patch
>
>
> The Driver class contains ~600 lines of code responsible for compiling the
> command. That means that from the command String a Plan needs to be created,
> and also a transaction needs to be started (in most of the cases). This is a
> thing done by the compile function, which has a lot of sub functions to help
> this task, while itself is also really big. All these codes should be put
> into a separate class, where it can do it's job without getting mixed with
> the other codes in the Driver.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22526) Extract Compiler from Driver
[
https://issues.apache.org/jira/browse/HIVE-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16981077#comment-16981077
]
Hive QA commented on HIVE-22526:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
36s{color} | {color:blue} ql in master has 1539 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
8s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s{color} | {color:green} ql: The patch generated 0 new + 33 unchanged - 5
fixed = 33 total (was 38) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
56s{color} | {color:red} ql generated 2 new + 1536 unchanged - 3 fixed = 1538
total (was 1539) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 28m 11s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Found reliance on default encoding in
org.apache.hadoop.hive.ql.Compiler.getExplainOutput(BaseSemanticAnalyzer,
QueryPlan, ASTNode):in
org.apache.hadoop.hive.ql.Compiler.getExplainOutput(BaseSemanticAnalyzer,
QueryPlan, ASTNode): java.io.ByteArrayOutputStream.toString() At
Compiler.java:[line 521] |
| | Found reliance on default encoding in
org.apache.hadoop.hive.ql.Compiler.getExplainOutput(BaseSemanticAnalyzer,
QueryPlan, ASTNode):in
org.apache.hadoop.hive.ql.Compiler.getExplainOutput(BaseSemanticAnalyzer,
QueryPlan, ASTNode): new java.io.PrintStream(OutputStream) At
Compiler.java:[line 504] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19581/dev-support/hive-personality.sh
|
| git revision | master / 1fcc5ce |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19581/yetus/new-findbugs-ql.html
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19581/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Extract Compiler from Driver
>
>
> Key: HIVE-22526
> URL: https://issues.apache.org/jira/browse/HIVE-22526
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22526.01.patch, HIVE-22526.02.patch,
> HIVE-22526.03.patch, HIVE-22526.04.patch
>
>
> The Driver class contains ~600 lines of code responsible for compiling the
> command. That means that from the command String a Plan needs to be created,
> and also a transaction needs to be started (in most of the cases). This is a
> thing done by the compile function, which has a lot of sub functions to help
> this task, while
[jira] [Updated] (HIVE-22526) Extract Compiler from Driver
[ https://issues.apache.org/jira/browse/HIVE-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22526: -- Attachment: HIVE-22526.04.patch > Extract Compiler from Driver > > > Key: HIVE-22526 > URL: https://issues.apache.org/jira/browse/HIVE-22526 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22526.01.patch, HIVE-22526.02.patch, > HIVE-22526.03.patch, HIVE-22526.04.patch > > > The Driver class contains ~600 lines of code responsible for compiling the > command. That means that from the command String a Plan needs to be created, > and also a transaction needs to be started (in most of the cases). This is a > thing done by the compile function, which has a lot of sub functions to help > this task, while itself is also really big. All these codes should be put > into a separate class, where it can do it's job without getting mixed with > the other codes in the Driver. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22532) PTFPPD may push limit incorrectly through Rank/DenseRank function
[
https://issues.apache.org/jira/browse/HIVE-22532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16980998#comment-16980998
]
Hive QA commented on HIVE-22532:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12986598/HIVE-22532.01.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 17718 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query44]
(batchId=302)
org.apache.hadoop.hive.schq.TestScheduledQueryIntegration.testScheduledQueryExecutionImpersonation
(batchId=279)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19580/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19580/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19580/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12986598 - PreCommit-HIVE-Build
> PTFPPD may push limit incorrectly through Rank/DenseRank function
> -
>
> Key: HIVE-22532
> URL: https://issues.apache.org/jira/browse/HIVE-22532
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22532.01.patch, HIVE-22532.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
