[jira] [Commented] (HIVE-22327) Repl: Ignore read-only transactions in notification log
[
https://issues.apache.org/jira/browse/HIVE-22327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968964#comment-16968964
]
Hive QA commented on HIVE-22327:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12985096/HIVE-22327.3.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17571 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19325/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19325/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19325/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12985096 - PreCommit-HIVE-Build
> Repl: Ignore read-only transactions in notification log
> ---
>
> Key: HIVE-22327
> URL: https://issues.apache.org/jira/browse/HIVE-22327
> Project: Hive
> Issue Type: Improvement
> Components: repl
>Reporter: Gopal Vijayaraghavan
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-22327.1.patch, HIVE-22327.2.patch,
> HIVE-22327.3.patch
>
>
> Read txns need not be replicated.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22467) Hive-1: does not set jetty request.header.size correctly incase of SSL set up
[ https://issues.apache.org/jira/browse/HIVE-22467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajkumar Singh updated HIVE-22467: -- Attachment: HIVE-22467.1-branch-1.patch Status: Patch Available (was: In Progress) > Hive-1: does not set jetty request.header.size correctly incase of SSL set up > - > > Key: HIVE-22467 > URL: https://issues.apache.org/jira/browse/HIVE-22467 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Affects Versions: 1.2.1, 1.0.0, 1.3.0 >Reporter: Rajkumar Singh >Assignee: Rajkumar Singh >Priority: Major > Attachments: HIVE-22467.1-branch-1.patch > > > on hive-1: > if user have SSL setup with HS2 then SslSelectChannelConnector override the > connector request header settings. > [https://github.com/apache/hive/blob/5740946859fcca44b5e453ef02534b1ec5edcbca/service/src/java/org/apache/hive/service/cli/thrift/ThriftHttpCLIService.java#L102] > which leads to HTTP 413 Full HEAD error for user connecting to HS2 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work started] (HIVE-22467) Hive-1: does not set jetty request.header.size correctly incase of SSL set up
[ https://issues.apache.org/jira/browse/HIVE-22467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-22467 started by Rajkumar Singh. - > Hive-1: does not set jetty request.header.size correctly incase of SSL set up > - > > Key: HIVE-22467 > URL: https://issues.apache.org/jira/browse/HIVE-22467 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Affects Versions: 1.0.0, 1.3.0, 1.2.1 >Reporter: Rajkumar Singh >Assignee: Rajkumar Singh >Priority: Major > > on hive-1: > if user have SSL setup with HS2 then SslSelectChannelConnector override the > connector request header settings. > [https://github.com/apache/hive/blob/5740946859fcca44b5e453ef02534b1ec5edcbca/service/src/java/org/apache/hive/service/cli/thrift/ThriftHttpCLIService.java#L102] > which leads to HTTP 413 Full HEAD error for user connecting to HS2 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22467) Hive-1: does not set jetty request.header.size correctly incase of SSL set up
[ https://issues.apache.org/jira/browse/HIVE-22467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajkumar Singh updated HIVE-22467: -- Description: on hive-1: if user have SSL setup with HS2 then SslSelectChannelConnector override the connector request header settings. [https://github.com/apache/hive/blob/5740946859fcca44b5e453ef02534b1ec5edcbca/service/src/java/org/apache/hive/service/cli/thrift/ThriftHttpCLIService.java#L102] which leads to HTTP 413 Full HEAD error for user connecting to HS2 was: on hive-1: if user have SSL setup with HS2 then SslSelectChannelConnector override the connector request header settings. [https://github.com/apache/hive/blob/5740946859fcca44b5e453ef02534b1ec5edcbca/service/src/java/org/apache/hive/service/cli/thrift/ThriftHttpCLIService.java#L102] > Hive-1: does not set jetty request.header.size correctly incase of SSL set up > - > > Key: HIVE-22467 > URL: https://issues.apache.org/jira/browse/HIVE-22467 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Affects Versions: 1.0.0, 1.3.0, 1.2.1 >Reporter: Rajkumar Singh >Assignee: Rajkumar Singh >Priority: Major > > on hive-1: > if user have SSL setup with HS2 then SslSelectChannelConnector override the > connector request header settings. > [https://github.com/apache/hive/blob/5740946859fcca44b5e453ef02534b1ec5edcbca/service/src/java/org/apache/hive/service/cli/thrift/ThriftHttpCLIService.java#L102] > which leads to HTTP 413 Full HEAD error for user connecting to HS2 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-22467) Hive-1: does not set jetty request.header.size correctly incase of SSL set up
[ https://issues.apache.org/jira/browse/HIVE-22467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajkumar Singh reassigned HIVE-22467: - > Hive-1: does not set jetty request.header.size correctly incase of SSL set up > - > > Key: HIVE-22467 > URL: https://issues.apache.org/jira/browse/HIVE-22467 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Affects Versions: 1.2.1, 1.0.0, 1.3.0 >Reporter: Rajkumar Singh >Assignee: Rajkumar Singh >Priority: Major > > on hive-1: > if user have SSL setup with HS2 then SslSelectChannelConnector override the > connector request header settings. > [https://github.com/apache/hive/blob/5740946859fcca44b5e453ef02534b1ec5edcbca/service/src/java/org/apache/hive/service/cli/thrift/ThriftHttpCLIService.java#L102] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22405) Add ColumnVector support for ProlepticCalendar
[
https://issues.apache.org/jira/browse/HIVE-22405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968928#comment-16968928
]
Hive QA commented on HIVE-22405:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12985092/HIVE-22405.02.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17572 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19324/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19324/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19324/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12985092 - PreCommit-HIVE-Build
> Add ColumnVector support for ProlepticCalendar
> --
>
> Key: HIVE-22405
> URL: https://issues.apache.org/jira/browse/HIVE-22405
> Project: Hive
> Issue Type: Bug
>Reporter: Owen O'Malley
>Assignee: László Bodor
>Priority: Major
> Attachments: HIVE-22405.01.patch, HIVE-22405.02.patch
>
>
> Hive recently moved its processing to the proleptic calendar, which has
> created some issues for users who have dates before 1580 AD.
> I'd propose extending the column vectors for times & dates to encode which
> calendar they are using.
> * create DateColumnVector that extends LongColumnVector
> * add a method to change calendars to both DateColumnVector and
> TimestampColumnVector.
> {code}
> /**
>* Change the calendar to or from proleptic. If the new and old values of
> the flag are the
>* same, nothing is done.
>* useProleptic - set the flag for the proleptic calendar
>* updateData - change the data to match the new value of the flag.
>*/
> void changeCalendar(useProleptic: boolean, updateData: boolean);
> /**
>* Detect whether this data is using the proleptic calendar.
>*/
> boolean usingProlepticCalendar();
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22448) CBO: Expand the multiple count distinct with a group-by key
[
https://issues.apache.org/jira/browse/HIVE-22448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968898#comment-16968898
]
Hive QA commented on HIVE-22448:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12985081/HIVE-22448.02.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17571 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19323/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19323/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19323/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12985081 - PreCommit-HIVE-Build
> CBO: Expand the multiple count distinct with a group-by key
> ---
>
> Key: HIVE-22448
> URL: https://issues.apache.org/jira/browse/HIVE-22448
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Gopal Vijayaraghavan
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22448.01.patch, HIVE-22448.02.patch,
> HIVE-22448.02.patch, HIVE-22448.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {code}
> create temporary table mytable1 (x integer, y integer, z integer, a integer);
> explain cbo
> select z, x, count(distinct y), count(distinct a)
> from mytable1
> group by z, x;
> explain cbo
> select count(distinct y), count(distinct a)
> from mytable1
> {code}
> The first is not vectorized, the second one is because of the grouping-set
> based rewrite for count distinct.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-21884) Scheduled query support
[
https://issues.apache.org/jira/browse/HIVE-21884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968883#comment-16968883
]
Hive QA commented on HIVE-21884:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
1s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 13m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 6m
58s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
45s{color} | {color:blue} standalone-metastore/metastore-common in master has
37 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
36s{color} | {color:blue} common in master has 65 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
16s{color} | {color:blue} standalone-metastore/metastore-server in master has
173 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
20s{color} | {color:blue} ql in master has 1550 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
42s{color} | {color:blue} service in master has 49 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
44s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
51s{color} | {color:blue} itests/util in master has 53 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 12m
23s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 16m
19s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
20s{color} | {color:red} standalone-metastore: The patch generated 2 new + 1678
unchanged - 16 fixed = 1680 total (was 1694) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
35s{color} | {color:green} The patch metastore-common passed checkstyle {color}
|
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
38s{color} | {color:red} common: The patch generated 2 new + 369 unchanged - 0
fixed = 371 total (was 369) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
23s{color} | {color:green} The patch metastore passed checkstyle {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 3m
2s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 2 new + 1184 unchanged - 16 fixed = 1186 total (was 1200) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
22s{color} | {color:red} ql: The patch generated 30 new + 683 unchanged - 5
fixed = 713 total (was 688) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
23s{color} | {color:green} service: The patch generated 0 new + 38 unchanged -
1 fixed = 38 total (was 39) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 3m
28s{color} | {color:red} root: The patch generated 44 new + 2853 unchanged - 25
fixed = 2897 total (was 2878) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} The patch hcatalog-unit passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
17s{color} | {color:green} The patch hive-minikdc passed checkstyle {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
[jira] [Commented] (HIVE-22448) CBO: Expand the multiple count distinct with a group-by key
[
https://issues.apache.org/jira/browse/HIVE-22448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968880#comment-16968880
]
Hive QA commented on HIVE-22448:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
14s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
11s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 6m
49s{color} | {color:blue} ql in master has 1550 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
39s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
40s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
0s{color} | {color:green} ql: The patch generated 0 new + 4 unchanged - 1
fixed = 4 total (was 5) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
40s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
25s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 39m 17s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19323/dev-support/hive-personality.sh
|
| git revision | master / 4cfb7a0 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql itests U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19323/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> CBO: Expand the multiple count distinct with a group-by key
> ---
>
> Key: HIVE-22448
> URL: https://issues.apache.org/jira/browse/HIVE-22448
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Gopal Vijayaraghavan
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22448.01.patch, HIVE-22448.02.patch,
> HIVE-22448.02.patch, HIVE-22448.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {code}
> create temporary table mytable1 (x integer, y integer, z integer, a integer);
> explain cbo
> select z, x, count(distinct y), count(distinct a)
> from mytable1
> group by z, x;
> explain cbo
> select count(distinct y), count(distinct a)
> from mytable1
> {code}
> The first is not vectorized, the second one is because of the grouping-set
> based rewrite for count distinct.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-21884) Scheduled query support
[
https://issues.apache.org/jira/browse/HIVE-21884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968860#comment-16968860
]
Hive QA commented on HIVE-21884:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12985079/HIVE-21884.26.patch
{color:green}SUCCESS:{color} +1 due to 20 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 17608 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb_schq]
(batchId=176)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[hybridgrace_hashjoin_2]
(batchId=111)
org.apache.hadoop.hive.metastore.client.TestMetastoreScheduledQueries.testCleanup[Embedded]
(batchId=225)
org.apache.hadoop.hive.metastore.client.TestMetastoreScheduledQueries.testCleanup[Remote]
(batchId=225)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19322/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19322/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19322/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12985079 - PreCommit-HIVE-Build
> Scheduled query support
> ---
>
> Key: HIVE-21884
> URL: https://issues.apache.org/jira/browse/HIVE-21884
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21844.04.patch, HIVE-21844.05.patch,
> HIVE-21844.06.patch, HIVE-21844.07.patch, HIVE-21844.08.patch,
> HIVE-21844.09.patch, HIVE-21844.15.patch, HIVE-21844.19.patch,
> HIVE-21844.21.patch, HIVE-21844.22.patch, HIVE-21884.01.patch,
> HIVE-21884.02.patch, HIVE-21884.03.patch, HIVE-21884.09.patch,
> HIVE-21884.10.patch, HIVE-21884.10.patch, HIVE-21884.11.patch,
> HIVE-21884.12.patch, HIVE-21884.13.patch, HIVE-21884.14.patch,
> HIVE-21884.14.patch, HIVE-21884.14.patch, HIVE-21884.16.patch,
> HIVE-21884.17.patch, HIVE-21884.18.patch, HIVE-21884.20.patch,
> HIVE-21884.23.patch, HIVE-21884.24.patch, HIVE-21884.25.patch,
> HIVE-21884.26.patch, Scheduled queries2.pdf
>
> Time Spent: 7h 40m
> Remaining Estimate: 0h
>
> design document:
> https://docs.google.com/document/d/1mJSFdJi_1cbxJTXC9QvGw2rQ3zzJkNfxOO6b5esmyCE/edit#
> in case the google doc is not reachable: [^Scheduled queries2.pdf]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-21198) Introduce a database object reference class
[
https://issues.apache.org/jira/browse/HIVE-21198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968824#comment-16968824
]
Hive QA commented on HIVE-21198:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12985072/HIVE-21198.10.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 59 failed/errored test(s), 17571 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_table]
(batchId=23)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[create_union_table]
(batchId=30)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ctas] (batchId=7)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ctas_colname]
(batchId=65)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ctas_uses_database_location]
(batchId=38)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[explain_ddl] (batchId=53)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[fileformat_sequencefile]
(batchId=52)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[fileformat_text]
(batchId=54)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby_duplicate_key]
(batchId=7)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input15] (batchId=1)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[inputddl1] (batchId=33)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[inputddl2] (batchId=13)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[inputddl3] (batchId=78)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[merge3] (batchId=65)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[nonReservedKeyWords]
(batchId=68)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[nonmr_fetch] (batchId=22)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[nullformatCTAS]
(batchId=44)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[nullformat] (batchId=48)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[orc_createas1]
(batchId=97)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parallel_orderby]
(batchId=60)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[serde_opencsv]
(batchId=83)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[serde_regex] (batchId=42)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoin_noskew]
(batchId=18)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoin_onesideskew]
(batchId=79)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[smb_mapjoin9]
(batchId=45)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[symlink_text_input_format]
(batchId=52)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union25] (batchId=77)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_decimal_6]
(batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_tablesample_rows]
(batchId=57)
org.apache.hadoop.hive.cli.TestContribCliDriver.testCliDriver[fileformat_base64]
(batchId=294)
org.apache.hadoop.hive.cli.TestContribCliDriver.testCliDriver[serde_regex]
(batchId=294)
org.apache.hadoop.hive.cli.TestContribNegativeCliDriver.testCliDriver[serde_regex]
(batchId=296)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_dynamic_partition]
(batchId=198)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_mv]
(batchId=198)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[rcfile_createas1]
(batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[ctas]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dynamic_partition_pruning]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[partition_ctas]
(batchId=173)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[semijoin_reddedup]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[temp_table]
(batchId=186)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_dml]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_join_result_complex]
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[union_top_level]
(batchId=178)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_char_varchar_1]
(batchId=187)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_decimal_6]
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_windowing_streaming]
(batchId=176)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorized_dynamic_partition_pruning]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[parallel_orderby]
(batchId=194)
[jira] [Commented] (HIVE-21198) Introduce a database object reference class
[
https://issues.apache.org/jira/browse/HIVE-21198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968803#comment-16968803
]
Hive QA commented on HIVE-21198:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
45s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
51s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
27s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
24s{color} | {color:blue} storage-api in master has 48 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
23s{color} | {color:blue} ql in master has 1550 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
33s{color} | {color:blue} accumulo-handler in master has 21 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
32s{color} | {color:blue} beeline in master has 48 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
37s{color} | {color:blue} hbase-handler in master has 15 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
41s{color} | {color:blue} hcatalog/core in master has 36 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
31s{color} | {color:blue} hcatalog/webhcat/java-client in master has 3 extant
Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
37s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
29s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
19s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 11s{color}
| {color:red} storage-api generated 2 new + 0 unchanged - 2 fixed = 2 total
(was 2) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} storage-api: The patch generated 0 new + 4 unchanged
- 1 fixed = 4 total (was 5) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} The patch metastore passed checkstyle {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
57s{color} | {color:red} ql: The patch generated 8 new + 1420 unchanged - 85
fixed = 1428 total (was 1505) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} The patch accumulo-handler passed checkstyle {color}
|
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} The patch beeline passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} The patch hbase-handler passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} hcatalog/core: The patch generated 0 new + 48
unchanged - 1 fixed = 48 total (was 49) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} The patch java-client passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
25s{color} | {color:red} ql generated 2 new + 1546 unchanged - 4 fixed = 1548
total (was 1550) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
38s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
|
[jira] [Commented] (HIVE-22414) Make LLAP CacheTags more memory efficient
[
https://issues.apache.org/jira/browse/HIVE-22414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968786#comment-16968786
]
Hive QA commented on HIVE-22414:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12985070/HIVE-22414.6.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17571 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testDynamicPartitionInsert
(batchId=353)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19320/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19320/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19320/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12985070 - PreCommit-HIVE-Build
> Make LLAP CacheTags more memory efficient
> -
>
> Key: HIVE-22414
> URL: https://issues.apache.org/jira/browse/HIVE-22414
> Project: Hive
> Issue Type: Improvement
> Components: llap
>Reporter: Ádám Szita
>Assignee: Ádám Szita
>Priority: Major
> Attachments: HIVE-22414.0.patch, HIVE-22414.1.patch,
> HIVE-22414.2.patch, HIVE-22414.3.patch, HIVE-22414.4.patch,
> HIVE-22414.5.patch, HIVE-22414.6.patch
>
>
> MultiPartitionCacheTag relies on LinkedLists.
> A LinkedList object that holds e.g. 2 nodes consumes 112 bytes roughly in
> this composition:
> * 16 bytes for LinkedList object header
> * 8 bytes for referring head
> * 8 bytes for referring tail
> * 80 = 2 x (16 bytes for LinkedList$Node header, 3 x 8 bytes for referring
> prev, next, item)
> This is a lot, so I propose to replace LinkedList in MultiPartitionCacheTag
> with a simple String array. (For a similar scenario an array would take 16 +
> 8 + 2 x 8 = 40 bytes, as per header, count, and 2 references for our actual
> objects).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-22466) Prevent synchronized calls in LlapDaemon configuration
[ https://issues.apache.org/jira/browse/HIVE-22466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mustafa Iman reassigned HIVE-22466: --- > Prevent synchronized calls in LlapDaemon configuration > -- > > Key: HIVE-22466 > URL: https://issues.apache.org/jira/browse/HIVE-22466 > Project: Hive > Issue Type: Bug > Components: llap >Affects Versions: 4.0.0 >Reporter: Mustafa Iman >Assignee: Mustafa Iman >Priority: Major > > LlapDaemonConfig extends from Hadoop's Configuration class. Configuration > class makes use of synchronized calls for both reads and writes to > configuration. LlapDaemon does not change configuration in runtime. We can > remove synchronized access to configuration in LlapConfiguration -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22414) Make LLAP CacheTags more memory efficient
[
https://issues.apache.org/jira/browse/HIVE-22414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968759#comment-16968759
]
Hive QA commented on HIVE-22414:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
5s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
27s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
24s{color} | {color:blue} storage-api in master has 48 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
47s{color} | {color:blue} llap-server in master has 90 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
30s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
24s{color} | {color:red} llap-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
23s{color} | {color:red} llap-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 23s{color}
| {color:red} llap-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
20s{color} | {color:red} llap-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
10s{color} | {color:red} storage-api generated 1 new + 28 unchanged - 0 fixed =
29 total (was 28) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 17m 9s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19320/dev-support/hive-personality.sh
|
| git revision | master / 4cfb7a0 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19320/yetus/patch-mvninstall-llap-server.txt
|
| compile |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19320/yetus/patch-compile-llap-server.txt
|
| javac |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19320/yetus/patch-compile-llap-server.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19320/yetus/patch-findbugs-llap-server.txt
|
| javadoc |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19320/yetus/diff-javadoc-javadoc-storage-api.txt
|
| modules | C: storage-api llap-server U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19320/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Make LLAP CacheTags more memory efficient
> -
>
> Key: HIVE-22414
> URL: https://issues.apache.org/jira/browse/HIVE-22414
> Project: Hive
> Issue Type: Improvement
> Components: llap
>Reporter: Ádám Szita
>Assignee: Ádám Szita
>Priority: Major
> Attachments: HIVE-22414.0.patch, HIVE-22414.1.patch,
> HIVE-22414.2.patch, HIVE-22414.3.patch, HIVE-22414.4.patch,
> HIVE-22414.5.patch, HIVE-22414.6.patch
>
>
> MultiPartitionCacheTag relies on LinkedLists.
> A
[jira] [Updated] (HIVE-22408) The fix for CALCITE-2991 creates wrong results on edge case
[ https://issues.apache.org/jira/browse/HIVE-22408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-22408: --- Status: Patch Available (was: Open) > The fix for CALCITE-2991 creates wrong results on edge case > --- > > Key: HIVE-22408 > URL: https://issues.apache.org/jira/browse/HIVE-22408 > Project: Hive > Issue Type: Bug > Components: CBO >Affects Versions: 3.1.2 >Reporter: Steve Carlin >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-22408.1.patch > > > The Calcite upgrade from 1.19 to 1.21 includes a fix for CALCITE-2991. This > patch causes a failure in subquery_ALL.q > In order to proceed with the upgrade, the patch was bypassed by creating the > HiveRelMdMaxRowCount class which overrides the behavior in Calcite. > In order to recreate the regression, remove this file and the references to > the class in HiveDefaultRelMetadataProvider. > This Jira is being created before the HIVE-22274 PR has been pushed, so > please wait until that is pushed before working on this. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22408) The fix for CALCITE-2991 creates wrong results on edge case
[ https://issues.apache.org/jira/browse/HIVE-22408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-22408: --- Attachment: HIVE-22408.1.patch > The fix for CALCITE-2991 creates wrong results on edge case > --- > > Key: HIVE-22408 > URL: https://issues.apache.org/jira/browse/HIVE-22408 > Project: Hive > Issue Type: Bug > Components: CBO >Affects Versions: 3.1.2 >Reporter: Steve Carlin >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-22408.1.patch > > > The Calcite upgrade from 1.19 to 1.21 includes a fix for CALCITE-2991. This > patch causes a failure in subquery_ALL.q > In order to proceed with the upgrade, the patch was bypassed by creating the > HiveRelMdMaxRowCount class which overrides the behavior in Calcite. > In order to recreate the regression, remove this file and the references to > the class in HiveDefaultRelMetadataProvider. > This Jira is being created before the HIVE-22274 PR has been pushed, so > please wait until that is pushed before working on this. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22238) PK/FK selectivity estimation underscales estimations
[ https://issues.apache.org/jira/browse/HIVE-22238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-22238: Attachment: HIVE-22238.10.patch > PK/FK selectivity estimation underscales estimations > > > Key: HIVE-22238 > URL: https://issues.apache.org/jira/browse/HIVE-22238 > Project: Hive > Issue Type: Bug > Components: Statistics >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22238.01.patch, HIVE-22238.02.patch, > HIVE-22238.03.patch, HIVE-22238.04.patch, HIVE-22238.05.patch, > HIVE-22238.05.patch, HIVE-22238.05.patch, HIVE-22238.05.patch, > HIVE-22238.05.patch, HIVE-22238.06.patch, HIVE-22238.06.patch, > HIVE-22238.07.patch, HIVE-22238.09.patch, HIVE-22238.10.patch > > Time Spent: 40m > Remaining Estimate: 0h > > at [this > point|https://github.com/apache/hive/blob/5098d155a1e6a164253f5fa98755273bc34085df/ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/StatsRulesProcFactory.java#L2182] > the parent operators rownum is scaled according to pkfkselectivity > however [pkfkselectivity is > computed|https://github.com/apache/hive/blob/5098d155a1e6a164253f5fa98755273bc34085df/ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/StatsRulesProcFactory.java#L2157] > on a whole subtree. > Scaling it by that amount will count in estimation already used when > parentstats was calculated...so depending on the number of upstream joins - > this may lead to severe underestimations > what happened was: > * optimization was able to push the filter to the other side of the join > * as a result the incoming data was already filtered > * scaling down by the PK selectiviy - was actually already there...but a new > "scaling" happened -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22461) NPE Metastore Transformer
[
https://issues.apache.org/jira/browse/HIVE-22461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968745#comment-16968745
]
Hive QA commented on HIVE-22461:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12985069/HIVE-22461.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17570 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19319/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19319/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19319/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12985069 - PreCommit-HIVE-Build
> NPE Metastore Transformer
> -
>
> Key: HIVE-22461
> URL: https://issues.apache.org/jira/browse/HIVE-22461
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.1.2
>Reporter: Yongzhi Chen
>Assignee: Yongzhi Chen
>Priority: Major
> Attachments: HIVE-22461.1.patch
>
>
> The stack looks as following:
> {noformat}
> 2019-10-08 18:09:12,198 INFO
> org.apache.hadoop.hive.metastore.MetastoreDefaultTransformer:
> [pool-6-thread-328]: Starting translation for processor
> Hiveserver2#3.1.2000.7.0.2.0...@vc0732.halxg.cloudera.com on list 1
> 2019-10-08 18:09:12,198 ERROR
> org.apache.hadoop.hive.metastore.RetryingHMSHandler: [pool-6-thread-328]:
> java.lang.NullPointerException
> at
> org.apache.hadoop.hive.metastore.MetastoreDefaultTransformer.transform(MetastoreDefaultTransformer.java:99)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getTableInternal(HiveMetaStore.java:3391)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table_req(HiveMetaStore.java:3352)
> at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
> at com.sun.proxy.$Proxy28.get_table_req(Unknown Source)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_table_req.getResult(ThriftHiveMetastore.java:16633)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_table_req.getResult(ThriftHiveMetastore.java:16617)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> at
> org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge.java:636)
> at
> org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge.java:631)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
> at
> org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:631)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> 2019-10-08 18:09:12,199 ERROR org.apache.thrift.server.TThreadPoolServer:
> [pool-6-thread-328]: Error occurred during processing of message.
> java.lang.NullPointerException: null
> at
> org.apache.hadoop.hive.metastore.MetastoreDefaultTransformer.transform(MetastoreDefaultTransformer.java:99)
> ~[hive-exec-3.1.2000.7.0.2.0-59.jar:3.1.2000.7.0.2.0-59]
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getTableInternal(HiveMetaStore.java:3391)
> ~[hive-exec-3.1.2000.7.0.2.0-59.jar:3.1.2000.7.0.2.0-59]
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table_req(HiveMetaStore.java:3352)
> ~[hive-exec-3.1.2000.7.0.2.0-59.jar:3.1.2000.7.0.2.0-59]
> at
[jira] [Commented] (HIVE-22461) NPE Metastore Transformer
[
https://issues.apache.org/jira/browse/HIVE-22461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968692#comment-16968692
]
Hive QA commented on HIVE-22461:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
23s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
13s{color} | {color:blue} standalone-metastore/metastore-server in master has
173 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
22s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 15m 41s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19319/dev-support/hive-personality.sh
|
| git revision | master / 4cfb7a0 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: standalone-metastore/metastore-server U:
standalone-metastore/metastore-server |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19319/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> NPE Metastore Transformer
> -
>
> Key: HIVE-22461
> URL: https://issues.apache.org/jira/browse/HIVE-22461
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.1.2
>Reporter: Yongzhi Chen
>Assignee: Yongzhi Chen
>Priority: Major
> Attachments: HIVE-22461.1.patch
>
>
> The stack looks as following:
> {noformat}
> 2019-10-08 18:09:12,198 INFO
> org.apache.hadoop.hive.metastore.MetastoreDefaultTransformer:
> [pool-6-thread-328]: Starting translation for processor
> Hiveserver2#3.1.2000.7.0.2.0...@vc0732.halxg.cloudera.com on list 1
> 2019-10-08 18:09:12,198 ERROR
> org.apache.hadoop.hive.metastore.RetryingHMSHandler: [pool-6-thread-328]:
> java.lang.NullPointerException
> at
> org.apache.hadoop.hive.metastore.MetastoreDefaultTransformer.transform(MetastoreDefaultTransformer.java:99)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getTableInternal(HiveMetaStore.java:3391)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table_req(HiveMetaStore.java:3352)
> at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
> at
>
[jira] [Commented] (HIVE-22444) Clean up Project POM Files
[
https://issues.apache.org/jira/browse/HIVE-22444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968672#comment-16968672
]
Hive QA commented on HIVE-22444:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
58s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 12m
49s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 17m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
12s{color} | {color:green} storage-api generated 0 new + 0 unchanged - 2 fixed
= 0 total (was 2) {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
20s{color} | {color:green} serde in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
11s{color} | {color:green} ql in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
15s{color} | {color:green} upgrade-acid in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m
31s{color} | {color:green} root generated 0 new + 32 unchanged - 2 fixed = 32
total (was 34) {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
14s{color} | {color:green} custom-serde in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
15s{color} | {color:green} custom-udfs in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
14s{color} | {color:green} udf-classloader-udf1 in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
13s{color} | {color:green} udf-classloader-udf2 in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
13s{color} | {color:green} udf-vectorized-badexample in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
24s{color} | {color:green} hcatalog-unit in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
35s{color} | {color:green} hive-blobstore in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
21s{color} | {color:green} hive-jmh in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
34s{color} | {color:green} hive-minikdc in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
47s{color} | {color:green} hive-unit in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
34s{color} | {color:green} hive-unit-hadoop2 in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
29s{color} | {color:green} qtest-accumulo in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
15s{color} | {color:green} qtest-druid in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
27s{color} | {color:green} qtest-kudu in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
34s{color} | {color:green} qtest-spark in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
13s{color} | {color:green} test-serde in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
31s{color} | {color:green} util in the patch passed. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} |
[jira] [Commented] (HIVE-22444) Clean up Project POM Files
[
https://issues.apache.org/jira/browse/HIVE-22444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968659#comment-16968659
]
Hive QA commented on HIVE-22444:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12985067/HIVE-22444.8.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 12 failed/errored test(s), 17494 tests
executed
*Failed tests:*
{noformat}
TestMiniDruidKafkaCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=299)
TestMiniHiveKafkaCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=299)
TestMiniLlapLocalCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=175)
[materialized_view_create_rewrite_3.q,correlationoptimizer2.q,cbo_gby_empty.q,resourceplan.q,schema_evol_text_nonvec_part_all_complex_llap_io.q,vectorization_short_regress.q,identity_project_remove_skip.q,mapjoin3.q,cross_product_check_1.q,bucket2.q,cbo_join.q,correlationoptimizer6.q,union_remove_26.q,cbo_rp_limit.q,convert_decimal64_to_decimal.q,vectorization_13.q,mapjoin_mapjoin.q,union2.q,groupby2.q,vector_decimal_precision.q,dynpart_sort_opt_vectorization.q,constraints_optimization.q,exchgpartition2lel.q,schema_evol_text_vecrow_part_llap_io.q,external_jdbc_table2.q,sample10.q,vectorized_timestamp_ints_casts.q,auto_sortmerge_join_2.q,bucketizedhiveinputformat.q,cte_mat_2.q]
org.apache.hive.minikdc.TestHs2HooksWithMiniKdc.org.apache.hive.minikdc.TestHs2HooksWithMiniKdc
(batchId=300)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc
(batchId=300)
org.apache.hive.minikdc.TestJdbcWithDBTokenStore.org.apache.hive.minikdc.TestJdbcWithDBTokenStore
(batchId=300)
org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs
(batchId=300)
org.apache.hive.minikdc.TestJdbcWithMiniKdc.org.apache.hive.minikdc.TestJdbcWithMiniKdc
(batchId=300)
org.apache.hive.minikdc.TestJdbcWithMiniKdcCookie.org.apache.hive.minikdc.TestJdbcWithMiniKdcCookie
(batchId=300)
org.apache.hive.minikdc.TestJdbcWithMiniKdcSQLAuthBinary.org.apache.hive.minikdc.TestJdbcWithMiniKdcSQLAuthBinary
(batchId=300)
org.apache.hive.minikdc.TestJdbcWithMiniKdcSQLAuthHttp.org.apache.hive.minikdc.TestJdbcWithMiniKdcSQLAuthHttp
(batchId=300)
org.apache.hive.minikdc.TestSSLWithMiniKdc.org.apache.hive.minikdc.TestSSLWithMiniKdc
(batchId=300)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19318/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19318/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19318/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 12 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12985067 - PreCommit-HIVE-Build
> Clean up Project POM Files
> --
>
> Key: HIVE-22444
> URL: https://issues.apache.org/jira/browse/HIVE-22444
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-22444.1.patch, HIVE-22444.2.patch,
> HIVE-22444.3.patch, HIVE-22444.4.patch, HIVE-22444.5.patch,
> HIVE-22444.6.patch, HIVE-22444.7.patch, HIVE-22444.8.patch
>
>
> # Address warnings in the build process
> # Use DependencyManagement in Root POM for ITest (see HIVE-22426)
> # General POM cleanup
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22463) Support Decimal64 column multiplication with decimal64 Column/Scalar
[ https://issues.apache.org/jira/browse/HIVE-22463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ramesh Kumar Thangarajan updated HIVE-22463: Status: Open (was: Patch Available) > Support Decimal64 column multiplication with decimal64 Column/Scalar > > > Key: HIVE-22463 > URL: https://issues.apache.org/jira/browse/HIVE-22463 > Project: Hive > Issue Type: Bug >Reporter: Ramesh Kumar Thangarajan >Assignee: Ramesh Kumar Thangarajan >Priority: Major > Attachments: HIVE-22463.1.patch, HIVE-22463.2.patch > > > Support Decimal64 column multiplication with decimal64 Column/Scalar -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22463) Support Decimal64 column multiplication with decimal64 Column/Scalar
[ https://issues.apache.org/jira/browse/HIVE-22463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ramesh Kumar Thangarajan updated HIVE-22463: Attachment: HIVE-22463.2.patch Status: Patch Available (was: Open) > Support Decimal64 column multiplication with decimal64 Column/Scalar > > > Key: HIVE-22463 > URL: https://issues.apache.org/jira/browse/HIVE-22463 > Project: Hive > Issue Type: Bug >Reporter: Ramesh Kumar Thangarajan >Assignee: Ramesh Kumar Thangarajan >Priority: Major > Attachments: HIVE-22463.1.patch, HIVE-22463.2.patch > > > Support Decimal64 column multiplication with decimal64 Column/Scalar -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22464) Implement support for NULLS FIRST/LAST in TopNKeyOperator
[
https://issues.apache.org/jira/browse/HIVE-22464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968596#comment-16968596
]
Hive QA commented on HIVE-22464:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12985060/HIVE-22464.2.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17570 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19317/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19317/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19317/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12985060 - PreCommit-HIVE-Build
> Implement support for NULLS FIRST/LAST in TopNKeyOperator
> -
>
> Key: HIVE-22464
> URL: https://issues.apache.org/jira/browse/HIVE-22464
> Project: Hive
> Issue Type: Improvement
> Components: Physical Optimizer
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22464.1.patch, HIVE-22464.2.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22327) Repl: Ignore read-only transactions in notification log
[ https://issues.apache.org/jira/browse/HIVE-22327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denys Kuzmenko updated HIVE-22327: -- Attachment: HIVE-22327.3.patch > Repl: Ignore read-only transactions in notification log > --- > > Key: HIVE-22327 > URL: https://issues.apache.org/jira/browse/HIVE-22327 > Project: Hive > Issue Type: Improvement > Components: repl >Reporter: Gopal Vijayaraghavan >Assignee: Denys Kuzmenko >Priority: Major > Attachments: HIVE-22327.1.patch, HIVE-22327.2.patch, > HIVE-22327.3.patch > > > Read txns need not be replicated. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22464) Implement support for NULLS FIRST/LAST in TopNKeyOperator
[
https://issues.apache.org/jira/browse/HIVE-22464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968559#comment-16968559
]
Hive QA commented on HIVE-22464:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
57s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
40s{color} | {color:blue} serde in master has 199 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
10s{color} | {color:blue} ql in master has 1550 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
22s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
25s{color} | {color:red} ql generated 1 new + 1547 unchanged - 3 fixed = 1548
total (was 1550) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
23s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 30m 49s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | org.apache.hadoop.hive.ql.exec.TopNKeyOperator$KeyWrapperComparator
implements Comparator but not Serializable At
TopNKeyOperator.java:Serializable At TopNKeyOperator.java:[lines 65-82] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19317/dev-support/hive-personality.sh
|
| git revision | master / 4cfb7a0 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19317/yetus/new-findbugs-ql.html
|
| modules | C: serde ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19317/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Implement support for NULLS FIRST/LAST in TopNKeyOperator
> -
>
> Key: HIVE-22464
> URL: https://issues.apache.org/jira/browse/HIVE-22464
> Project: Hive
> Issue Type: Improvement
> Components: Physical Optimizer
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22464.1.patch, HIVE-22464.2.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22445) LazySimpleSerDe toString Prints Memory Address instead of Value
[
https://issues.apache.org/jira/browse/HIVE-22445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968553#comment-16968553
]
David Mollitor commented on HIVE-22445:
---
[~mgergely] Are you able to review this one? Thanks!
> LazySimpleSerDe toString Prints Memory Address instead of Value
> ---
>
> Key: HIVE-22445
> URL: https://issues.apache.org/jira/browse/HIVE-22445
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 3.2.0
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Trivial
> Attachments: HIVE-22445.1.patch
>
>
> {code:none}
> 2019-11-01T10:03:49,228 INFO [pool-23-thread-1] exec.FileSinkOperator: Using
> serializer : class
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe[[[B@983dd25]:[_col0,
> _col1]:[struct {code}
> You can see that the second value prints the memory address of the array, not
> the contents of the array.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22462) Error Information Lost in GenericUDTFGetSplits
[
https://issues.apache.org/jira/browse/HIVE-22462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968552#comment-16968552
]
David Mollitor commented on HIVE-22462:
---
[~mgergely] Are you able to review this one? Thanks!
> Error Information Lost in GenericUDTFGetSplits
> --
>
> Key: HIVE-22462
> URL: https://issues.apache.org/jira/browse/HIVE-22462
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 3.2.0
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HIVE-22462.1.patch
>
>
> I was recently looking at some logs from a failed unit test and saw:
>
> {code:none}
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> org.apache.hadoop.hive.ql.metadata.HiveException: Failed to create temp
> table: null
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> org.apache.hadoop.hive.ql.metadata.HiveException: Failed to create temp
> table: null at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDTFGetSplits2.process(GenericUDTFGetSplits2.java:81)
> at
> org.apache.hadoop.hive.ql.exec.UDTFOperator.process(UDTFOperator.java:116) at
> org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:888) at
> org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:95)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:888) at
> org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.jav
> {code}
> Error information was lost... useless 'null' string is written.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22405) Add ColumnVector support for ProlepticCalendar
[
https://issues.apache.org/jira/browse/HIVE-22405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
László Bodor updated HIVE-22405:
Attachment: HIVE-22405.02.patch
> Add ColumnVector support for ProlepticCalendar
> --
>
> Key: HIVE-22405
> URL: https://issues.apache.org/jira/browse/HIVE-22405
> Project: Hive
> Issue Type: Bug
>Reporter: Owen O'Malley
>Assignee: László Bodor
>Priority: Major
> Attachments: HIVE-22405.01.patch, HIVE-22405.02.patch
>
>
> Hive recently moved its processing to the proleptic calendar, which has
> created some issues for users who have dates before 1580 AD.
> I'd propose extending the column vectors for times & dates to encode which
> calendar they are using.
> * create DateColumnVector that extends LongColumnVector
> * add a method to change calendars to both DateColumnVector and
> TimestampColumnVector.
> {code}
> /**
>* Change the calendar to or from proleptic. If the new and old values of
> the flag are the
>* same, nothing is done.
>* useProleptic - set the flag for the proleptic calendar
>* updateData - change the data to match the new value of the flag.
>*/
> void changeCalendar(useProleptic: boolean, updateData: boolean);
> /**
>* Detect whether this data is using the proleptic calendar.
>*/
> boolean usingProlepticCalendar();
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22238) PK/FK selectivity estimation underscales estimations
[
https://issues.apache.org/jira/browse/HIVE-22238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968518#comment-16968518
]
Hive QA commented on HIVE-22238:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12985059/HIVE-22238.09.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19316/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19316/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19316/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2019-11-06 17:15:48.258
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-19316/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2019-11-06 17:15:48.262
+ cd apache-github-source-source
+ git fetch origin
>From https://github.com/apache/hive
d0bd071..4cfb7a0 master -> origin/master
+ git reset --hard HEAD
HEAD is now at d0bd071 HIVE-22292: Implement Hypothetical-Set Aggregate
Functions (Krisztian Kasa, reviewed by Jesus Camacho Rodriguez)
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is behind 'origin/master' by 1 commit, and can be fast-forwarded.
(use "git pull" to update your local branch)
+ git reset --hard origin/master
HEAD is now at 4cfb7a0 HIVE-22311: Propagate min/max column values from
statistics to the optimizer for timestamp type (Jesus Camacho Rodriguez,
reviewed by Miklos Gergely)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2019-11-06 17:15:50.484
+ rm -rf ../yetus_PreCommit-HIVE-Build-19316
+ mkdir ../yetus_PreCommit-HIVE-Build-19316
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-19316
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-19316/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error: patch failed:
ql/src/test/results/clientpositive/llap/vector_interval_mapjoin.q.out:240
Falling back to three-way merge...
Applied patch to
'ql/src/test/results/clientpositive/llap/vector_interval_mapjoin.q.out' with
conflicts.
error: patch failed:
ql/src/test/results/clientpositive/vector_interval_mapjoin.q.out:259
Falling back to three-way merge...
Applied patch to
'ql/src/test/results/clientpositive/vector_interval_mapjoin.q.out' with
conflicts.
Going to apply patch with: git apply -p0
/data/hiveptest/working/scratch/build.patch:368: trailing whitespace.
/data/hiveptest/working/scratch/build.patch:370: trailing whitespace.
/data/hiveptest/working/scratch/build.patch:374: trailing whitespace.
/data/hiveptest/working/scratch/build.patch:412: trailing whitespace.
st.type_name='galaxy class'
/data/hiveptest/working/scratch/build.patch:429: trailing whitespace.
st.type_name='galaxy class'
error: patch failed:
ql/src/test/results/clientpositive/llap/vector_interval_mapjoin.q.out:240
Falling back to three-way merge...
Applied patch to
'ql/src/test/results/clientpositive/llap/vector_interval_mapjoin.q.out' with
conflicts.
error: patch failed:
ql/src/test/results/clientpositive/vector_interval_mapjoin.q.out:259
Falling back to three-way merge...
Applied patch to
'ql/src/test/results/clientpositive/vector_interval_mapjoin.q.out' with
conflicts.
U ql/src/test/results/clientpositive/llap/vector_interval_mapjoin.q.out
U ql/src/test/results/clientpositive/vector_interval_mapjoin.q.out
warning: squelched 32 whitespace errors
warning: 37 lines add whitespace errors.
+ result=1
+ '[' 1 -ne 0 ']'
+ rm -rf yetus_PreCommit-HIVE-Build-19316
+ exit 1
'
{noformat}
This
[jira] [Commented] (HIVE-22327) Repl: Ignore read-only transactions in notification log
[
https://issues.apache.org/jira/browse/HIVE-22327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968515#comment-16968515
]
Hive QA commented on HIVE-22327:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12985058/HIVE-22327.2.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17570 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19315/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19315/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19315/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12985058 - PreCommit-HIVE-Build
> Repl: Ignore read-only transactions in notification log
> ---
>
> Key: HIVE-22327
> URL: https://issues.apache.org/jira/browse/HIVE-22327
> Project: Hive
> Issue Type: Improvement
> Components: repl
>Reporter: Gopal Vijayaraghavan
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-22327.1.patch, HIVE-22327.2.patch
>
>
> Read txns need not be replicated.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-21884) Scheduled query support
[
https://issues.apache.org/jira/browse/HIVE-21884?focusedWorklogId=339481=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-339481
]
ASF GitHub Bot logged work on HIVE-21884:
-
Author: ASF GitHub Bot
Created on: 06/Nov/19 16:44
Start Date: 06/Nov/19 16:44
Worklog Time Spent: 10m
Work Description: kgyrtkirk commented on pull request #794: HIVE-21884
URL: https://github.com/apache/hive/pull/794#discussion_r343176720
##
File path: itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java
##
@@ -209,14 +211,19 @@ public QTestUtil(QTestArguments testArgs) throws
Exception {
conf.set("test.data.dir", datasetHandler.getDataDir(conf));
dispatcher.register("dataset", datasetHandler);
dispatcher.register("replace", replaceHandler);
+dispatcher.register("scheduledqueryservice", new
QTestScheduledQueryServiceProvider());
+dispatcher.register("scheduledquerycleaner", new
QTestScheduledQueryCleaner());
String scriptsDir = getScriptsDir();
this.initScript = scriptsDir + File.separator + testArgs.getInitScript();
this.cleanupScript = scriptsDir + File.separator +
testArgs.getCleanupScript();
postInit();
+
conf.setVar(HiveConf.ConfVars.HIVE_SCHEDULED_QUERIES_EXECUTOR_IDLE_SLEEP_TIME,
"1s");
Review comment:
yes; this was placed here in the beginning...I would agree with you - but I
try to move in the direction of decomposing qtestutil by adding some qtest
option - the scheduledqueryservice has a qtest option ; I moved this "set" to
there - so that way it doesn't poison this file
thank you!
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 339481)
Time Spent: 7h 40m (was: 7.5h)
> Scheduled query support
> ---
>
> Key: HIVE-21884
> URL: https://issues.apache.org/jira/browse/HIVE-21884
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21844.04.patch, HIVE-21844.05.patch,
> HIVE-21844.06.patch, HIVE-21844.07.patch, HIVE-21844.08.patch,
> HIVE-21844.09.patch, HIVE-21844.15.patch, HIVE-21844.19.patch,
> HIVE-21844.21.patch, HIVE-21844.22.patch, HIVE-21884.01.patch,
> HIVE-21884.02.patch, HIVE-21884.03.patch, HIVE-21884.09.patch,
> HIVE-21884.10.patch, HIVE-21884.10.patch, HIVE-21884.11.patch,
> HIVE-21884.12.patch, HIVE-21884.13.patch, HIVE-21884.14.patch,
> HIVE-21884.14.patch, HIVE-21884.14.patch, HIVE-21884.16.patch,
> HIVE-21884.17.patch, HIVE-21884.18.patch, HIVE-21884.20.patch,
> HIVE-21884.23.patch, HIVE-21884.24.patch, HIVE-21884.25.patch,
> HIVE-21884.26.patch, Scheduled queries2.pdf
>
> Time Spent: 7h 40m
> Remaining Estimate: 0h
>
> design document:
> https://docs.google.com/document/d/1mJSFdJi_1cbxJTXC9QvGw2rQ3zzJkNfxOO6b5esmyCE/edit#
> in case the google doc is not reachable: [^Scheduled queries2.pdf]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-21884) Scheduled query support
[
https://issues.apache.org/jira/browse/HIVE-21884?focusedWorklogId=339478=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-339478
]
ASF GitHub Bot logged work on HIVE-21884:
-
Author: ASF GitHub Bot
Created on: 06/Nov/19 16:44
Start Date: 06/Nov/19 16:44
Worklog Time Spent: 10m
Work Description: kgyrtkirk commented on pull request #794: HIVE-21884
URL: https://github.com/apache/hive/pull/794#discussion_r343172608
##
File path: common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
##
@@ -4773,7 +4778,24 @@ private static void
populateLlapDaemonVarsSet(Set llapDaemonVarsSetLocal
HIVE_QUERY_REEXECUTION_STATS_CACHE_SIZE("hive.query.reexecution.stats.cache.size",
100_000,
"Size of the runtime statistics cache. Unit is: OperatorStat entry; a
query plan consist ~100."),
HIVE_QUERY_PLANMAPPER_LINK_RELNODES("hive.query.planmapper.link.relnodes",
true,
-"Wether to link Calcite nodes to runtime statistics."),
+"Whether to link Calcite nodes to runtime statistics."),
+
+
HIVE_SCHEDULED_QUERIES_EXECUTOR_ENABLED("hive.scheduled.queries.executor.enabled",
true,
+"Controls whether HS2 will run scheduled query executor."),
+HIVE_SCHEDULED_QUERIES_NAMESPACE("hive.scheduled.queries.namespace",
"hive",
+"Sets the scheduled query namespace to be used. New scheduled queries
are created in this namespace;"
++ "and execution is also bound to the namespace"),
+
HIVE_SCHEDULED_QUERIES_EXECUTOR_IDLE_SLEEP_TIME("hive.scheduled.queries.executor.idle.sleep.time",
"60s",
+new TimeValidator(TimeUnit.SECONDS),
+"Time to sleep between quering for the presence of a scheduled
query."),
+
HIVE_SCHEDULED_QUERIES_EXECUTOR_PROGRESS_REPORT_INTERVAL("hive.scheduled.queries.executor.progress.report.interval",
+"60s",
+new TimeValidator(TimeUnit.SECONDS),
+"While scheduled queries are in flight; "
++ "a background update happens periodically to report the actual
state of the query"),
+
HIVE_SECURITY_AUTHORIZATION_SCHEDULED_QUERIES_SUPPORTED("hive.security.authorization.scheduled.queries.supported",
Review comment:
yes; because if the authorization doesn't support scheduled query requestes
- it will (most probably) just throw exceptions...
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 339478)
Time Spent: 7.5h (was: 7h 20m)
> Scheduled query support
> ---
>
> Key: HIVE-21884
> URL: https://issues.apache.org/jira/browse/HIVE-21884
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21844.04.patch, HIVE-21844.05.patch,
> HIVE-21844.06.patch, HIVE-21844.07.patch, HIVE-21844.08.patch,
> HIVE-21844.09.patch, HIVE-21844.15.patch, HIVE-21844.19.patch,
> HIVE-21844.21.patch, HIVE-21844.22.patch, HIVE-21884.01.patch,
> HIVE-21884.02.patch, HIVE-21884.03.patch, HIVE-21884.09.patch,
> HIVE-21884.10.patch, HIVE-21884.10.patch, HIVE-21884.11.patch,
> HIVE-21884.12.patch, HIVE-21884.13.patch, HIVE-21884.14.patch,
> HIVE-21884.14.patch, HIVE-21884.14.patch, HIVE-21884.16.patch,
> HIVE-21884.17.patch, HIVE-21884.18.patch, HIVE-21884.20.patch,
> HIVE-21884.23.patch, HIVE-21884.24.patch, HIVE-21884.25.patch,
> HIVE-21884.26.patch, Scheduled queries2.pdf
>
> Time Spent: 7.5h
> Remaining Estimate: 0h
>
> design document:
> https://docs.google.com/document/d/1mJSFdJi_1cbxJTXC9QvGw2rQ3zzJkNfxOO6b5esmyCE/edit#
> in case the google doc is not reachable: [^Scheduled queries2.pdf]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-21884) Scheduled query support
[
https://issues.apache.org/jira/browse/HIVE-21884?focusedWorklogId=339479=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-339479
]
ASF GitHub Bot logged work on HIVE-21884:
-
Author: ASF GitHub Bot
Created on: 06/Nov/19 16:44
Start Date: 06/Nov/19 16:44
Worklog Time Spent: 10m
Work Description: kgyrtkirk commented on pull request #794: HIVE-21884
URL: https://github.com/apache/hive/pull/794#discussion_r336005708
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/scheduled/ScheduledQueryExecutionService.java
##
@@ -106,6 +123,7 @@ private void processQuery(ScheduledQueryPollResponse q) {
}
}
+//FIXME transition?( )
Review comment:
it turned out that this "synchronized" and also a the lastState field is
also unneccessary...
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 339479)
Time Spent: 7h 40m (was: 7.5h)
> Scheduled query support
> ---
>
> Key: HIVE-21884
> URL: https://issues.apache.org/jira/browse/HIVE-21884
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21844.04.patch, HIVE-21844.05.patch,
> HIVE-21844.06.patch, HIVE-21844.07.patch, HIVE-21844.08.patch,
> HIVE-21844.09.patch, HIVE-21844.15.patch, HIVE-21844.19.patch,
> HIVE-21844.21.patch, HIVE-21844.22.patch, HIVE-21884.01.patch,
> HIVE-21884.02.patch, HIVE-21884.03.patch, HIVE-21884.09.patch,
> HIVE-21884.10.patch, HIVE-21884.10.patch, HIVE-21884.11.patch,
> HIVE-21884.12.patch, HIVE-21884.13.patch, HIVE-21884.14.patch,
> HIVE-21884.14.patch, HIVE-21884.14.patch, HIVE-21884.16.patch,
> HIVE-21884.17.patch, HIVE-21884.18.patch, HIVE-21884.20.patch,
> HIVE-21884.23.patch, HIVE-21884.24.patch, HIVE-21884.25.patch,
> HIVE-21884.26.patch, Scheduled queries2.pdf
>
> Time Spent: 7h 40m
> Remaining Estimate: 0h
>
> design document:
> https://docs.google.com/document/d/1mJSFdJi_1cbxJTXC9QvGw2rQ3zzJkNfxOO6b5esmyCE/edit#
> in case the google doc is not reachable: [^Scheduled queries2.pdf]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-21884) Scheduled query support
[
https://issues.apache.org/jira/browse/HIVE-21884?focusedWorklogId=339480=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-339480
]
ASF GitHub Bot logged work on HIVE-21884:
-
Author: ASF GitHub Bot
Created on: 06/Nov/19 16:44
Start Date: 06/Nov/19 16:44
Worklog Time Spent: 10m
Work Description: kgyrtkirk commented on pull request #794: HIVE-21884
URL: https://github.com/apache/hive/pull/794#discussion_r343173701
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/scheduled/ScheduledQueryExecutionService.java
##
@@ -0,0 +1,179 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hive.ql.scheduled;
+
+import java.io.Closeable;
+import java.io.IOException;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Executors;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.hadoop.hive.conf.HiveConf;
+import org.apache.hadoop.hive.conf.HiveConf.ConfVars;
+import org.apache.hadoop.hive.metastore.api.QueryState;
+import org.apache.hadoop.hive.metastore.api.ScheduledQueryKey;
+import org.apache.hadoop.hive.metastore.api.ScheduledQueryPollResponse;
+import org.apache.hadoop.hive.metastore.api.ScheduledQueryProgressInfo;
+import org.apache.hadoop.hive.ql.DriverFactory;
+import org.apache.hadoop.hive.ql.IDriver;
+import org.apache.hadoop.hive.ql.processors.CommandProcessorException;
+import org.apache.hadoop.hive.ql.session.SessionState;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.util.concurrent.ThreadFactoryBuilder;
+
+public class ScheduledQueryExecutionService implements Closeable {
+
+ private static final Logger LOG =
LoggerFactory.getLogger(ScheduledQueryExecutionService.class);
+
+ private ScheduledQueryExecutionContext context;
+ private ScheduledQueryExecutor worker;
+
+ public static ScheduledQueryExecutionService
startScheduledQueryExecutorService(HiveConf conf) {
+MetastoreBasedScheduledQueryService qService = new
MetastoreBasedScheduledQueryService(conf);
+ExecutorService executor =
+Executors.newCachedThreadPool(
+new
ThreadFactoryBuilder().setDaemon(true).setNameFormat("Scheduled Query Thread
%d").build());
+ScheduledQueryExecutionContext ctx = new
ScheduledQueryExecutionContext(executor, conf, qService);
+return new ScheduledQueryExecutionService(ctx);
+ }
+
+ public ScheduledQueryExecutionService(ScheduledQueryExecutionContext ctx) {
+context = ctx;
+ctx.executor.submit(worker = new ScheduledQueryExecutor());
+ctx.executor.submit(new ProgressReporter());
+ }
+
+ static boolean isTerminalState(QueryState state) {
+return state == QueryState.FINISHED || state == QueryState.ERRORED;
+ }
+
+ class ScheduledQueryExecutor implements Runnable {
+
+private ScheduledQueryProgressInfo info;
+
+@Override
+public void run() {
+ while (true) {
+ScheduledQueryPollResponse q =
context.schedulerService.scheduledQueryPoll();
+if (q.isSetExecutionId()) {
+ try{
+processQuery(q);
+ } catch (Throwable t) {
+LOG.error("Unexpected exception during scheduled query
processing", t);
+ }
+} else {
+ try {
+Thread.sleep(context.getIdleSleepTime());
+ } catch (InterruptedException e) {
+Thread.currentThread().interrupt();
+LOG.info("interrupted");
+break;
+ }
+}
+ }
+}
+
+public synchronized void reportQueryProgress() {
+ if (info != null) {
+LOG.info("Reporting query progress of {} as {} err:{}",
info.getScheduledExecutionId(), info.getState(),
+info.getErrorMessage());
+context.schedulerService.scheduledQueryProgress(info);
+if (isTerminalState(info.getState())) {
+ info = null;
+}
+ }
+}
+
+private void processQuery(ScheduledQueryPollResponse q) {
+ SessionState state = null;
+ try {
+HiveConf conf = new
[jira] [Work logged] (HIVE-21198) Introduce a database object reference class
[
https://issues.apache.org/jira/browse/HIVE-21198?focusedWorklogId=339473=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-339473
]
ASF GitHub Bot logged work on HIVE-21198:
-
Author: ASF GitHub Bot
Created on: 06/Nov/19 16:43
Start Date: 06/Nov/19 16:43
Worklog Time Spent: 10m
Work Description: kgyrtkirk commented on pull request #550: HIVE-21198
Introduce a database object reference class
URL: https://github.com/apache/hive/pull/550#discussion_r343195426
##
File path:
hcatalog/core/src/main/java/org/apache/hive/hcatalog/cli/SemanticAnalysis/HCatSemanticAnalyzer.java
##
@@ -336,7 +336,7 @@ protected void
authorizeDDLWork(HiveSemanticAnalyzerHookContext cntxt, Hive hive
} else if (ddlDesc instanceof AlterTableSetLocationDesc) {
AlterTableSetLocationDesc alterTable =
(AlterTableSetLocationDesc)ddlDesc;
Table table = hive.getTable(SessionState.get().getCurrentDatabase(),
- Utilities.getDbTableName(alterTable.getTableName())[1], false);
+ Utilities.getDbTableName(alterTable.getDbTableName())[1], false);
Review comment:
I think we might want to mark Utilities.getDbTableName as deprecated or
:dagger: it
can be done in a followup as well
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 339473)
Time Spent: 2.5h (was: 2h 20m)
> Introduce a database object reference class
> ---
>
> Key: HIVE-21198
> URL: https://issues.apache.org/jira/browse/HIVE-21198
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: David Lavati
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21198.1.patch, HIVE-21198.10.patch,
> HIVE-21198.2.patch, HIVE-21198.3.patch, HIVE-21198.4.patch,
> HIVE-21198.5.patch, HIVE-21198.6.patch, HIVE-21198.7.patch,
> HIVE-21198.8.patch, HIVE-21198.9.patch
>
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> There are many places in which "{databasename}.{tablename}" is passed as a
> single string; there are some places where the they travel as 2 separate
> arguments.
> Idea would be to introduce a simple immutable class with 2 fields ; and pass
> these informations together. Making this better is required if we would be
> wanting to enable dot in tablenames
> HIVE-16907, HIVE-21151
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-21198) Introduce a database object reference class
[
https://issues.apache.org/jira/browse/HIVE-21198?focusedWorklogId=339476=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-339476
]
ASF GitHub Bot logged work on HIVE-21198:
-
Author: ASF GitHub Bot
Created on: 06/Nov/19 16:43
Start Date: 06/Nov/19 16:43
Worklog Time Spent: 10m
Work Description: kgyrtkirk commented on pull request #550: HIVE-21198
Introduce a database object reference class
URL: https://github.com/apache/hive/pull/550#discussion_r343204479
##
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##
@@ -1331,6 +1331,21 @@ public Table getTable(final String dbName, final String
tableName) throws HiveEx
}
}
+ /**
+ * Returns metadata of the table
+ *
+ * @param tableName
+ * the tableName object
+ * @return the table
+ * @exception HiveException
+ * if there's an internal error or if the table doesn't exist
+ */
+ public Table getTable(TableName tableName) throws HiveException {
+
+return tableName.getDb() == null ? this.getTable(tableName.getTable(),
true) : this
Review comment:
a table without db is not invalid? is it only needed temporarilt - until
this whole thing is done?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 339476)
> Introduce a database object reference class
> ---
>
> Key: HIVE-21198
> URL: https://issues.apache.org/jira/browse/HIVE-21198
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: David Lavati
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21198.1.patch, HIVE-21198.10.patch,
> HIVE-21198.2.patch, HIVE-21198.3.patch, HIVE-21198.4.patch,
> HIVE-21198.5.patch, HIVE-21198.6.patch, HIVE-21198.7.patch,
> HIVE-21198.8.patch, HIVE-21198.9.patch
>
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> There are many places in which "{databasename}.{tablename}" is passed as a
> single string; there are some places where the they travel as 2 separate
> arguments.
> Idea would be to introduce a simple immutable class with 2 fields ; and pass
> these informations together. Making this better is required if we would be
> wanting to enable dot in tablenames
> HIVE-16907, HIVE-21151
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-21198) Introduce a database object reference class
[
https://issues.apache.org/jira/browse/HIVE-21198?focusedWorklogId=339474=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-339474
]
ASF GitHub Bot logged work on HIVE-21198:
-
Author: ASF GitHub Bot
Created on: 06/Nov/19 16:43
Start Date: 06/Nov/19 16:43
Worklog Time Spent: 10m
Work Description: kgyrtkirk commented on pull request #550: HIVE-21198
Introduce a database object reference class
URL: https://github.com/apache/hive/pull/550#discussion_r343199404
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/ddl/table/AbstractAlterTableDesc.java
##
@@ -62,8 +62,8 @@ public AlterTableType getType() {
}
@Explain(displayName = "table name", explainLevels = { Level.USER,
Level.DEFAULT, Level.EXTENDED })
- public String getTableName() {
-return tableName;
+ public String getDbTableName() {
Review comment:
in this file we have "FullTableName" and "DbTableName" - I think it would
help to define what is what in the TableName class or something; and the (try)
to use the same concepts consequently (of course this can be done later as
well)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 339474)
Time Spent: 2.5h (was: 2h 20m)
> Introduce a database object reference class
> ---
>
> Key: HIVE-21198
> URL: https://issues.apache.org/jira/browse/HIVE-21198
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: David Lavati
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21198.1.patch, HIVE-21198.10.patch,
> HIVE-21198.2.patch, HIVE-21198.3.patch, HIVE-21198.4.patch,
> HIVE-21198.5.patch, HIVE-21198.6.patch, HIVE-21198.7.patch,
> HIVE-21198.8.patch, HIVE-21198.9.patch
>
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> There are many places in which "{databasename}.{tablename}" is passed as a
> single string; there are some places where the they travel as 2 separate
> arguments.
> Idea would be to introduce a simple immutable class with 2 fields ; and pass
> these informations together. Making this better is required if we would be
> wanting to enable dot in tablenames
> HIVE-16907, HIVE-21151
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-21198) Introduce a database object reference class
[
https://issues.apache.org/jira/browse/HIVE-21198?focusedWorklogId=339472=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-339472
]
ASF GitHub Bot logged work on HIVE-21198:
-
Author: ASF GitHub Bot
Created on: 06/Nov/19 16:43
Start Date: 06/Nov/19 16:43
Worklog Time Spent: 10m
Work Description: kgyrtkirk commented on pull request #550: HIVE-21198
Introduce a database object reference class
URL: https://github.com/apache/hive/pull/550#discussion_r343202348
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/ddl/table/misc/TruncateTableDesc.java
##
@@ -39,8 +39,7 @@
public class TruncateTableDesc implements DDLDescWithWriteId, Serializable {
private static final long serialVersionUID = 1L;
- private final String tableName;
- private final String fullTableName;
Review comment:
yeah...this seems to be one place where this tablename git confusing ...
:+1:
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 339472)
Time Spent: 2h 20m (was: 2h 10m)
> Introduce a database object reference class
> ---
>
> Key: HIVE-21198
> URL: https://issues.apache.org/jira/browse/HIVE-21198
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: David Lavati
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21198.1.patch, HIVE-21198.10.patch,
> HIVE-21198.2.patch, HIVE-21198.3.patch, HIVE-21198.4.patch,
> HIVE-21198.5.patch, HIVE-21198.6.patch, HIVE-21198.7.patch,
> HIVE-21198.8.patch, HIVE-21198.9.patch
>
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> There are many places in which "{databasename}.{tablename}" is passed as a
> single string; there are some places where the they travel as 2 separate
> arguments.
> Idea would be to introduce a simple immutable class with 2 fields ; and pass
> these informations together. Making this better is required if we would be
> wanting to enable dot in tablenames
> HIVE-16907, HIVE-21151
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-21198) Introduce a database object reference class
[
https://issues.apache.org/jira/browse/HIVE-21198?focusedWorklogId=339477=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-339477
]
ASF GitHub Bot logged work on HIVE-21198:
-
Author: ASF GitHub Bot
Created on: 06/Nov/19 16:43
Start Date: 06/Nov/19 16:43
Worklog Time Spent: 10m
Work Description: kgyrtkirk commented on pull request #550: HIVE-21198
Introduce a database object reference class
URL: https://github.com/apache/hive/pull/550#discussion_r343195724
##
File path:
hcatalog/core/src/test/java/org/apache/hive/hcatalog/cli/TestSemanticAnalysis.java
##
@@ -403,7 +403,7 @@ public void testCTLPass() throws Exception {
query = "create table like_table like junit_sem_analysis";
hcatDriver.run(query);
//Table tbl = client.getTable(MetaStoreUtils.DEFAULT_DATABASE_NAME,
likeTbl);
-//assertEquals(likeTbl,tbl.getTableName());
+//assertEquals(likeTbl,tbl.getDbTableName());
Review comment:
:rofl:
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 339477)
Time Spent: 2h 40m (was: 2.5h)
> Introduce a database object reference class
> ---
>
> Key: HIVE-21198
> URL: https://issues.apache.org/jira/browse/HIVE-21198
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: David Lavati
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21198.1.patch, HIVE-21198.10.patch,
> HIVE-21198.2.patch, HIVE-21198.3.patch, HIVE-21198.4.patch,
> HIVE-21198.5.patch, HIVE-21198.6.patch, HIVE-21198.7.patch,
> HIVE-21198.8.patch, HIVE-21198.9.patch
>
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> There are many places in which "{databasename}.{tablename}" is passed as a
> single string; there are some places where the they travel as 2 separate
> arguments.
> Idea would be to introduce a simple immutable class with 2 fields ; and pass
> these informations together. Making this better is required if we would be
> wanting to enable dot in tablenames
> HIVE-16907, HIVE-21151
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-21198) Introduce a database object reference class
[
https://issues.apache.org/jira/browse/HIVE-21198?focusedWorklogId=339475=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-339475
]
ASF GitHub Bot logged work on HIVE-21198:
-
Author: ASF GitHub Bot
Created on: 06/Nov/19 16:43
Start Date: 06/Nov/19 16:43
Worklog Time Spent: 10m
Work Description: kgyrtkirk commented on pull request #550: HIVE-21198
Introduce a database object reference class
URL: https://github.com/apache/hive/pull/550#discussion_r343197173
##
File path:
metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreUtils.java
##
@@ -154,7 +154,7 @@ static public Deserializer getDeserializer(Configuration
conf,
ObjectInspector oi = deserializer.getObjectInspector();
String[] names = tableName.split("\\.");
String last_name = names[names.length - 1];
-for (int i = 1; i < names.length; i++) {
+for (int i = 2; i < names.length; i++) {
Review comment:
I'm not sure what's happening here ; but this =1 / =2 looks wierd
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 339475)
> Introduce a database object reference class
> ---
>
> Key: HIVE-21198
> URL: https://issues.apache.org/jira/browse/HIVE-21198
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: David Lavati
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21198.1.patch, HIVE-21198.10.patch,
> HIVE-21198.2.patch, HIVE-21198.3.patch, HIVE-21198.4.patch,
> HIVE-21198.5.patch, HIVE-21198.6.patch, HIVE-21198.7.patch,
> HIVE-21198.8.patch, HIVE-21198.9.patch
>
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> There are many places in which "{databasename}.{tablename}" is passed as a
> single string; there are some places where the they travel as 2 separate
> arguments.
> Idea would be to introduce a simple immutable class with 2 fields ; and pass
> these informations together. Making this better is required if we would be
> wanting to enable dot in tablenames
> HIVE-16907, HIVE-21151
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22327) Repl: Ignore read-only transactions in notification log
[
https://issues.apache.org/jira/browse/HIVE-22327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968480#comment-16968480
]
Hive QA commented on HIVE-22327:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
54s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
20s{color} | {color:blue} standalone-metastore/metastore-server in master has
173 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
42s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
49s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
29s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
26s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 7 new + 550 unchanged - 2 fixed = 557 total (was 552) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
15s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 3s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19315/dev-support/hive-personality.sh
|
| git revision | master / 4cfb7a0 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19315/yetus/diff-checkstyle-standalone-metastore_metastore-server.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19315/yetus/patch-asflicense-problems.txt
|
| modules | C: standalone-metastore/metastore-server itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19315/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Repl: Ignore read-only transactions in notification log
> ---
>
> Key: HIVE-22327
> URL: https://issues.apache.org/jira/browse/HIVE-22327
> Project: Hive
> Issue Type: Improvement
> Components: repl
>Reporter: Gopal Vijayaraghavan
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-22327.1.patch, HIVE-22327.2.patch
>
>
> Read txns need not be replicated.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22448) CBO: Expand the multiple count distinct with a group-by key
[
https://issues.apache.org/jira/browse/HIVE-22448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-22448:
---
Attachment: HIVE-22448.02.patch
> CBO: Expand the multiple count distinct with a group-by key
> ---
>
> Key: HIVE-22448
> URL: https://issues.apache.org/jira/browse/HIVE-22448
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Gopal Vijayaraghavan
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22448.01.patch, HIVE-22448.02.patch,
> HIVE-22448.02.patch, HIVE-22448.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {code}
> create temporary table mytable1 (x integer, y integer, z integer, a integer);
> explain cbo
> select z, x, count(distinct y), count(distinct a)
> from mytable1
> group by z, x;
> explain cbo
> select count(distinct y), count(distinct a)
> from mytable1
> {code}
> The first is not vectorized, the second one is because of the grouping-set
> based rewrite for count distinct.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22459) Hive datadiff function provided inconsistent results when hive.ferch.task.conversion is set to none
[
https://issues.apache.org/jira/browse/HIVE-22459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968474#comment-16968474
]
Chiran Ravani commented on HIVE-22459:
--
[~Rajkumar Singh] Thank you for the reply, I tried it on the master, and
observed the same problems.
> Hive datadiff function provided inconsistent results when
> hive.ferch.task.conversion is set to none
> ---
>
> Key: HIVE-22459
> URL: https://issues.apache.org/jira/browse/HIVE-22459
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 3.0.0
>Reporter: Chiran Ravani
>Priority: Critical
>
> Hive datadiff function provided inconsistent results when
> hive.ferch.task.conversion to more
> Below is output, whereas in Hive 1.2 the results are consistent
> Note: Same query works well on Hive 3 when hive.ferch.task.conversion is set
> to none
> Steps to reproduce the problem.
> {code:java}
> 0: jdbc:hive2://c1113-node2.squadron.support.> select datetimecol from
> testdatediff where datediff(cast(current_timestamp as string),
> datetimecol)<183;
> INFO : Compiling
> command(queryId=hive_20191105103636_1dff22a1-02f3-48a8-b076-0b91272f2268):
> select datetimecol from testdatediff where datediff(cast(current_timestamp as
> string), datetimecol)<183
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema:
> Schema(fieldSchemas:[FieldSchema(name:datetimecol, type:string,
> comment:null)], properties:null)
> INFO : Completed compiling
> command(queryId=hive_20191105103636_1dff22a1-02f3-48a8-b076-0b91272f2268);
> Time taken: 0.479 seconds
> INFO : Executing
> command(queryId=hive_20191105103636_1dff22a1-02f3-48a8-b076-0b91272f2268):
> select datetimecol from testdatediff where datediff(cast(current_timestamp as
> string), datetimecol)<183
> INFO : Completed executing
> command(queryId=hive_20191105103636_1dff22a1-02f3-48a8-b076-0b91272f2268);
> Time taken: 0.013 seconds
> INFO : OK
> +--+
> | datetimecol |
> +--+
> | 2019-07-24 |
> +--+
> 1 row selected (0.797 seconds)
> 0: jdbc:hive2://c1113-node2.squadron.support.>
> {code}
> After setting fetch task conversion as none.
> {code:java}
> 0: jdbc:hive2://c1113-node2.squadron.support.> set
> hive.fetch.task.conversion=none;
> No rows affected (0.017 seconds)
> 0: jdbc:hive2://c1113-node2.squadron.support.> set hive.fetch.task.conversion;
> +--+
> | set |
> +--+
> | hive.fetch.task.conversion=none |
> +--+
> 1 row selected (0.015 seconds)
> 0: jdbc:hive2://c1113-node2.squadron.support.> select datetimecol from
> testdatediff where datediff(cast(current_timestamp as string),
> datetimecol)<183;
> INFO : Compiling
> command(queryId=hive_20191105103709_0c38e446-09cf-45dd-9553-365146f42452):
> select datetimecol from testdatediff where datediff(cast(current_timestamp as
> string), datetimecol)<183
> ++
> | datetimecol |
> ++
> | 2019-09-09T10:45:49+02:00 |
> | 2019-07-24 |
> ++
> 2 rows selected (5.327 seconds)
> 0: jdbc:hive2://c1113-node2.squadron.support.>
> {code}
> Steps to reproduce
> {code}
> create external table testdatediff(datetimecol string) stored as orc;
> insert into testdatediff values ('2019-09-09T10:45:49+02:00'),('2019-07-24');
> select datetimecol from testdatediff where datediff(cast(current_timestamp as
> string), datetimecol)<183;
> set hive.ferch.task.conversion=none;
> select datetimecol from testdatediff where datediff(cast(current_timestamp as
> string), datetimecol)<183;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-21884) Scheduled query support
[ https://issues.apache.org/jira/browse/HIVE-21884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-21884: Attachment: HIVE-21884.26.patch > Scheduled query support > --- > > Key: HIVE-21884 > URL: https://issues.apache.org/jira/browse/HIVE-21884 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Labels: pull-request-available > Attachments: HIVE-21844.04.patch, HIVE-21844.05.patch, > HIVE-21844.06.patch, HIVE-21844.07.patch, HIVE-21844.08.patch, > HIVE-21844.09.patch, HIVE-21844.15.patch, HIVE-21844.19.patch, > HIVE-21844.21.patch, HIVE-21844.22.patch, HIVE-21884.01.patch, > HIVE-21884.02.patch, HIVE-21884.03.patch, HIVE-21884.09.patch, > HIVE-21884.10.patch, HIVE-21884.10.patch, HIVE-21884.11.patch, > HIVE-21884.12.patch, HIVE-21884.13.patch, HIVE-21884.14.patch, > HIVE-21884.14.patch, HIVE-21884.14.patch, HIVE-21884.16.patch, > HIVE-21884.17.patch, HIVE-21884.18.patch, HIVE-21884.20.patch, > HIVE-21884.23.patch, HIVE-21884.24.patch, HIVE-21884.25.patch, > HIVE-21884.26.patch, Scheduled queries2.pdf > > Time Spent: 7h 20m > Remaining Estimate: 0h > > design document: > https://docs.google.com/document/d/1mJSFdJi_1cbxJTXC9QvGw2rQ3zzJkNfxOO6b5esmyCE/edit# > in case the google doc is not reachable: [^Scheduled queries2.pdf] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-15157) Partition Table With timestamp type on S3 storage --> Error in getting fields from serde.Invalid Field null
[ https://issues.apache.org/jira/browse/HIVE-15157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968470#comment-16968470 ] Jesus Camacho Rodriguez commented on HIVE-15157: [~prasanth_j], can you review it? https://github.com/apache/hive/pull/840 Thanks > Partition Table With timestamp type on S3 storage --> Error in getting fields > from serde.Invalid Field null > --- > > Key: HIVE-15157 > URL: https://issues.apache.org/jira/browse/HIVE-15157 > Project: Hive > Issue Type: Bug > Components: Clients >Affects Versions: 2.1.0 > Environment: JDK 1.8 101 >Reporter: thauvin damien >Assignee: Jesus Camacho Rodriguez >Priority: Critical > Labels: pull-request-available, timestamp > Attachments: HIVE-15157.01.patch, HIVE-15157.02.patch, > HIVE-15157.03.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Hello > I get the error above when i try to perform : > hive> DESCRIBE formatted table partition (tsbucket='2016-10-28 16%3A00%3A00'); > FAILED: Execution Error, return code 1 from > org.apache.hadoop.hive.ql.exec.DDLTask. Error in getting fields from > serde.Invalid Field null > Here is the description of the issue. > --External table Hive with dynamic partition enable on Aws S3 storage. > --Partition Table with timestamp type . > When i perform "show partition table;" everything is fine : > hive> show partitions table; > OK > tsbucket=2016-10-01 11%3A00%3A00 > tsbucket=2016-10-28 16%3A00%3A00 > And when i perform "describe FORMATTED table;" everything is fine > Is this a bug ? > The stacktrace of hive.log : > 2016-11-08T10:30:20,868 ERROR [ac3e0d48-22c5-4d04-a788-aeb004ea94f3 > main([])]: exec.DDLTask (DDLTask.java:failed(574)) - > org.apache.hadoop.hive.ql.metadata.HiveException: Error in getting fields > from serde.Invalid Field null > at > org.apache.hadoop.hive.ql.metadata.Hive.getFieldsFromDeserializer(Hive.java:3414) > at > org.apache.hadoop.hive.ql.exec.DDLTask.describeTable(DDLTask.java:3109) > at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:408) > at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197) > at > org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100) > at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1858) > at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1562) > at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1313) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1084) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1072) > at > org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:232) > at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:183) > at > org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:399) > at > org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:776) > at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:714) > at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:641) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.apache.hadoop.util.RunJar.run(RunJar.java:221) > at org.apache.hadoop.util.RunJar.main(RunJar.java:136) > Caused by: MetaException(message:Invalid Field null) > at > org.apache.hadoop.hive.metastore.MetaStoreUtils.getFieldsFromDeserializer(MetaStoreUtils.java:1336) > at > org.apache.hadoop.hive.ql.metadata.Hive.getFieldsFromDeserializer(Hive.java:3409) > ... 21 more -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-22311) Propagate min/max column values from statistics to the optimizer for timestamp type
[ https://issues.apache.org/jira/browse/HIVE-22311?focusedWorklogId=339438=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-339438 ] ASF GitHub Bot logged work on HIVE-22311: - Author: ASF GitHub Bot Created on: 06/Nov/19 15:55 Start Date: 06/Nov/19 15:55 Worklog Time Spent: 10m Work Description: asfgit commented on pull request #821: HIVE-22311 URL: https://github.com/apache/hive/pull/821 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 339438) Time Spent: 40m (was: 0.5h) > Propagate min/max column values from statistics to the optimizer for > timestamp type > --- > > Key: HIVE-22311 > URL: https://issues.apache.org/jira/browse/HIVE-22311 > Project: Hive > Issue Type: Bug > Components: Statistics >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22311.01.patch, HIVE-22311.01.patch, > HIVE-22311.02.patch, HIVE-22311.03.patch, HIVE-22311.04.patch, > HIVE-22311.patch > > Time Spent: 40m > Remaining Estimate: 0h > > Currently stats annotation does not consider timestamp type e.g. for > estimates with range predicates. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-21198) Introduce a database object reference class
[
https://issues.apache.org/jira/browse/HIVE-21198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Lavati updated HIVE-21198:
Attachment: HIVE-21198.10.patch
> Introduce a database object reference class
> ---
>
> Key: HIVE-21198
> URL: https://issues.apache.org/jira/browse/HIVE-21198
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: David Lavati
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21198.1.patch, HIVE-21198.10.patch,
> HIVE-21198.2.patch, HIVE-21198.3.patch, HIVE-21198.4.patch,
> HIVE-21198.5.patch, HIVE-21198.6.patch, HIVE-21198.7.patch,
> HIVE-21198.8.patch, HIVE-21198.9.patch
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> There are many places in which "{databasename}.{tablename}" is passed as a
> single string; there are some places where the they travel as 2 separate
> arguments.
> Idea would be to introduce a simple immutable class with 2 fields ; and pass
> these informations together. Making this better is required if we would be
> wanting to enable dot in tablenames
> HIVE-16907, HIVE-21151
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22465) Add ssl conf in TezConfigurationFactory
[
https://issues.apache.org/jira/browse/HIVE-22465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968455#comment-16968455
]
Hive QA commented on HIVE-22465:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12985044/HIVE-22465.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17569 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19314/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19314/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19314/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12985044 - PreCommit-HIVE-Build
> Add ssl conf in TezConfigurationFactory
> ---
>
> Key: HIVE-22465
> URL: https://issues.apache.org/jira/browse/HIVE-22465
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 4.0.0
>Reporter: Rajesh Balamohan
>Assignee: Rajesh Balamohan
>Priority: Minor
> Attachments: HIVE-22465.1.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22414) Make LLAP CacheTags more memory efficient
[ https://issues.apache.org/jira/browse/HIVE-22414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968441#comment-16968441 ] Ádám Szita commented on HIVE-22414: --- Also included a feature so that partition keys and values become separable through encoding - this could be beneficial if e.g. in the future we'd want to filter buffers based on partition (values). > Make LLAP CacheTags more memory efficient > - > > Key: HIVE-22414 > URL: https://issues.apache.org/jira/browse/HIVE-22414 > Project: Hive > Issue Type: Improvement > Components: llap >Reporter: Ádám Szita >Assignee: Ádám Szita >Priority: Major > Attachments: HIVE-22414.0.patch, HIVE-22414.1.patch, > HIVE-22414.2.patch, HIVE-22414.3.patch, HIVE-22414.4.patch, > HIVE-22414.5.patch, HIVE-22414.6.patch > > > MultiPartitionCacheTag relies on LinkedLists. > A LinkedList object that holds e.g. 2 nodes consumes 112 bytes roughly in > this composition: > * 16 bytes for LinkedList object header > * 8 bytes for referring head > * 8 bytes for referring tail > * 80 = 2 x (16 bytes for LinkedList$Node header, 3 x 8 bytes for referring > prev, next, item) > This is a lot, so I propose to replace LinkedList in MultiPartitionCacheTag > with a simple String array. (For a similar scenario an array would take 16 + > 8 + 2 x 8 = 40 bytes, as per header, count, and 2 references for our actual > objects). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22414) Make LLAP CacheTags more memory efficient
[ https://issues.apache.org/jira/browse/HIVE-22414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ádám Szita updated HIVE-22414: -- Attachment: HIVE-22414.6.patch > Make LLAP CacheTags more memory efficient > - > > Key: HIVE-22414 > URL: https://issues.apache.org/jira/browse/HIVE-22414 > Project: Hive > Issue Type: Improvement > Components: llap >Reporter: Ádám Szita >Assignee: Ádám Szita >Priority: Major > Attachments: HIVE-22414.0.patch, HIVE-22414.1.patch, > HIVE-22414.2.patch, HIVE-22414.3.patch, HIVE-22414.4.patch, > HIVE-22414.5.patch, HIVE-22414.6.patch > > > MultiPartitionCacheTag relies on LinkedLists. > A LinkedList object that holds e.g. 2 nodes consumes 112 bytes roughly in > this composition: > * 16 bytes for LinkedList object header > * 8 bytes for referring head > * 8 bytes for referring tail > * 80 = 2 x (16 bytes for LinkedList$Node header, 3 x 8 bytes for referring > prev, next, item) > This is a lot, so I propose to replace LinkedList in MultiPartitionCacheTag > with a simple String array. (For a similar scenario an array would take 16 + > 8 + 2 x 8 = 40 bytes, as per header, count, and 2 references for our actual > objects). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22461) NPE Metastore Transformer
[
https://issues.apache.org/jira/browse/HIVE-22461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yongzhi Chen updated HIVE-22461:
Status: Patch Available (was: In Progress)
> NPE Metastore Transformer
> -
>
> Key: HIVE-22461
> URL: https://issues.apache.org/jira/browse/HIVE-22461
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.1.2
>Reporter: Yongzhi Chen
>Assignee: Yongzhi Chen
>Priority: Major
> Attachments: HIVE-22461.1.patch
>
>
> The stack looks as following:
> {noformat}
> 2019-10-08 18:09:12,198 INFO
> org.apache.hadoop.hive.metastore.MetastoreDefaultTransformer:
> [pool-6-thread-328]: Starting translation for processor
> Hiveserver2#3.1.2000.7.0.2.0...@vc0732.halxg.cloudera.com on list 1
> 2019-10-08 18:09:12,198 ERROR
> org.apache.hadoop.hive.metastore.RetryingHMSHandler: [pool-6-thread-328]:
> java.lang.NullPointerException
> at
> org.apache.hadoop.hive.metastore.MetastoreDefaultTransformer.transform(MetastoreDefaultTransformer.java:99)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getTableInternal(HiveMetaStore.java:3391)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table_req(HiveMetaStore.java:3352)
> at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
> at com.sun.proxy.$Proxy28.get_table_req(Unknown Source)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_table_req.getResult(ThriftHiveMetastore.java:16633)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_table_req.getResult(ThriftHiveMetastore.java:16617)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> at
> org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge.java:636)
> at
> org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge.java:631)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
> at
> org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:631)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> 2019-10-08 18:09:12,199 ERROR org.apache.thrift.server.TThreadPoolServer:
> [pool-6-thread-328]: Error occurred during processing of message.
> java.lang.NullPointerException: null
> at
> org.apache.hadoop.hive.metastore.MetastoreDefaultTransformer.transform(MetastoreDefaultTransformer.java:99)
> ~[hive-exec-3.1.2000.7.0.2.0-59.jar:3.1.2000.7.0.2.0-59]
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getTableInternal(HiveMetaStore.java:3391)
> ~[hive-exec-3.1.2000.7.0.2.0-59.jar:3.1.2000.7.0.2.0-59]
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table_req(HiveMetaStore.java:3352)
> ~[hive-exec-3.1.2000.7.0.2.0-59.jar:3.1.2000.7.0.2.0-59]
> at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source) ~[?:?]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_141]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_141]
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
> ~[hive-exec-3.1.2000.7.0.2.0-59.jar:3.1.2000.7.0.2.0-59]
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
> ~[hive-exec-3.1.2000.7.0.2.0-59.jar:3.1.2000.7.0.2.0-59]
> at com.sun.proxy.$Proxy28.get_table_req(Unknown Source) ~[?:?]
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_table_req.getResult(ThriftHiveMetastore.java:16633)
> ~[hive-exec-3.1.2000.7.0.2.0-59.jar:3.1.2000.7.0.2.0-59]
> at
>
[jira] [Commented] (HIVE-22461) NPE Metastore Transformer
[
https://issues.apache.org/jira/browse/HIVE-22461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968422#comment-16968422
]
Yongzhi Chen commented on HIVE-22461:
-
Attached patch to check if isSetSd to prevent NPE during HMS transformer
transform.
> NPE Metastore Transformer
> -
>
> Key: HIVE-22461
> URL: https://issues.apache.org/jira/browse/HIVE-22461
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.1.2
>Reporter: Yongzhi Chen
>Assignee: Yongzhi Chen
>Priority: Major
> Attachments: HIVE-22461.1.patch
>
>
> The stack looks as following:
> {noformat}
> 2019-10-08 18:09:12,198 INFO
> org.apache.hadoop.hive.metastore.MetastoreDefaultTransformer:
> [pool-6-thread-328]: Starting translation for processor
> Hiveserver2#3.1.2000.7.0.2.0...@vc0732.halxg.cloudera.com on list 1
> 2019-10-08 18:09:12,198 ERROR
> org.apache.hadoop.hive.metastore.RetryingHMSHandler: [pool-6-thread-328]:
> java.lang.NullPointerException
> at
> org.apache.hadoop.hive.metastore.MetastoreDefaultTransformer.transform(MetastoreDefaultTransformer.java:99)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getTableInternal(HiveMetaStore.java:3391)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table_req(HiveMetaStore.java:3352)
> at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
> at com.sun.proxy.$Proxy28.get_table_req(Unknown Source)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_table_req.getResult(ThriftHiveMetastore.java:16633)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_table_req.getResult(ThriftHiveMetastore.java:16617)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> at
> org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge.java:636)
> at
> org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge.java:631)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
> at
> org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:631)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> 2019-10-08 18:09:12,199 ERROR org.apache.thrift.server.TThreadPoolServer:
> [pool-6-thread-328]: Error occurred during processing of message.
> java.lang.NullPointerException: null
> at
> org.apache.hadoop.hive.metastore.MetastoreDefaultTransformer.transform(MetastoreDefaultTransformer.java:99)
> ~[hive-exec-3.1.2000.7.0.2.0-59.jar:3.1.2000.7.0.2.0-59]
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getTableInternal(HiveMetaStore.java:3391)
> ~[hive-exec-3.1.2000.7.0.2.0-59.jar:3.1.2000.7.0.2.0-59]
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table_req(HiveMetaStore.java:3352)
> ~[hive-exec-3.1.2000.7.0.2.0-59.jar:3.1.2000.7.0.2.0-59]
> at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source) ~[?:?]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_141]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_141]
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
> ~[hive-exec-3.1.2000.7.0.2.0-59.jar:3.1.2000.7.0.2.0-59]
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
> ~[hive-exec-3.1.2000.7.0.2.0-59.jar:3.1.2000.7.0.2.0-59]
> at com.sun.proxy.$Proxy28.get_table_req(Unknown Source) ~[?:?]
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_table_req.getResult(ThriftHiveMetastore.java:16633)
>
[jira] [Commented] (HIVE-22465) Add ssl conf in TezConfigurationFactory
[
https://issues.apache.org/jira/browse/HIVE-22465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968423#comment-16968423
]
Hive QA commented on HIVE-22465:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
12s{color} | {color:blue} ql in master has 1550 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 31s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19314/dev-support/hive-personality.sh
|
| git revision | master / d0bd071 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19314/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Add ssl conf in TezConfigurationFactory
> ---
>
> Key: HIVE-22465
> URL: https://issues.apache.org/jira/browse/HIVE-22465
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 4.0.0
>Reporter: Rajesh Balamohan
>Assignee: Rajesh Balamohan
>Priority: Minor
> Attachments: HIVE-22465.1.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22461) NPE Metastore Transformer
[
https://issues.apache.org/jira/browse/HIVE-22461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yongzhi Chen updated HIVE-22461:
Attachment: HIVE-22461.1.patch
> NPE Metastore Transformer
> -
>
> Key: HIVE-22461
> URL: https://issues.apache.org/jira/browse/HIVE-22461
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.1.2
>Reporter: Yongzhi Chen
>Assignee: Yongzhi Chen
>Priority: Major
> Attachments: HIVE-22461.1.patch
>
>
> The stack looks as following:
> {noformat}
> 2019-10-08 18:09:12,198 INFO
> org.apache.hadoop.hive.metastore.MetastoreDefaultTransformer:
> [pool-6-thread-328]: Starting translation for processor
> Hiveserver2#3.1.2000.7.0.2.0...@vc0732.halxg.cloudera.com on list 1
> 2019-10-08 18:09:12,198 ERROR
> org.apache.hadoop.hive.metastore.RetryingHMSHandler: [pool-6-thread-328]:
> java.lang.NullPointerException
> at
> org.apache.hadoop.hive.metastore.MetastoreDefaultTransformer.transform(MetastoreDefaultTransformer.java:99)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getTableInternal(HiveMetaStore.java:3391)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table_req(HiveMetaStore.java:3352)
> at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
> at com.sun.proxy.$Proxy28.get_table_req(Unknown Source)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_table_req.getResult(ThriftHiveMetastore.java:16633)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_table_req.getResult(ThriftHiveMetastore.java:16617)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> at
> org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge.java:636)
> at
> org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge.java:631)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
> at
> org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:631)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> 2019-10-08 18:09:12,199 ERROR org.apache.thrift.server.TThreadPoolServer:
> [pool-6-thread-328]: Error occurred during processing of message.
> java.lang.NullPointerException: null
> at
> org.apache.hadoop.hive.metastore.MetastoreDefaultTransformer.transform(MetastoreDefaultTransformer.java:99)
> ~[hive-exec-3.1.2000.7.0.2.0-59.jar:3.1.2000.7.0.2.0-59]
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getTableInternal(HiveMetaStore.java:3391)
> ~[hive-exec-3.1.2000.7.0.2.0-59.jar:3.1.2000.7.0.2.0-59]
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table_req(HiveMetaStore.java:3352)
> ~[hive-exec-3.1.2000.7.0.2.0-59.jar:3.1.2000.7.0.2.0-59]
> at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source) ~[?:?]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_141]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_141]
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
> ~[hive-exec-3.1.2000.7.0.2.0-59.jar:3.1.2000.7.0.2.0-59]
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
> ~[hive-exec-3.1.2000.7.0.2.0-59.jar:3.1.2000.7.0.2.0-59]
> at com.sun.proxy.$Proxy28.get_table_req(Unknown Source) ~[?:?]
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_table_req.getResult(ThriftHiveMetastore.java:16633)
> ~[hive-exec-3.1.2000.7.0.2.0-59.jar:3.1.2000.7.0.2.0-59]
> at
>
[jira] [Work started] (HIVE-22461) NPE Metastore Transformer
[
https://issues.apache.org/jira/browse/HIVE-22461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HIVE-22461 started by Yongzhi Chen.
---
> NPE Metastore Transformer
> -
>
> Key: HIVE-22461
> URL: https://issues.apache.org/jira/browse/HIVE-22461
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.1.2
>Reporter: Yongzhi Chen
>Assignee: Yongzhi Chen
>Priority: Major
>
> The stack looks as following:
> {noformat}
> 2019-10-08 18:09:12,198 INFO
> org.apache.hadoop.hive.metastore.MetastoreDefaultTransformer:
> [pool-6-thread-328]: Starting translation for processor
> Hiveserver2#3.1.2000.7.0.2.0...@vc0732.halxg.cloudera.com on list 1
> 2019-10-08 18:09:12,198 ERROR
> org.apache.hadoop.hive.metastore.RetryingHMSHandler: [pool-6-thread-328]:
> java.lang.NullPointerException
> at
> org.apache.hadoop.hive.metastore.MetastoreDefaultTransformer.transform(MetastoreDefaultTransformer.java:99)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getTableInternal(HiveMetaStore.java:3391)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table_req(HiveMetaStore.java:3352)
> at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
> at com.sun.proxy.$Proxy28.get_table_req(Unknown Source)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_table_req.getResult(ThriftHiveMetastore.java:16633)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_table_req.getResult(ThriftHiveMetastore.java:16617)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> at
> org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge.java:636)
> at
> org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge.java:631)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
> at
> org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:631)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> 2019-10-08 18:09:12,199 ERROR org.apache.thrift.server.TThreadPoolServer:
> [pool-6-thread-328]: Error occurred during processing of message.
> java.lang.NullPointerException: null
> at
> org.apache.hadoop.hive.metastore.MetastoreDefaultTransformer.transform(MetastoreDefaultTransformer.java:99)
> ~[hive-exec-3.1.2000.7.0.2.0-59.jar:3.1.2000.7.0.2.0-59]
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getTableInternal(HiveMetaStore.java:3391)
> ~[hive-exec-3.1.2000.7.0.2.0-59.jar:3.1.2000.7.0.2.0-59]
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table_req(HiveMetaStore.java:3352)
> ~[hive-exec-3.1.2000.7.0.2.0-59.jar:3.1.2000.7.0.2.0-59]
> at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source) ~[?:?]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_141]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_141]
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
> ~[hive-exec-3.1.2000.7.0.2.0-59.jar:3.1.2000.7.0.2.0-59]
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
> ~[hive-exec-3.1.2000.7.0.2.0-59.jar:3.1.2000.7.0.2.0-59]
> at com.sun.proxy.$Proxy28.get_table_req(Unknown Source) ~[?:?]
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_table_req.getResult(ThriftHiveMetastore.java:16633)
> ~[hive-exec-3.1.2000.7.0.2.0-59.jar:3.1.2000.7.0.2.0-59]
> at
>
[jira] [Updated] (HIVE-22444) Clean up Project POM Files
[ https://issues.apache.org/jira/browse/HIVE-22444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-22444: -- Attachment: HIVE-22444.8.patch > Clean up Project POM Files > -- > > Key: HIVE-22444 > URL: https://issues.apache.org/jira/browse/HIVE-22444 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Attachments: HIVE-22444.1.patch, HIVE-22444.2.patch, > HIVE-22444.3.patch, HIVE-22444.4.patch, HIVE-22444.5.patch, > HIVE-22444.6.patch, HIVE-22444.7.patch, HIVE-22444.8.patch > > > # Address warnings in the build process > # Use DependencyManagement in Root POM for ITest (see HIVE-22426) > # General POM cleanup -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22444) Clean up Project POM Files
[ https://issues.apache.org/jira/browse/HIVE-22444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-22444: -- Status: Patch Available (was: Open) > Clean up Project POM Files > -- > > Key: HIVE-22444 > URL: https://issues.apache.org/jira/browse/HIVE-22444 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Attachments: HIVE-22444.1.patch, HIVE-22444.2.patch, > HIVE-22444.3.patch, HIVE-22444.4.patch, HIVE-22444.5.patch, > HIVE-22444.6.patch, HIVE-22444.7.patch, HIVE-22444.8.patch > > > # Address warnings in the build process > # Use DependencyManagement in Root POM for ITest (see HIVE-22426) > # General POM cleanup -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22444) Clean up Project POM Files
[ https://issues.apache.org/jira/browse/HIVE-22444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-22444: -- Status: Open (was: Patch Available) > Clean up Project POM Files > -- > > Key: HIVE-22444 > URL: https://issues.apache.org/jira/browse/HIVE-22444 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Attachments: HIVE-22444.1.patch, HIVE-22444.2.patch, > HIVE-22444.3.patch, HIVE-22444.4.patch, HIVE-22444.5.patch, > HIVE-22444.6.patch, HIVE-22444.7.patch > > > # Address warnings in the build process > # Use DependencyManagement in Root POM for ITest (see HIVE-22426) > # General POM cleanup -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22405) Add ColumnVector support for ProlepticCalendar
[
https://issues.apache.org/jira/browse/HIVE-22405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968391#comment-16968391
]
Hive QA commented on HIVE-22405:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12985038/HIVE-22405.01.patch
{color:green}SUCCESS:{color} +1 due to 5 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 17571 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[date_udf] (batchId=34)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_case_when_1]
(batchId=94)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorized_date_funcs]
(batchId=85)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_text_vecrow_part_llap_io]
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_text_vecrow_table_llap_io]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_scalar]
(batchId=172)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[udaf_collect_set_2]
(batchId=180)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_case_when_1]
(batchId=186)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorized_date_funcs]
(batchId=183)
org.apache.hadoop.hive.llap.security.TestLlapSignerImpl.testSigning
(batchId=364)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19313/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19313/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19313/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 10 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12985038 - PreCommit-HIVE-Build
> Add ColumnVector support for ProlepticCalendar
> --
>
> Key: HIVE-22405
> URL: https://issues.apache.org/jira/browse/HIVE-22405
> Project: Hive
> Issue Type: Bug
>Reporter: Owen O'Malley
>Assignee: László Bodor
>Priority: Major
> Attachments: HIVE-22405.01.patch
>
>
> Hive recently moved its processing to the proleptic calendar, which has
> created some issues for users who have dates before 1580 AD.
> I'd propose extending the column vectors for times & dates to encode which
> calendar they are using.
> * create DateColumnVector that extends LongColumnVector
> * add a method to change calendars to both DateColumnVector and
> TimestampColumnVector.
> {code}
> /**
>* Change the calendar to or from proleptic. If the new and old values of
> the flag are the
>* same, nothing is done.
>* useProleptic - set the flag for the proleptic calendar
>* updateData - change the data to match the new value of the flag.
>*/
> void changeCalendar(useProleptic: boolean, updateData: boolean);
> /**
>* Detect whether this data is using the proleptic calendar.
>*/
> boolean usingProlepticCalendar();
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22405) Add ColumnVector support for ProlepticCalendar
[
https://issues.apache.org/jira/browse/HIVE-22405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968367#comment-16968367
]
Hive QA commented on HIVE-22405:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
2s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
57s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
24s{color} | {color:blue} storage-api in master has 48 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
14s{color} | {color:blue} ql in master has 1550 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
16s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
12s{color} | {color:red} storage-api: The patch generated 4 new + 12 unchanged
- 6 fixed = 16 total (was 18) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
43s{color} | {color:red} ql: The patch generated 2 new + 379 unchanged - 1
fixed = 381 total (was 380) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 2 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
31s{color} | {color:red} storage-api generated 8 new + 48 unchanged - 0 fixed =
56 total (was 48) {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
22s{color} | {color:red} ql generated 2 new + 1548 unchanged - 2 fixed = 1550
total (was 1550) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 29m 51s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:storage-api |
| | Call to method of static java.text.DateFormat in
org.apache.hadoop.hive.ql.exec.vector.DateColumnVector.updateDataAccordingProlepticSetting()
At DateColumnVector.java:java.text.DateFormat in
org.apache.hadoop.hive.ql.exec.vector.DateColumnVector.updateDataAccordingProlepticSetting()
At DateColumnVector.java:[line 72] |
| | Call to method of static java.text.DateFormat in
org.apache.hadoop.hive.ql.exec.vector.DateColumnVector.updateDataAccordingProlepticSetting()
At DateColumnVector.java:java.text.DateFormat in
org.apache.hadoop.hive.ql.exec.vector.DateColumnVector.updateDataAccordingProlepticSetting()
At DateColumnVector.java:[line 71] |
| | org.apache.hadoop.hive.ql.exec.vector.DateColumnVector.GREGORIAN_CALENDAR
is static field of type java.util.Calendar, which isn't thread safe In
DateColumnVector.java:type java.util.Calendar, which isn't thread safe In
DateColumnVector.java |
| |
org.apache.hadoop.hive.ql.exec.vector.DateColumnVector.PROLEPTIC_GREGORIAN_CALENDAR
is static field of type java.util.Calendar, which isn't thread safe In
DateColumnVector.java:type java.util.Calendar, which isn't thread safe In
DateColumnVector.java |
| | Call to method of static java.text.DateFormat in
[jira] [Commented] (HIVE-22374) Upgrade commons-compress version to 1.19
[
https://issues.apache.org/jira/browse/HIVE-22374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968363#comment-16968363
]
Sumin Byeon commented on HIVE-22374:
[~abstractdog], it's sum...@gmail.com.
> Upgrade commons-compress version to 1.19
>
>
> Key: HIVE-22374
> URL: https://issues.apache.org/jira/browse/HIVE-22374
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Sumin Byeon
>Assignee: Sumin Byeon
>Priority: Critical
> Fix For: 4.0.0
>
> Attachments: HIVE-22374.1.patch, HIVE-22374.2.patch
>
>
> As described in
> [CVE-2019-12402|https://nvd.nist.gov/vuln/detail/CVE-2019-12402],
> {{commons-compress:1.18}} has an issue where certain inputs may cause an
> infinite loop which leads to a denial of service attack.
> This patch simply upgrades {{common-compress}} versions from 1.18 to 1.19
> which is the latest minor version at the date of filing this issue ([Maven
> repo|https://mvnrepository.com/artifact/org.apache.commons/commons-compress]).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22374) Upgrade commons-compress version to 1.19
[
https://issues.apache.org/jira/browse/HIVE-22374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968355#comment-16968355
]
László Bodor commented on HIVE-22374:
-
[~shortbread]: could you please provide your email which is assigned to your
apache jira account? (needed for commit author)
> Upgrade commons-compress version to 1.19
>
>
> Key: HIVE-22374
> URL: https://issues.apache.org/jira/browse/HIVE-22374
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Sumin Byeon
>Assignee: Sumin Byeon
>Priority: Critical
> Fix For: 4.0.0
>
> Attachments: HIVE-22374.1.patch, HIVE-22374.2.patch
>
>
> As described in
> [CVE-2019-12402|https://nvd.nist.gov/vuln/detail/CVE-2019-12402],
> {{commons-compress:1.18}} has an issue where certain inputs may cause an
> infinite loop which leads to a denial of service attack.
> This patch simply upgrades {{common-compress}} versions from 1.18 to 1.19
> which is the latest minor version at the date of filing this issue ([Maven
> repo|https://mvnrepository.com/artifact/org.apache.commons/commons-compress]).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22464) Implement support for NULLS FIRST/LAST in TopNKeyOperator
[ https://issues.apache.org/jira/browse/HIVE-22464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-22464: -- Attachment: HIVE-22464.2.patch > Implement support for NULLS FIRST/LAST in TopNKeyOperator > - > > Key: HIVE-22464 > URL: https://issues.apache.org/jira/browse/HIVE-22464 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Reporter: Krisztian Kasa >Assignee: Krisztian Kasa >Priority: Major > Attachments: HIVE-22464.1.patch, HIVE-22464.2.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22464) Implement support for NULLS FIRST/LAST in TopNKeyOperator
[ https://issues.apache.org/jira/browse/HIVE-22464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-22464: -- Status: Open (was: Patch Available) > Implement support for NULLS FIRST/LAST in TopNKeyOperator > - > > Key: HIVE-22464 > URL: https://issues.apache.org/jira/browse/HIVE-22464 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Reporter: Krisztian Kasa >Assignee: Krisztian Kasa >Priority: Major > Attachments: HIVE-22464.1.patch, HIVE-22464.2.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22464) Implement support for NULLS FIRST/LAST in TopNKeyOperator
[ https://issues.apache.org/jira/browse/HIVE-22464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-22464: -- Status: Patch Available (was: Open) > Implement support for NULLS FIRST/LAST in TopNKeyOperator > - > > Key: HIVE-22464 > URL: https://issues.apache.org/jira/browse/HIVE-22464 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Reporter: Krisztian Kasa >Assignee: Krisztian Kasa >Priority: Major > Attachments: HIVE-22464.1.patch, HIVE-22464.2.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22374) Upgrade commons-compress version to 1.19
[
https://issues.apache.org/jira/browse/HIVE-22374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
László Bodor updated HIVE-22374:
Fix Version/s: 4.0.0
> Upgrade commons-compress version to 1.19
>
>
> Key: HIVE-22374
> URL: https://issues.apache.org/jira/browse/HIVE-22374
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Sumin Byeon
>Assignee: Sumin Byeon
>Priority: Critical
> Fix For: 4.0.0
>
> Attachments: HIVE-22374.1.patch, HIVE-22374.2.patch
>
>
> As described in
> [CVE-2019-12402|https://nvd.nist.gov/vuln/detail/CVE-2019-12402],
> {{commons-compress:1.18}} has an issue where certain inputs may cause an
> infinite loop which leads to a denial of service attack.
> This patch simply upgrades {{common-compress}} versions from 1.18 to 1.19
> which is the latest minor version at the date of filing this issue ([Maven
> repo|https://mvnrepository.com/artifact/org.apache.commons/commons-compress]).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22464) Implement support for NULLS FIRST/LAST in TopNKeyOperator
[
https://issues.apache.org/jira/browse/HIVE-22464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968339#comment-16968339
]
Hive QA commented on HIVE-22464:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12985037/HIVE-22464.1.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 17569 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[topnkey] (batchId=45)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[topnkey]
(batchId=111)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19312/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19312/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19312/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12985037 - PreCommit-HIVE-Build
> Implement support for NULLS FIRST/LAST in TopNKeyOperator
> -
>
> Key: HIVE-22464
> URL: https://issues.apache.org/jira/browse/HIVE-22464
> Project: Hive
> Issue Type: Improvement
> Components: Physical Optimizer
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22464.1.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22327) Repl: Ignore read-only transactions in notification log
[ https://issues.apache.org/jira/browse/HIVE-22327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denys Kuzmenko updated HIVE-22327: -- Attachment: HIVE-22327.2.patch > Repl: Ignore read-only transactions in notification log > --- > > Key: HIVE-22327 > URL: https://issues.apache.org/jira/browse/HIVE-22327 > Project: Hive > Issue Type: Improvement > Components: repl >Reporter: Gopal Vijayaraghavan >Assignee: Denys Kuzmenko >Priority: Major > Attachments: HIVE-22327.1.patch, HIVE-22327.2.patch > > > Read txns need not be replicated. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22238) PK/FK selectivity estimation underscales estimations
[ https://issues.apache.org/jira/browse/HIVE-22238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-22238: Attachment: HIVE-22238.09.patch > PK/FK selectivity estimation underscales estimations > > > Key: HIVE-22238 > URL: https://issues.apache.org/jira/browse/HIVE-22238 > Project: Hive > Issue Type: Bug > Components: Statistics >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22238.01.patch, HIVE-22238.02.patch, > HIVE-22238.03.patch, HIVE-22238.04.patch, HIVE-22238.05.patch, > HIVE-22238.05.patch, HIVE-22238.05.patch, HIVE-22238.05.patch, > HIVE-22238.05.patch, HIVE-22238.06.patch, HIVE-22238.06.patch, > HIVE-22238.07.patch, HIVE-22238.09.patch > > Time Spent: 40m > Remaining Estimate: 0h > > at [this > point|https://github.com/apache/hive/blob/5098d155a1e6a164253f5fa98755273bc34085df/ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/StatsRulesProcFactory.java#L2182] > the parent operators rownum is scaled according to pkfkselectivity > however [pkfkselectivity is > computed|https://github.com/apache/hive/blob/5098d155a1e6a164253f5fa98755273bc34085df/ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/StatsRulesProcFactory.java#L2157] > on a whole subtree. > Scaling it by that amount will count in estimation already used when > parentstats was calculated...so depending on the number of upstream joins - > this may lead to severe underestimations > what happened was: > * optimization was able to push the filter to the other side of the join > * as a result the incoming data was already filtered > * scaling down by the PK selectiviy - was actually already there...but a new > "scaling" happened -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Reopened] (HIVE-1450) always catch exception when invoke executeUpdate in jdbc
[
https://issues.apache.org/jira/browse/HIVE-1450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Lavati reopened HIVE-1450:
> always catch exception when invoke executeUpdate in jdbc
>
>
> Key: HIVE-1450
> URL: https://issues.apache.org/jira/browse/HIVE-1450
> Project: Hive
> Issue Type: Bug
> Components: JDBC
>Reporter: Alexey Diomin
>Priority: Major
> Attachments: HIVE-1450.D4293.1.patch
>
>
> Request executed in hive, but always return exception
> error in ./jdbc/src/java/org/apache/hadoop/hive/jdbc/HiveStatement.java
> {code}
> public int executeUpdate(String sql) throws SQLException {
> try {
> client.execute(sql);
> } catch (Exception ex) {
> throw new SQLException(ex.toString());
> }
> throw new SQLException("Method not supported");
> }
> {code}
> executeQuery work correct
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HIVE-1450) always catch exception when invoke executeUpdate in jdbc
[
https://issues.apache.org/jira/browse/HIVE-1450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Lavati resolved HIVE-1450.
Resolution: Won't Do
> always catch exception when invoke executeUpdate in jdbc
>
>
> Key: HIVE-1450
> URL: https://issues.apache.org/jira/browse/HIVE-1450
> Project: Hive
> Issue Type: Bug
> Components: JDBC
>Reporter: Alexey Diomin
>Priority: Major
> Attachments: HIVE-1450.D4293.1.patch
>
>
> Request executed in hive, but always return exception
> error in ./jdbc/src/java/org/apache/hadoop/hive/jdbc/HiveStatement.java
> {code}
> public int executeUpdate(String sql) throws SQLException {
> try {
> client.execute(sql);
> } catch (Exception ex) {
> throw new SQLException(ex.toString());
> }
> throw new SQLException("Method not supported");
> }
> {code}
> executeQuery work correct
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (HIVE-22369) Handle HiveTableFunctionScan at return path
[ https://issues.apache.org/jira/browse/HIVE-22369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16954652#comment-16954652 ] Miklos Gergely edited comment on HIVE-22369 at 11/6/19 12:14 PM: - [~jcamachorodriguez] one way to handle this issue would be this: HiveCalciteUtil.getTopLevelSelect could accept a HiveTableFunctionScan as top level element as well, then HiveOpConverter could have a visit(HiveTableFunctionScan) method, that generates on Operator. Is this the correct way to do it? Please advise. was (Author: mgergely): [~jcamachorodriguez] one way to handle this issue would be this: HiveCalciteUtil.getTopLevelSelect could accept a HiveTableFunctionScan as top level element as well, then HiveConverter could have a visit(HiveTableFunctionScan) method, that generates on Operator. Is this the correct way to do it? Please advise. > Handle HiveTableFunctionScan at return path > --- > > Key: HIVE-22369 > URL: https://issues.apache.org/jira/browse/HIVE-22369 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 4.0.0 > > > The > [optimizedOptiqPlan|https://github.com/apache/hive/blob/5c91d324f22c2ae47e234e76a9bc5ee1a71e6a70/ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java#L1573] > at CalcitePlanner.getOptimizedHiveOPDag is ultimately generated by > CalcitePlanner.internalGenSelectLogicalPlan, which may either provide a > [HiveProject|https://github.com/apache/hive/blob/5c91d324f22c2ae47e234e76a9bc5ee1a71e6a70/ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java#L4831] > or a > [HiveTableFunctionScan|https://github.com/apache/hive/blob/5c91d324f22c2ae47e234e76a9bc5ee1a71e6a70/ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java#L4776]. > When HiveCalciteUtil.getTopLevelSelect is invoked on this it is looking for > a > [HiveProject|https://github.com/apache/hive/blob/5c91d324f22c2ae47e234e76a9bc5ee1a71e6a70/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveCalciteUtil.java#L633] > node in the tree, which if won't find in case of a HiveTableFunctionScan was > returned. This is why TestNewGetSplitsFormat is failing with return path. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22464) Implement support for NULLS FIRST/LAST in TopNKeyOperator
[
https://issues.apache.org/jira/browse/HIVE-22464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968315#comment-16968315
]
Hive QA commented on HIVE-22464:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
53s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
42s{color} | {color:blue} serde in master has 199 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
9s{color} | {color:blue} ql in master has 1550 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
18s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch 4 line(s) with tabs. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
25s{color} | {color:red} ql generated 1 new + 1547 unchanged - 3 fixed = 1548
total (was 1550) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 30m 24s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | org.apache.hadoop.hive.ql.exec.TopNKeyOperator$KeyWrapperComparator
implements Comparator but not Serializable At
TopNKeyOperator.java:Serializable At TopNKeyOperator.java:[lines 65-82] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19312/dev-support/hive-personality.sh
|
| git revision | master / d0bd071 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19312/yetus/whitespace-tabs.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19312/yetus/new-findbugs-ql.html
|
| modules | C: serde ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19312/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Implement support for NULLS FIRST/LAST in TopNKeyOperator
> -
>
> Key: HIVE-22464
> URL: https://issues.apache.org/jira/browse/HIVE-22464
> Project: Hive
> Issue Type: Improvement
> Components: Physical Optimizer
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22464.1.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22465) Add ssl conf in TezConfigurationFactory
[ https://issues.apache.org/jira/browse/HIVE-22465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HIVE-22465: Summary: Add ssl conf in TezConfigurationFactory (was: Add ssl-client conf in TezConfigurationFactory) > Add ssl conf in TezConfigurationFactory > --- > > Key: HIVE-22465 > URL: https://issues.apache.org/jira/browse/HIVE-22465 > Project: Hive > Issue Type: Improvement >Reporter: Rajesh Balamohan >Assignee: Rajesh Balamohan >Priority: Minor > Attachments: HIVE-22465.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22465) Add ssl conf in TezConfigurationFactory
[ https://issues.apache.org/jira/browse/HIVE-22465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HIVE-22465: Affects Version/s: 4.0.0 > Add ssl conf in TezConfigurationFactory > --- > > Key: HIVE-22465 > URL: https://issues.apache.org/jira/browse/HIVE-22465 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0 >Reporter: Rajesh Balamohan >Assignee: Rajesh Balamohan >Priority: Minor > Attachments: HIVE-22465.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22465) Add ssl-client conf in TezConfigurationFactory
[ https://issues.apache.org/jira/browse/HIVE-22465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HIVE-22465: Status: Patch Available (was: Open) > Add ssl-client conf in TezConfigurationFactory > -- > > Key: HIVE-22465 > URL: https://issues.apache.org/jira/browse/HIVE-22465 > Project: Hive > Issue Type: Improvement >Reporter: Rajesh Balamohan >Priority: Minor > Attachments: HIVE-22465.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-22465) Add ssl-client conf in TezConfigurationFactory
[ https://issues.apache.org/jira/browse/HIVE-22465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan reassigned HIVE-22465: --- Assignee: Rajesh Balamohan > Add ssl-client conf in TezConfigurationFactory > -- > > Key: HIVE-22465 > URL: https://issues.apache.org/jira/browse/HIVE-22465 > Project: Hive > Issue Type: Improvement >Reporter: Rajesh Balamohan >Assignee: Rajesh Balamohan >Priority: Minor > Attachments: HIVE-22465.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22465) Add ssl-client conf in TezConfigurationFactory
[ https://issues.apache.org/jira/browse/HIVE-22465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HIVE-22465: Attachment: HIVE-22465.1.patch > Add ssl-client conf in TezConfigurationFactory > -- > > Key: HIVE-22465 > URL: https://issues.apache.org/jira/browse/HIVE-22465 > Project: Hive > Issue Type: Improvement >Reporter: Rajesh Balamohan >Priority: Minor > Attachments: HIVE-22465.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22238) PK/FK selectivity estimation underscales estimations
[
https://issues.apache.org/jira/browse/HIVE-22238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968280#comment-16968280
]
Hive QA commented on HIVE-22238:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12985034/HIVE-22238.07.patch
{color:green}SUCCESS:{color} +1 due to 4 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 189 failed/errored test(s), 17572 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_join_pkfk]
(batchId=16)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_outer_join6]
(batchId=46)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[estimate_pkfk_push]
(batchId=174)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_notin]
(batchId=182)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_scalar]
(batchId=173)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_inner_join]
(batchId=182)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[cbo_query23]
(batchId=300)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[cbo_query6]
(batchId=300)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query10]
(batchId=300)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query11]
(batchId=300)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query12]
(batchId=300)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query13]
(batchId=300)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query14]
(batchId=300)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query15]
(batchId=300)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query16]
(batchId=300)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query17]
(batchId=300)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query18]
(batchId=300)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query19]
(batchId=300)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query1]
(batchId=300)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query1b]
(batchId=300)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query20]
(batchId=300)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query23]
(batchId=300)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query25]
(batchId=300)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query26]
(batchId=300)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query27]
(batchId=300)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query29]
(batchId=300)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query2]
(batchId=300)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query30]
(batchId=300)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query31]
(batchId=300)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query32]
(batchId=300)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query33]
(batchId=300)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query34]
(batchId=300)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query35]
(batchId=300)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query36]
(batchId=300)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query38]
(batchId=300)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query3]
(batchId=300)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query40]
(batchId=300)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query42]
(batchId=300)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query43]
(batchId=300)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query45]
(batchId=300)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query46]
(batchId=300)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query47]
(batchId=300)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query48]
(batchId=300)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query49]
(batchId=300)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query4]
(batchId=300)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query50]
(batchId=300)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query51]
(batchId=300)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query52]
(batchId=300)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query53]
(batchId=300)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query54]
(batchId=300)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query55]
[jira] [Commented] (HIVE-22238) PK/FK selectivity estimation underscales estimations
[
https://issues.apache.org/jira/browse/HIVE-22238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968272#comment-16968272
]
Hive QA commented on HIVE-22238:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
49s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
49s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
42s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
11s{color} | {color:blue} ql in master has 1550 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 8m
0s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
25s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
32s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
42s{color} | {color:red} ql: The patch generated 3 new + 102 unchanged - 0
fixed = 105 total (was 102) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m
0s{color} | {color:red} root: The patch generated 3 new + 102 unchanged - 0
fixed = 105 total (was 102) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 6 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
21s{color} | {color:red} ql generated 1 new + 1550 unchanged - 0 fixed = 1551
total (was 1550) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 7m
54s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 66m 16s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Unread field:AccurateEstimatesCheckerHook.java:[line 61] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19311/dev-support/hive-personality.sh
|
| git revision | master / d0bd071 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19311/yetus/diff-checkstyle-ql.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19311/yetus/diff-checkstyle-root.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19311/yetus/whitespace-eol.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19311/yetus/new-findbugs-ql.html
|
| modules | C: ql . itests U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19311/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> PK/FK selectivity estimation underscales estimations
>
>
> Key: HIVE-22238
> URL: https://issues.apache.org/jira/browse/HIVE-22238
> Project: Hive
> Issue Type: Bug
> Components: Statistics
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22238.01.patch, HIVE-22238.02.patch,
[jira] [Updated] (HIVE-22405) Add ColumnVector support for ProlepticCalendar
[
https://issues.apache.org/jira/browse/HIVE-22405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
László Bodor updated HIVE-22405:
Status: Patch Available (was: Open)
> Add ColumnVector support for ProlepticCalendar
> --
>
> Key: HIVE-22405
> URL: https://issues.apache.org/jira/browse/HIVE-22405
> Project: Hive
> Issue Type: Bug
>Reporter: Owen O'Malley
>Assignee: László Bodor
>Priority: Major
> Attachments: HIVE-22405.01.patch
>
>
> Hive recently moved its processing to the proleptic calendar, which has
> created some issues for users who have dates before 1580 AD.
> I'd propose extending the column vectors for times & dates to encode which
> calendar they are using.
> * create DateColumnVector that extends LongColumnVector
> * add a method to change calendars to both DateColumnVector and
> TimestampColumnVector.
> {code}
> /**
>* Change the calendar to or from proleptic. If the new and old values of
> the flag are the
>* same, nothing is done.
>* useProleptic - set the flag for the proleptic calendar
>* updateData - change the data to match the new value of the flag.
>*/
> void changeCalendar(useProleptic: boolean, updateData: boolean);
> /**
>* Detect whether this data is using the proleptic calendar.
>*/
> boolean usingProlepticCalendar();
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22405) Add ColumnVector support for ProlepticCalendar
[
https://issues.apache.org/jira/browse/HIVE-22405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
László Bodor updated HIVE-22405:
Attachment: HIVE-22405.01.patch
> Add ColumnVector support for ProlepticCalendar
> --
>
> Key: HIVE-22405
> URL: https://issues.apache.org/jira/browse/HIVE-22405
> Project: Hive
> Issue Type: Bug
>Reporter: Owen O'Malley
>Assignee: László Bodor
>Priority: Major
> Attachments: HIVE-22405.01.patch
>
>
> Hive recently moved its processing to the proleptic calendar, which has
> created some issues for users who have dates before 1580 AD.
> I'd propose extending the column vectors for times & dates to encode which
> calendar they are using.
> * create DateColumnVector that extends LongColumnVector
> * add a method to change calendars to both DateColumnVector and
> TimestampColumnVector.
> {code}
> /**
>* Change the calendar to or from proleptic. If the new and old values of
> the flag are the
>* same, nothing is done.
>* useProleptic - set the flag for the proleptic calendar
>* updateData - change the data to match the new value of the flag.
>*/
> void changeCalendar(useProleptic: boolean, updateData: boolean);
> /**
>* Detect whether this data is using the proleptic calendar.
>*/
> boolean usingProlepticCalendar();
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22464) Implement support for NULLS FIRST/LAST in TopNKeyOperator
[ https://issues.apache.org/jira/browse/HIVE-22464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-22464: -- Status: Patch Available (was: Open) > Implement support for NULLS FIRST/LAST in TopNKeyOperator > - > > Key: HIVE-22464 > URL: https://issues.apache.org/jira/browse/HIVE-22464 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Reporter: Krisztian Kasa >Assignee: Krisztian Kasa >Priority: Major > Attachments: HIVE-22464.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22464) Implement support for NULLS FIRST/LAST in TopNKeyOperator
[ https://issues.apache.org/jira/browse/HIVE-22464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-22464: -- Attachment: HIVE-22464.1.patch > Implement support for NULLS FIRST/LAST in TopNKeyOperator > - > > Key: HIVE-22464 > URL: https://issues.apache.org/jira/browse/HIVE-22464 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Reporter: Krisztian Kasa >Assignee: Krisztian Kasa >Priority: Major > Attachments: HIVE-22464.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-22347) Break up DDLSemanticAnalyzer - extract Other analyzers
[ https://issues.apache.org/jira/browse/HIVE-22347?focusedWorklogId=339274=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-339274 ] ASF GitHub Bot logged work on HIVE-22347: - Author: ASF GitHub Bot Created on: 06/Nov/19 10:09 Start Date: 06/Nov/19 10:09 Worklog Time Spent: 10m Work Description: miklosgergely commented on pull request #817: HIVE-22347 Break up DDLSemanticAnalyzer - extract Other analyzers URL: https://github.com/apache/hive/pull/817#discussion_r343011859 ## File path: ql/src/java/org/apache/hadoop/hive/ql/ddl/misc/insert/commit/hook/InsertCommitHookOperation.java ## @@ -16,7 +16,7 @@ * limitations under the License. */ -package org.apache.hadoop.hive.ql.ddl.misc; +package org.apache.hadoop.hive.ql.ddl.misc.insert.commit.hook; Review comment: I think that it is a good structure that each operation have their own package, it makes things cleaner. Putting everything under misc would break this structure, and also has a potential problem if in the future there would be more operations under misc, it may be a huge package with an endless list of classes. I'm ok with renaming the sub packages: cache.metadata -> metadata insert.commit.hook -> hooks msck -> msck remove.flag -> flags (?) show.conf -> conf Would that be ok? I don't see which of these operations are related enough to be put under the same category, so we'll still have 5 packages, but if in the future we'll have some new configuration, or metadata, or hook related stuff, they can go to the same package, or under some sub packages. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 339274) Time Spent: 0.5h (was: 20m) > Break up DDLSemanticAnalyzer - extract Other analyzers > -- > > Key: HIVE-22347 > URL: https://issues.apache.org/jira/browse/HIVE-22347 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > Attachments: HIVE-22347.01.patch, HIVE-22347.02.patch > > Time Spent: 0.5h > Remaining Estimate: 0h > > DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is > to refactor it in order to have everything cut into more handleable classes > under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each analyzers > * have a package for each operation, containing an analyzer, a description, > and an operation, so the amount of classes under a package is more manageable > Step #8: extract all the rest of the analyzers from DDLSemanticAnalyzer, > which can not be classified otherwise, and move them under the new package. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22311) Propagate min/max column values from statistics to the optimizer for timestamp type
[ https://issues.apache.org/jira/browse/HIVE-22311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968228#comment-16968228 ] Miklos Gergely commented on HIVE-22311: --- +1 > Propagate min/max column values from statistics to the optimizer for > timestamp type > --- > > Key: HIVE-22311 > URL: https://issues.apache.org/jira/browse/HIVE-22311 > Project: Hive > Issue Type: Bug > Components: Statistics >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22311.01.patch, HIVE-22311.01.patch, > HIVE-22311.02.patch, HIVE-22311.03.patch, HIVE-22311.04.patch, > HIVE-22311.patch > > Time Spent: 0.5h > Remaining Estimate: 0h > > Currently stats annotation does not consider timestamp type e.g. for > estimates with range predicates. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22238) PK/FK selectivity estimation underscales estimations
[ https://issues.apache.org/jira/browse/HIVE-22238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-22238: Attachment: HIVE-22238.07.patch > PK/FK selectivity estimation underscales estimations > > > Key: HIVE-22238 > URL: https://issues.apache.org/jira/browse/HIVE-22238 > Project: Hive > Issue Type: Bug > Components: Statistics >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22238.01.patch, HIVE-22238.02.patch, > HIVE-22238.03.patch, HIVE-22238.04.patch, HIVE-22238.05.patch, > HIVE-22238.05.patch, HIVE-22238.05.patch, HIVE-22238.05.patch, > HIVE-22238.05.patch, HIVE-22238.06.patch, HIVE-22238.06.patch, > HIVE-22238.07.patch > > Time Spent: 40m > Remaining Estimate: 0h > > at [this > point|https://github.com/apache/hive/blob/5098d155a1e6a164253f5fa98755273bc34085df/ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/StatsRulesProcFactory.java#L2182] > the parent operators rownum is scaled according to pkfkselectivity > however [pkfkselectivity is > computed|https://github.com/apache/hive/blob/5098d155a1e6a164253f5fa98755273bc34085df/ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/StatsRulesProcFactory.java#L2157] > on a whole subtree. > Scaling it by that amount will count in estimation already used when > parentstats was calculated...so depending on the number of upstream joins - > this may lead to severe underestimations > what happened was: > * optimization was able to push the filter to the other side of the join > * as a result the incoming data was already filtered > * scaling down by the PK selectiviy - was actually already there...but a new > "scaling" happened -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22463) Support Decimal64 column multiplication with decimal64 Column/Scalar
[
https://issues.apache.org/jira/browse/HIVE-22463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968185#comment-16968185
]
Hive QA commented on HIVE-22463:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12985011/HIVE-22463.1.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19310/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19310/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19310/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2019-11-06 08:51:23.181
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-19310/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2019-11-06 08:51:23.184
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at d0bd071 HIVE-22292: Implement Hypothetical-Set Aggregate
Functions (Krisztian Kasa, reviewed by Jesus Camacho Rodriguez)
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at d0bd071 HIVE-22292: Implement Hypothetical-Set Aggregate
Functions (Krisztian Kasa, reviewed by Jesus Camacho Rodriguez)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2019-11-06 08:51:24.336
+ rm -rf ../yetus_PreCommit-HIVE-Build-19310
+ mkdir ../yetus_PreCommit-HIVE-Build-19310
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-19310
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-19310/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error:
a/ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizationContext.java:
does not exist in index
error:
a/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPDivide.java:
does not exist in index
error:
a/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPMultiply.java:
does not exist in index
error:
a/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToDecimal.java:
does not exist in index
error: a/vector-code-gen/src/org/apache/hadoop/hive/tools/GenVectorCode.java:
does not exist in index
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizationContext.java:1778
Falling back to three-way merge...
Applied patch to
'ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizationContext.java'
with conflicts.
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPDivide.java:49
Falling back to three-way merge...
Applied patch to
'ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPDivide.java'
with conflicts.
error: patch failed:
vector-code-gen/src/org/apache/hadoop/hive/tools/GenVectorCode.java:309
Falling back to three-way merge...
Applied patch to
'vector-code-gen/src/org/apache/hadoop/hive/tools/GenVectorCode.java' cleanly.
Going to apply patch with: git apply -p1
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizationContext.java:1778
Falling back to three-way merge...
Applied patch to
'ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizationContext.java'
with conflicts.
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPDivide.java:49
Falling back to three-way merge...
Applied patch to
'ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPDivide.java'
with conflicts.
error: patch failed:
vector-code-gen/src/org/apache/hadoop/hive/tools/GenVectorCode.java:309
Falling back to
[jira] [Commented] (HIVE-15157) Partition Table With timestamp type on S3 storage --> Error in getting fields from serde.Invalid Field null
[
https://issues.apache.org/jira/browse/HIVE-15157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968183#comment-16968183
]
Hive QA commented on HIVE-15157:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12985002/HIVE-15157.03.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17570 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19309/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19309/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19309/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12985002 - PreCommit-HIVE-Build
> Partition Table With timestamp type on S3 storage --> Error in getting fields
> from serde.Invalid Field null
> ---
>
> Key: HIVE-15157
> URL: https://issues.apache.org/jira/browse/HIVE-15157
> Project: Hive
> Issue Type: Bug
> Components: Clients
>Affects Versions: 2.1.0
> Environment: JDK 1.8 101
>Reporter: thauvin damien
>Assignee: Jesus Camacho Rodriguez
>Priority: Critical
> Labels: pull-request-available, timestamp
> Attachments: HIVE-15157.01.patch, HIVE-15157.02.patch,
> HIVE-15157.03.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Hello
> I get the error above when i try to perform :
> hive> DESCRIBE formatted table partition (tsbucket='2016-10-28 16%3A00%3A00');
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask. Error in getting fields from
> serde.Invalid Field null
> Here is the description of the issue.
> --External table Hive with dynamic partition enable on Aws S3 storage.
> --Partition Table with timestamp type .
> When i perform "show partition table;" everything is fine :
> hive> show partitions table;
> OK
> tsbucket=2016-10-01 11%3A00%3A00
> tsbucket=2016-10-28 16%3A00%3A00
> And when i perform "describe FORMATTED table;" everything is fine
> Is this a bug ?
> The stacktrace of hive.log :
> 2016-11-08T10:30:20,868 ERROR [ac3e0d48-22c5-4d04-a788-aeb004ea94f3
> main([])]: exec.DDLTask (DDLTask.java:failed(574)) -
> org.apache.hadoop.hive.ql.metadata.HiveException: Error in getting fields
> from serde.Invalid Field null
> at
> org.apache.hadoop.hive.ql.metadata.Hive.getFieldsFromDeserializer(Hive.java:3414)
> at
> org.apache.hadoop.hive.ql.exec.DDLTask.describeTable(DDLTask.java:3109)
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:408)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1858)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1562)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1313)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1084)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1072)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:232)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:183)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:399)
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:776)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:714)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:641)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> Caused by: MetaException(message:Invalid Field null)
> at
> org.apache.hadoop.hive.metastore.MetaStoreUtils.getFieldsFromDeserializer(MetaStoreUtils.java:1336)
> at
> org.apache.hadoop.hive.ql.metadata.Hive.getFieldsFromDeserializer(Hive.java:3409)
> ... 21 more
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-22464) Implement support for NULLS FIRST/LAST in TopNKeyOperator
[ https://issues.apache.org/jira/browse/HIVE-22464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa reassigned HIVE-22464: - > Implement support for NULLS FIRST/LAST in TopNKeyOperator > - > > Key: HIVE-22464 > URL: https://issues.apache.org/jira/browse/HIVE-22464 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Reporter: Krisztian Kasa >Assignee: Krisztian Kasa >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-15157) Partition Table With timestamp type on S3 storage --> Error in getting fields from serde.Invalid Field null
[
https://issues.apache.org/jira/browse/HIVE-15157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968146#comment-16968146
]
Hive QA commented on HIVE-15157:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
56s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
39s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
12s{color} | {color:blue} ql in master has 1550 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
21s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
25s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 29m 5s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19309/dev-support/hive-personality.sh
|
| git revision | master / d0bd071 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql itests/hive-blobstore U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19309/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Partition Table With timestamp type on S3 storage --> Error in getting fields
> from serde.Invalid Field null
> ---
>
> Key: HIVE-15157
> URL: https://issues.apache.org/jira/browse/HIVE-15157
> Project: Hive
> Issue Type: Bug
> Components: Clients
>Affects Versions: 2.1.0
> Environment: JDK 1.8 101
>Reporter: thauvin damien
>Assignee: Jesus Camacho Rodriguez
>Priority: Critical
> Labels: pull-request-available, timestamp
> Attachments: HIVE-15157.01.patch, HIVE-15157.02.patch,
> HIVE-15157.03.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Hello
> I get the error above when i try to perform :
> hive> DESCRIBE formatted table partition (tsbucket='2016-10-28 16%3A00%3A00');
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask. Error in getting fields from
> serde.Invalid Field null
> Here is the description of the issue.
> --External table Hive with dynamic partition enable on Aws S3 storage.
> --Partition Table with timestamp type .
> When i perform "show partition table;" everything is fine :
> hive> show partitions table;
> OK
> tsbucket=2016-10-01 11%3A00%3A00
> tsbucket=2016-10-28 16%3A00%3A00
>
