[jira] [Commented] (HIVE-22720) Optimise AuthenticationProviderFactory::getAuthenticationProvider
[
https://issues.apache.org/jira/browse/HIVE-22720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015699#comment-17015699
]
Hive QA commented on HIVE-22720:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12990779/HIVE-22720.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17876 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20179/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20179/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20179/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12990779 - PreCommit-HIVE-Build
> Optimise AuthenticationProviderFactory::getAuthenticationProvider
> -
>
> Key: HIVE-22720
> URL: https://issues.apache.org/jira/browse/HIVE-22720
> Project: Hive
> Issue Type: Improvement
>Reporter: Rajesh Balamohan
>Priority: Major
> Attachments: HIVE-22720.1.patch, HIVE-22720.2.patch, Screenshot
> 2020-01-13 at 10.07.34 AM.jpg
>
>
> !Screenshot 2020-01-13 at 10.07.34 AM.jpg|width=439,height=269!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22663) Quote all table and column names or do not quote any
[
https://issues.apache.org/jira/browse/HIVE-22663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Chovan updated HIVE-22663:
-
Attachment: HIVE-22663.4.patch
> Quote all table and column names or do not quote any
>
>
> Key: HIVE-22663
> URL: https://issues.apache.org/jira/browse/HIVE-22663
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2, Standalone Metastore
>Affects Versions: 4.0.0
>Reporter: Ashutosh Bapat
>Assignee: Zoltan Chovan
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22663.2.patch, HIVE-22663.3.patch,
> HIVE-22663.4.patch, HIVE-22663.patch
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> The change in HIVE-22546 is causing following stack trace when I run Hive
> with PostgreSQL as backend db for the metastore.
> 0: jdbc:hive2://localhost:1> create database dumpdb with
> ('repl.source.for'='1,2,3');0: jdbc:hive2://localhost:1> create database
> dumpdb with ('repl.source.for'='1,2,3');Error: Error while compiling
> statement: FAILED: ParseException line 1:28 missing KW_DBPROPERTIES at '('
> near '' (state=42000,code=4)0: jdbc:hive2://localhost:1> create
> database dumpdb with dbproperties ('repl.source.for'='1,2,3');ERROR : FAILED:
> Hive Internal Error: org.apache.hadoop.hive.ql.lockmgr.LockException(Error
> communicating with the
> metastore)org.apache.hadoop.hive.ql.lockmgr.LockException: Error
> communicating with the metastore at
> org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.commitTxn(DbTxnManager.java:541)
> at
> org.apache.hadoop.hive.ql.Driver.releaseLocksAndCommitOrRollback(Driver.java:687)
> at
> org.apache.hadoop.hive.ql.Driver.releaseLocksAndCommitOrRollback(Driver.java:653)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:969)
> ... stack trace clipped
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)Caused by:
> MetaException(message:Unable to update transaction database
> org.postgresql.util.PSQLException: ERROR: relation
> "materialization_rebuild_locks" does not exist Position: 13 at
> org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2440)
> at
> org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2183)
> at
> org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:308)
> at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:441) at
> org.postgresql.jdbc.PgStatement.execute(PgStatement.java:365) at
> This happens because the table names in all the queries in TxnHandler.java
> (including the one at 1312, which causes this stack trace) are not quoting
> the table names. All the tablenames and column names should be quoted there.
> Just the change in HIVE-22546 won't suffice.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22727) Add hive db schema changes introduced in HIVE-21884 to the schema upgrade scripts
[ https://issues.apache.org/jira/browse/HIVE-22727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015688#comment-17015688 ] Zoltan Chovan commented on HIVE-22727: -- No problem, thanks for the review! > Add hive db schema changes introduced in HIVE-21884 to the schema upgrade > scripts > - > > Key: HIVE-22727 > URL: https://issues.apache.org/jira/browse/HIVE-22727 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Chovan >Assignee: Zoltan Chovan >Priority: Minor > Labels: pull-request-available > Attachments: HIVE-22727.patch > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22720) Optimise AuthenticationProviderFactory::getAuthenticationProvider
[
https://issues.apache.org/jira/browse/HIVE-22720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015670#comment-17015670
]
Hive QA commented on HIVE-22720:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
39s{color} | {color:blue} service in master has 50 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 13m 18s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20179/dev-support/hive-personality.sh
|
| git revision | master / 72d56f4 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: service U: service |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20179/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Optimise AuthenticationProviderFactory::getAuthenticationProvider
> -
>
> Key: HIVE-22720
> URL: https://issues.apache.org/jira/browse/HIVE-22720
> Project: Hive
> Issue Type: Improvement
>Reporter: Rajesh Balamohan
>Priority: Major
> Attachments: HIVE-22720.1.patch, HIVE-22720.2.patch, Screenshot
> 2020-01-13 at 10.07.34 AM.jpg
>
>
> !Screenshot 2020-01-13 at 10.07.34 AM.jpg|width=439,height=269!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22724) Reduce number of DB calls in ObjectStore, TxnHandler
[ https://issues.apache.org/jira/browse/HIVE-22724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015667#comment-17015667 ] Rajesh Balamohan commented on HIVE-22724: - Test failure is unrelated to this patch and works locally. Resubmitting same patch for one more run. > Reduce number of DB calls in ObjectStore, TxnHandler > > > Key: HIVE-22724 > URL: https://issues.apache.org/jira/browse/HIVE-22724 > Project: Hive > Issue Type: Improvement >Reporter: Rajesh Balamohan >Assignee: Rajesh Balamohan >Priority: Minor > Attachments: HIVE-22724.1.patch, HIVE-22724.2.patch, > HIVE-22724.3.patch, HIVE-22724.4.patch, Screenshot 2020-01-14 at 4.55.12 > AM.png > > > !Screenshot 2020-01-14 at 4.55.12 AM.png|width=668,height=310! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22724) Reduce number of DB calls in ObjectStore, TxnHandler
[ https://issues.apache.org/jira/browse/HIVE-22724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HIVE-22724: Attachment: HIVE-22724.4.patch > Reduce number of DB calls in ObjectStore, TxnHandler > > > Key: HIVE-22724 > URL: https://issues.apache.org/jira/browse/HIVE-22724 > Project: Hive > Issue Type: Improvement >Reporter: Rajesh Balamohan >Assignee: Rajesh Balamohan >Priority: Minor > Attachments: HIVE-22724.1.patch, HIVE-22724.2.patch, > HIVE-22724.3.patch, HIVE-22724.4.patch, Screenshot 2020-01-14 at 4.55.12 > AM.png > > > !Screenshot 2020-01-14 at 4.55.12 AM.png|width=668,height=310! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22366) Multiple metastore calls for same table and constraints during planning
[
https://issues.apache.org/jira/browse/HIVE-22366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015653#comment-17015653
]
Hive QA commented on HIVE-22366:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12990851/HIVE-22366.4.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 17876 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[hybridgrace_hashjoin_2]
(batchId=114)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[cbo_query10]
(batchId=303)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[cbo_query18]
(batchId=303)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[cbo_query23]
(batchId=303)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[cbo_query35]
(batchId=303)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[cbo_query69]
(batchId=303)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[query10]
(batchId=303)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[query23]
(batchId=303)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[query35]
(batchId=303)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[query69]
(batchId=303)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20178/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20178/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20178/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 10 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12990851 - PreCommit-HIVE-Build
> Multiple metastore calls for same table and constraints during planning
> ---
>
> Key: HIVE-22366
> URL: https://issues.apache.org/jira/browse/HIVE-22366
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-22366.1.patch, HIVE-22366.2.patch,
> HIVE-22366.3.patch, HIVE-22366.4.patch
>
>
> e.g. for a simple query like
> {code:sql}
> explain select count(i) from test2 group by j limit 3;
> {code}
> planner makes the following calls:
> {code}
> 2019-10-17T22:44:49,892 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr cmd=get_table :
> tbl=hive.default.test2
> 2019-10-17T22:44:49,908 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_primary_keys : tbl=hive.default.test2
> 2019-10-17T22:44:49,910 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_foreign_keys : parentdb=null parenttbl=null foreigndb=default
> foreigntbl=test2
> 2019-10-17T22:44:49,911 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_unique_constraints : tbl=hive.default.test2
> 2019-10-17T22:44:49,913 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_not_null_constraints : tbl=hive.default.test2
> 2019-10-17T22:44:49,979 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_partitions : tbl=hive.default.test2
> 2019-10-17T22:44:49,997 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_aggr_stats_for: table=hive.default.test2
> 2019-10-17T22:44:50,014 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr cmd=get_table :
> tbl=hive.default.test2
> 2019-10-17T22:44:50,023 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_primary_keys : tbl=hive.default.test2
> 2019-10-17T22:44:50,024 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_foreign_keys : parentdb=null parenttbl=null foreigndb=default
> foreigntbl=test2
> 2019-10-17T22:44:50,025 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab
[jira] [Commented] (HIVE-22366) Multiple metastore calls for same table and constraints during planning
[
https://issues.apache.org/jira/browse/HIVE-22366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015636#comment-17015636
]
Hive QA commented on HIVE-22366:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
41s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
33s{color} | {color:blue} standalone-metastore/metastore-common in master has
37 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
1s{color} | {color:blue} ql in master has 1531 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
4s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
55s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 34m 59s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20178/dev-support/hive-personality.sh
|
| git revision | master / 72d56f4 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: standalone-metastore/metastore-common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20178/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Multiple metastore calls for same table and constraints during planning
> ---
>
> Key: HIVE-22366
> URL: https://issues.apache.org/jira/browse/HIVE-22366
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-22366.1.patch, HIVE-22366.2.patch,
> HIVE-22366.3.patch, HIVE-22366.4.patch
>
>
> e.g. for a simple query like
> {code:sql}
> explain select count(i) from test2 group by j limit 3;
> {code}
> planner makes the following calls:
> {code}
> 2019-10-17T22:44:49,892 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr cmd=get_table :
> tbl=hive.default.test2
> 2019-10-17T22:44:49,908 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_primary_keys : tbl=hive.default.test2
> 2019-10-17T22:44:49,910 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg
[jira] [Commented] (HIVE-20974) TezTask should set task exception on failures
[ https://issues.apache.org/jira/browse/HIVE-20974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015628#comment-17015628 ] Ashutosh Chauhan commented on HIVE-20974: - [~rajesh.balamohan] looks like test cases needs to be updated. > TezTask should set task exception on failures > - > > Key: HIVE-20974 > URL: https://issues.apache.org/jira/browse/HIVE-20974 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Rajesh Balamohan >Assignee: Rajesh Balamohan >Priority: Trivial > Attachments: HIVE-20974.1.patch > > > TezTask logs the error as "Failed to execute tez graph" and proceeds further. > "TaskRunner.runSequentail()" code would not be able to get these exceptions > for TezTask. If there are any failure hooks configured, these exceptions > wouldn't show up. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22725) Lazy evaluate HiveMetastore::fireReadTablePreEvent table computation
[ https://issues.apache.org/jira/browse/HIVE-22725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015625#comment-17015625 ] Rajesh Balamohan commented on HIVE-22725: - Fixing test cases. > Lazy evaluate HiveMetastore::fireReadTablePreEvent table computation > > > Key: HIVE-22725 > URL: https://issues.apache.org/jira/browse/HIVE-22725 > Project: Hive > Issue Type: Improvement >Reporter: Rajesh Balamohan >Assignee: Rajesh Balamohan >Priority: Minor > Attachments: HIVE-22725.1.patch, HIVE-22725.2.patch, > HIVE-22725.3.patch, image-2020-01-14-13-22-54-483.png > > > "TransactionalValidationListener" gets added in the pre-event listeners of > HMS by default. > [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java#L559] > This causes issue in short select queries, as table details are computed for > any partition lookups. > [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java#L4984] > > !image-2020-01-14-13-22-54-483.png|width=579,height=202! > > It would be good to lazy evaluate table lookup in this codepath. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22725) Lazy evaluate HiveMetastore::fireReadTablePreEvent table computation
[ https://issues.apache.org/jira/browse/HIVE-22725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HIVE-22725: Attachment: HIVE-22725.3.patch > Lazy evaluate HiveMetastore::fireReadTablePreEvent table computation > > > Key: HIVE-22725 > URL: https://issues.apache.org/jira/browse/HIVE-22725 > Project: Hive > Issue Type: Improvement >Reporter: Rajesh Balamohan >Assignee: Rajesh Balamohan >Priority: Minor > Attachments: HIVE-22725.1.patch, HIVE-22725.2.patch, > HIVE-22725.3.patch, image-2020-01-14-13-22-54-483.png > > > "TransactionalValidationListener" gets added in the pre-event listeners of > HMS by default. > [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java#L559] > This causes issue in short select queries, as table details are computed for > any partition lookups. > [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java#L4984] > > !image-2020-01-14-13-22-54-483.png|width=579,height=202! > > It would be good to lazy evaluate table lookup in this codepath. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22648) Upgrade Parquet to 1.11.0
[
https://issues.apache.org/jira/browse/HIVE-22648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015622#comment-17015622
]
Hive QA commented on HIVE-22648:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12990848/HIVE-22648.3.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17876 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20177/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20177/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20177/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12990848 - PreCommit-HIVE-Build
> Upgrade Parquet to 1.11.0
> -
>
> Key: HIVE-22648
> URL: https://issues.apache.org/jira/browse/HIVE-22648
> Project: Hive
> Issue Type: Improvement
> Components: File Formats
>Reporter: Marta Kuczora
>Assignee: Marta Kuczora
>Priority: Major
> Labels: Parquet, parquet
> Attachments: HIVE-22648.1.patch, HIVE-22648.2.patch,
> HIVE-22648.3.patch
>
>
> Upgrade the Parquet version to 1.11.0.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22648) Upgrade Parquet to 1.11.0
[
https://issues.apache.org/jira/browse/HIVE-22648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015614#comment-17015614
]
Hive QA commented on HIVE-22648:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
42s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 7m
32s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 7m
45s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 49m 33s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc xml compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20177/dev-support/hive-personality.sh
|
| git revision | master / 72d56f4 |
| Default Java | 1.8.0_111 |
| modules | C: ql . U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20177/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Upgrade Parquet to 1.11.0
> -
>
> Key: HIVE-22648
> URL: https://issues.apache.org/jira/browse/HIVE-22648
> Project: Hive
> Issue Type: Improvement
> Components: File Formats
>Reporter: Marta Kuczora
>Assignee: Marta Kuczora
>Priority: Major
> Labels: Parquet, parquet
> Attachments: HIVE-22648.1.patch, HIVE-22648.2.patch,
> HIVE-22648.3.patch
>
>
> Upgrade the Parquet version to 1.11.0.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22515) Support cast to decimal64 in Vectorization
[ https://issues.apache.org/jira/browse/HIVE-22515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015601#comment-17015601 ] Ramesh Kumar Thangarajan commented on HIVE-22515: - [~hashutosh] [~gopalv], Can you please review [https://github.com/apache/hive/pull/880] and provide any comments? > Support cast to decimal64 in Vectorization > -- > > Key: HIVE-22515 > URL: https://issues.apache.org/jira/browse/HIVE-22515 > Project: Hive > Issue Type: Bug >Reporter: Ramesh Kumar Thangarajan >Assignee: Ramesh Kumar Thangarajan >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22515.5.patch, HIVE-22515.8.patch, > HIVE-22515.9.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Support cast to decimal64 in Vectorization -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22381) JMH tests for Decimal64 and Decimal division on a column over scalar
[ https://issues.apache.org/jira/browse/HIVE-22381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015602#comment-17015602 ] Ramesh Kumar Thangarajan commented on HIVE-22381: - Yes Ashutosh, I will retest this patch and make any changes needed before we can push this. > JMH tests for Decimal64 and Decimal division on a column over scalar > > > Key: HIVE-22381 > URL: https://issues.apache.org/jira/browse/HIVE-22381 > Project: Hive > Issue Type: Bug >Reporter: Ramesh Kumar Thangarajan >Assignee: Ramesh Kumar Thangarajan >Priority: Major > Attachments: HIVE-22381.1.patch, errors.xml > > > JMH tests for Decimal64 and Decimal division on a column over scalar -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22515) Support cast to decimal64 in Vectorization
[ https://issues.apache.org/jira/browse/HIVE-22515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-22515: -- Labels: pull-request-available (was: ) > Support cast to decimal64 in Vectorization > -- > > Key: HIVE-22515 > URL: https://issues.apache.org/jira/browse/HIVE-22515 > Project: Hive > Issue Type: Bug >Reporter: Ramesh Kumar Thangarajan >Assignee: Ramesh Kumar Thangarajan >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22515.5.patch, HIVE-22515.8.patch, > HIVE-22515.9.patch > > > Support cast to decimal64 in Vectorization -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-22515) Support cast to decimal64 in Vectorization
[ https://issues.apache.org/jira/browse/HIVE-22515?focusedWorklogId=372117=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372117 ] ASF GitHub Bot logged work on HIVE-22515: - Author: ASF GitHub Bot Created on: 15/Jan/20 03:30 Start Date: 15/Jan/20 03:30 Worklog Time Spent: 10m Work Description: ramesh0201 commented on pull request #880: HIVE-22515 Support cast to decimal64 in Vectorization URL: https://github.com/apache/hive/pull/880 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 372117) Remaining Estimate: 0h Time Spent: 10m > Support cast to decimal64 in Vectorization > -- > > Key: HIVE-22515 > URL: https://issues.apache.org/jira/browse/HIVE-22515 > Project: Hive > Issue Type: Bug >Reporter: Ramesh Kumar Thangarajan >Assignee: Ramesh Kumar Thangarajan >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22515.5.patch, HIVE-22515.8.patch, > HIVE-22515.9.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Support cast to decimal64 in Vectorization -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22515) Support cast to decimal64 in Vectorization
[
https://issues.apache.org/jira/browse/HIVE-22515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015590#comment-17015590
]
Hive QA commented on HIVE-22515:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12990813/HIVE-22515.9.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17878 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20176/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20176/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20176/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12990813 - PreCommit-HIVE-Build
> Support cast to decimal64 in Vectorization
> --
>
> Key: HIVE-22515
> URL: https://issues.apache.org/jira/browse/HIVE-22515
> Project: Hive
> Issue Type: Bug
>Reporter: Ramesh Kumar Thangarajan
>Assignee: Ramesh Kumar Thangarajan
>Priority: Major
> Attachments: HIVE-22515.5.patch, HIVE-22515.8.patch,
> HIVE-22515.9.patch
>
>
> Support cast to decimal64 in Vectorization
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22515) Support cast to decimal64 in Vectorization
[
https://issues.apache.org/jira/browse/HIVE-22515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015564#comment-17015564
]
Hive QA commented on HIVE-22515:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
38s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
11s{color} | {color:blue} ql in master has 1531 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
46s{color} | {color:red} ql: The patch generated 30 new + 338 unchanged - 40
fixed = 368 total (was 378) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
18s{color} | {color:red} ql generated 1 new + 1530 unchanged - 1 fixed = 1531
total (was 1531) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 26m 25s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Redundant nullcheck of filterExpr, which is known to be non-null in
org.apache.hadoop.hive.ql.exec.vector.VectorizationContext.createDecimal64VectorExpression(Class,
List, VectorExpressionDescriptor$Mode, boolean, int, TypeInfo,
DataTypePhysicalVariation, boolean, GenericUDF) Redundant null check at
VectorizationContext.java:is known to be non-null in
org.apache.hadoop.hive.ql.exec.vector.VectorizationContext.createDecimal64VectorExpression(Class,
List, VectorExpressionDescriptor$Mode, boolean, int, TypeInfo,
DataTypePhysicalVariation, boolean, GenericUDF) Redundant null check at
VectorizationContext.java:[line 1945] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20176/dev-support/hive-personality.sh
|
| git revision | master / 72d56f4 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20176/yetus/diff-checkstyle-ql.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20176/yetus/new-findbugs-ql.html
|
| modules | C: ql itests U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20176/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Support cast to decimal64 in Vectorization
> --
>
> Key: HIVE-22515
> URL: https://issues.apache.org/jira/browse/HIVE-22515
> Project: Hive
> Issue Type: Bug
>Reporter: Ramesh Kumar Thangarajan
>Assignee: Ramesh Kumar Thangarajan
>Priority: Major
> Attachments: HIVE-22515.5.patch,
[jira] [Commented] (HIVE-20288) DROP TABLE throws any error/warning while an attempt is made to drop a table which does not exist .
[ https://issues.apache.org/jira/browse/HIVE-20288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015548#comment-17015548 ] KarlManong commented on HIVE-20288: --- accoding to the document “https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-DropTable“ , the default 'hive.exec.drop.ignorenonexistent' should be false > DROP TABLE throws any error/warning while an attempt is made to drop a table > which does not exist . > --- > > Key: HIVE-20288 > URL: https://issues.apache.org/jira/browse/HIVE-20288 > Project: Hive > Issue Type: Bug >Affects Versions: 1.2.1, 2.1.1 >Reporter: vaibhav >Priority: Major > > When I do DROP TABLE ( for a table which really does not exist in the > database) in hive .It really don't throw any warning/error and just show OK > (the same message as the table is present and was dropped ). > = > hive> show tables; > OK > Time taken: 0.921 seconds > hive> > > drop table tabledoesnotexist; > OK > Time taken: 0.355 seconds > > > Hive.log : > 2018-08-01 09:36:07,575 ERROR [main]: metadata.Hive > (Hive.java:getTable(1215)) - Table tabledoesnotexist not found: > temp.tabledoesnotexist table not found > > hive> show tables; > OK > Time taken: 2.096 seconds > hive> drop table if exists tabledoesnotexist; > OK > Time taken: 0.145 seconds > > hive> create table test(id int); > OK > Time taken: 1.258 seconds > hive> drop table test; > OK > Time taken: 0.561 seconds > > > In general, the database should return an Error if the table is not present. > Eg : MySQL: > mysql> drop table test; > [Code: 1051, SQL State: 42S02] Unknown table 'test' > > > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22719) Remove Log from HiveConf::getLogIdVar
[
https://issues.apache.org/jira/browse/HIVE-22719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015544#comment-17015544
]
Hive QA commented on HIVE-22719:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12990780/HIVE-22719.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17876 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20175/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20175/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20175/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12990780 - PreCommit-HIVE-Build
> Remove Log from HiveConf::getLogIdVar
> -
>
> Key: HIVE-22719
> URL: https://issues.apache.org/jira/browse/HIVE-22719
> Project: Hive
> Issue Type: Improvement
>Reporter: Rajesh Balamohan
>Priority: Trivial
> Attachments: ExecuteStatemt_getResult.jpg, HIVE-22719.1.patch,
> HIVE-22719.2.patch
>
>
> Log statement gets in the hotpath when executing large number of tiny sql
> statements.
> !ExecuteStatemt_getResult.jpg|width=260,height=177!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22381) JMH tests for Decimal64 and Decimal division on a column over scalar
[ https://issues.apache.org/jira/browse/HIVE-22381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015522#comment-17015522 ] Ashutosh Chauhan commented on HIVE-22381: - [~rameshkumar] these jmh tests are still useful, right? shall we get this in? > JMH tests for Decimal64 and Decimal division on a column over scalar > > > Key: HIVE-22381 > URL: https://issues.apache.org/jira/browse/HIVE-22381 > Project: Hive > Issue Type: Bug >Reporter: Ramesh Kumar Thangarajan >Assignee: Ramesh Kumar Thangarajan >Priority: Major > Attachments: HIVE-22381.1.patch, errors.xml > > > JMH tests for Decimal64 and Decimal division on a column over scalar -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22719) Remove Log from HiveConf::getLogIdVar
[
https://issues.apache.org/jira/browse/HIVE-22719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015521#comment-17015521
]
Hive QA commented on HIVE-22719:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
33s{color} | {color:blue} common in master has 63 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
13s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 13m 10s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20175/dev-support/hive-personality.sh
|
| git revision | master / 72d56f4 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: common U: common |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20175/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Remove Log from HiveConf::getLogIdVar
> -
>
> Key: HIVE-22719
> URL: https://issues.apache.org/jira/browse/HIVE-22719
> Project: Hive
> Issue Type: Improvement
>Reporter: Rajesh Balamohan
>Priority: Trivial
> Attachments: ExecuteStatemt_getResult.jpg, HIVE-22719.1.patch,
> HIVE-22719.2.patch
>
>
> Log statement gets in the hotpath when executing large number of tiny sql
> statements.
> !ExecuteStatemt_getResult.jpg|width=260,height=177!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22724) Reduce number of DB calls in ObjectStore, TxnHandler
[
https://issues.apache.org/jira/browse/HIVE-22724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015505#comment-17015505
]
Hive QA commented on HIVE-22724:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12990821/HIVE-22724.3.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17876 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_memcheck]
(batchId=48)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20174/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20174/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20174/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12990821 - PreCommit-HIVE-Build
> Reduce number of DB calls in ObjectStore, TxnHandler
>
>
> Key: HIVE-22724
> URL: https://issues.apache.org/jira/browse/HIVE-22724
> Project: Hive
> Issue Type: Improvement
>Reporter: Rajesh Balamohan
>Assignee: Rajesh Balamohan
>Priority: Minor
> Attachments: HIVE-22724.1.patch, HIVE-22724.2.patch,
> HIVE-22724.3.patch, Screenshot 2020-01-14 at 4.55.12 AM.png
>
>
> !Screenshot 2020-01-14 at 4.55.12 AM.png|width=668,height=310!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22724) Reduce number of DB calls in ObjectStore, TxnHandler
[
https://issues.apache.org/jira/browse/HIVE-22724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015468#comment-17015468
]
Hive QA commented on HIVE-22724:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
24s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
11s{color} | {color:blue} standalone-metastore/metastore-server in master has
181 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
25s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 1 new + 906 unchanged - 1 fixed = 907 total (was 907) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 15m 40s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20174/dev-support/hive-personality.sh
|
| git revision | master / 72d56f4 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20174/yetus/diff-checkstyle-standalone-metastore_metastore-server.txt
|
| modules | C: standalone-metastore/metastore-server U:
standalone-metastore/metastore-server |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20174/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Reduce number of DB calls in ObjectStore, TxnHandler
>
>
> Key: HIVE-22724
> URL: https://issues.apache.org/jira/browse/HIVE-22724
> Project: Hive
> Issue Type: Improvement
>Reporter: Rajesh Balamohan
>Assignee: Rajesh Balamohan
>Priority: Minor
> Attachments: HIVE-22724.1.patch, HIVE-22724.2.patch,
> HIVE-22724.3.patch, Screenshot 2020-01-14 at 4.55.12 AM.png
>
>
> !Screenshot 2020-01-14 at 4.55.12 AM.png|width=668,height=310!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22725) Lazy evaluate HiveMetastore::fireReadTablePreEvent table computation
[
https://issues.apache.org/jira/browse/HIVE-22725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015455#comment-17015455
]
Hive QA commented on HIVE-22725:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12990833/HIVE-22725.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 68 failed/errored test(s), 17876 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.metastore.TestEmbeddedHiveMetaStore.testPartitionFilter
(batchId=230)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStore.testPartitionFilter
(batchId=232)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreCustomAuth.testPartitionFilter
(batchId=236)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testPartitionFilter
(batchId=231)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testPartitionFilter
(batchId=238)
org.apache.hadoop.hive.metastore.TestSetUGIOnBothClientServer.testPartitionFilter
(batchId=228)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testPartitionFilter
(batchId=226)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testPartitionFilter
(batchId=237)
org.apache.hadoop.hive.metastore.client.TestGetPartitions.getPartitionsByNamesBogusCatalog[Embedded]
(batchId=229)
org.apache.hadoop.hive.metastore.client.TestGetPartitions.getPartitionsByNamesBogusCatalog[Remote]
(batchId=229)
org.apache.hadoop.hive.metastore.client.TestGetPartitions.testGetPartitionWithAuthInfoNullDbName[Embedded]
(batchId=229)
org.apache.hadoop.hive.metastore.client.TestGetPartitions.testGetPartitionWithAuthInfoNullDbName[Remote]
(batchId=229)
org.apache.hadoop.hive.metastore.client.TestGetPartitions.testGetPartitionWithAuthInfoNullTblName[Embedded]
(batchId=229)
org.apache.hadoop.hive.metastore.client.TestGetPartitions.testGetPartitionWithAuthInfoNullTblName[Remote]
(batchId=229)
org.apache.hadoop.hive.metastore.client.TestGetPartitions.testGetPartitionsByNamesNoDb[Embedded]
(batchId=229)
org.apache.hadoop.hive.metastore.client.TestGetPartitions.testGetPartitionsByNamesNoDb[Remote]
(batchId=229)
org.apache.hadoop.hive.metastore.client.TestGetPartitions.testGetPartitionsByNamesNoTable[Embedded]
(batchId=229)
org.apache.hadoop.hive.metastore.client.TestGetPartitions.testGetPartitionsByNamesNoTable[Remote]
(batchId=229)
org.apache.hadoop.hive.metastore.client.TestListPartitions.listPartitionNamesBogusCatalog[Embedded]
(batchId=227)
org.apache.hadoop.hive.metastore.client.TestListPartitions.listPartitionNamesBogusCatalog[Remote]
(batchId=227)
org.apache.hadoop.hive.metastore.client.TestListPartitions.testListPartitionNamesByValuesNullDbName[Embedded]
(batchId=227)
org.apache.hadoop.hive.metastore.client.TestListPartitions.testListPartitionNamesByValuesNullDbName[Remote]
(batchId=227)
org.apache.hadoop.hive.metastore.client.TestListPartitions.testListPartitionNamesByValuesNullTblName[Embedded]
(batchId=227)
org.apache.hadoop.hive.metastore.client.TestListPartitions.testListPartitionNamesByValuesNullTblName[Remote]
(batchId=227)
org.apache.hadoop.hive.metastore.client.TestListPartitions.testListPartitionNamesNoDbName[Embedded]
(batchId=227)
org.apache.hadoop.hive.metastore.client.TestListPartitions.testListPartitionNamesNoDbName[Remote]
(batchId=227)
org.apache.hadoop.hive.metastore.client.TestListPartitions.testListPartitionNamesNoDb[Embedded]
(batchId=227)
org.apache.hadoop.hive.metastore.client.TestListPartitions.testListPartitionNamesNoDb[Remote]
(batchId=227)
org.apache.hadoop.hive.metastore.client.TestListPartitions.testListPartitionNamesNoTable[Embedded]
(batchId=227)
org.apache.hadoop.hive.metastore.client.TestListPartitions.testListPartitionNamesNoTable[Remote]
(batchId=227)
org.apache.hadoop.hive.metastore.client.TestListPartitions.testListPartitionNamesNoTblName[Embedded]
(batchId=227)
org.apache.hadoop.hive.metastore.client.TestListPartitions.testListPartitionNamesNoTblName[Remote]
(batchId=227)
org.apache.hadoop.hive.metastore.client.TestListPartitions.testListPartitionNamesNullDbName[Embedded]
(batchId=227)
org.apache.hadoop.hive.metastore.client.TestListPartitions.testListPartitionNamesNullDbName[Remote]
(batchId=227)
org.apache.hadoop.hive.metastore.client.TestListPartitions.testListPartitionNamesNullTblName[Embedded]
(batchId=227)
org.apache.hadoop.hive.metastore.client.TestListPartitions.testListPartitionNamesNullTblName[Remote]
(batchId=227)
org.apache.hadoop.hive.metastore.client.TestListPartitions.testListPartitionsAllNullDbName[Embedded]
(batchId=227)
org.apache.hadoop.hive.metastore.client.TestListPartitions.testListPartitionsAllNullDbName[Remote]
(batchId=227)
org.apache.hadoop.hive.metastore.client.TestListPartitions.testListPartitionsAllNullTblName[Embedded]
(batchId=227)
[jira] [Commented] (HIVE-22725) Lazy evaluate HiveMetastore::fireReadTablePreEvent table computation
[
https://issues.apache.org/jira/browse/HIVE-22725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015414#comment-17015414
]
Hive QA commented on HIVE-22725:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
21s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
19s{color} | {color:blue} standalone-metastore/metastore-server in master has
181 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
20s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 15m 56s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20173/dev-support/hive-personality.sh
|
| git revision | master / 72d56f4 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: standalone-metastore/metastore-server U:
standalone-metastore/metastore-server |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20173/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Lazy evaluate HiveMetastore::fireReadTablePreEvent table computation
>
>
> Key: HIVE-22725
> URL: https://issues.apache.org/jira/browse/HIVE-22725
> Project: Hive
> Issue Type: Improvement
>Reporter: Rajesh Balamohan
>Assignee: Rajesh Balamohan
>Priority: Minor
> Attachments: HIVE-22725.1.patch, HIVE-22725.2.patch,
> image-2020-01-14-13-22-54-483.png
>
>
> "TransactionalValidationListener" gets added in the pre-event listeners of
> HMS by default.
> [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java#L559]
> This causes issue in short select queries, as table details are computed for
> any partition lookups.
> [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java#L4984]
>
> !image-2020-01-14-13-22-54-483.png|width=579,height=202!
>
> It would be good to lazy evaluate table lookup in this codepath.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-21341) Sensible defaults : hive.server2.idle.operation.timeout and hive.server2.idle.session.timeout are too high
[
https://issues.apache.org/jira/browse/HIVE-21341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015392#comment-17015392
]
Hive QA commented on HIVE-21341:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12990775/HIVE-21341.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17876 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[timestamptz_2]
(batchId=90)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20172/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20172/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20172/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12990775 - PreCommit-HIVE-Build
> Sensible defaults : hive.server2.idle.operation.timeout and
> hive.server2.idle.session.timeout are too high
> --
>
> Key: HIVE-21341
> URL: https://issues.apache.org/jira/browse/HIVE-21341
> Project: Hive
> Issue Type: Improvement
> Components: Configuration, HiveServer2
>Reporter: Ashutosh Chauhan
>Assignee: Ashutosh Chauhan
>Priority: Major
> Attachments: HIVE-21341.1.patch, HIVE-21341.2.patch, HIVE-21341.patch
>
>
> Defaults are too high, which results in extra resources being held too long
> in HS2 memory.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22653) Remove commons-lang leftovers
[
https://issues.apache.org/jira/browse/HIVE-22653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015391#comment-17015391
]
Hive QA commented on HIVE-22653:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
32s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 17m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 10m
21s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
23s{color} | {color:blue} storage-api in master has 58 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
17s{color} | {color:blue} shims/common in master has 6 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
21s{color} | {color:blue} shims/0.23 in master has 7 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
23s{color} | {color:blue} standalone-metastore/metastore-common in master has
37 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
35s{color} | {color:blue} common in master has 63 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
39s{color} | {color:blue} serde in master has 198 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
28s{color} | {color:blue} llap-common in master has 90 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
26s{color} | {color:blue} llap-tez in master has 18 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
24s{color} | {color:blue} spark-client in master has 10 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
13s{color} | {color:blue} standalone-metastore/metastore-server in master has
181 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
0s{color} | {color:blue} ql in master has 1531 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
41s{color} | {color:blue} llap-server in master has 90 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
39s{color} | {color:blue} service in master has 50 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
31s{color} | {color:blue} accumulo-handler in master has 20 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
27s{color} | {color:blue} jdbc in master has 16 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
29s{color} | {color:blue} beeline in master has 48 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
29s{color} | {color:blue} cli in master has 9 extant Findbugs warnings. {color}
|
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
28s{color} | {color:blue} druid-handler in master has 3 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
34s{color} | {color:blue} hbase-handler in master has 15 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
37s{color} | {color:blue} hcatalog/core in master has 36 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
26s{color} | {color:blue} hcatalog/hcatalog-pig-adapter in master has 2 extant
Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
27s{color} | {color:blue} hcatalog/server-extensions in master has 3 extant
Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
28s{color} | {color:blue} hcatalog/webhcat/java-client in master has 3 extant
[jira] [Updated] (HIVE-22400) UDF minute with time returns NULL
[
https://issues.apache.org/jira/browse/HIVE-22400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-22400:
---
Fix Version/s: 4.0.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
Pushed to master, thanks [~nareshpr]!
> UDF minute with time returns NULL
> -
>
> Key: HIVE-22400
> URL: https://issues.apache.org/jira/browse/HIVE-22400
> Project: Hive
> Issue Type: Bug
> Components: UDF
>Affects Versions: 3.1.0
>Reporter: Nikunj
>Assignee: Naresh P R
>Priority: Minor
> Fix For: 4.0.0
>
> Attachments: HIVE-22400.1.patch, HIVE-22400.2.patch,
> HIVE-22400.3.patch, HIVE-22400.patch
>
>
> [impadmin@impetus-g031 ~]$ beeline
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in
> [jar:file:/usr/hdp/3.1.0.0-78/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/usr/hdp/3.1.0.0-78/hadoop/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
> Connecting to
> jdbc:hive2://impetus-dsrv11.impetus.co.in:2181,ct-n0066.impetus.co.in:2181,ct-n0092.impetus.co.in:2181/default;principal=hive/_h...@impetus.co.in;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2
> 19/10/24 19:03:42 [main]: INFO jdbc.HiveConnection: Connected to
> ct-n0092.impetus.co.in:1
> Connected to: Apache Hive (version 3.1.0.3.1.0.0-78)
> Driver: Hive JDBC (version 3.1.0.3.1.0.0-78)
> Transaction isolation: TRANSACTION_REPEATABLE_READ
> Beeline version 3.1.0.3.1.0.0-78 by Apache Hive
> 0: jdbc:hive2://impetus-dsrv11.impetus.co.in:> select minute('12:58:59');
> INFO : Compiling
> command(queryId=hive_20191024190401_bc517191-bd20-4f5a-b5f5-44f762c2d395):
> select minute('12:58:59')
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:_c0,
> type:int, comment:null)], properties:null)
> INFO : Completed compiling
> command(queryId=hive_20191024190401_bc517191-bd20-4f5a-b5f5-44f762c2d395);
> Time taken: 0.427 seconds
> INFO : Executing
> command(queryId=hive_20191024190401_bc517191-bd20-4f5a-b5f5-44f762c2d395):
> select minute('12:58:59')
> INFO : Completed executing
> command(queryId=hive_20191024190401_bc517191-bd20-4f5a-b5f5-44f762c2d395);
> Time taken: 0.003 seconds
> INFO : OK
> +---+
> | _c0 |
> +---+
> | NULL |
> +---+
> 1 row selected (0.739 seconds)
> 0: jdbc:hive2://impetus-dsrv11.impetus.co.in:>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22489) Reduce Sink operator should order nulls by parameter
[
https://issues.apache.org/jira/browse/HIVE-22489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22489:
--
Attachment: HIVE-22489.13.patch
> Reduce Sink operator should order nulls by parameter
> -
>
> Key: HIVE-22489
> URL: https://issues.apache.org/jira/browse/HIVE-22489
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22489.1.patch, HIVE-22489.10.patch,
> HIVE-22489.10.patch, HIVE-22489.11.patch, HIVE-22489.12.patch,
> HIVE-22489.13.patch, HIVE-22489.2.patch, HIVE-22489.3.patch,
> HIVE-22489.3.patch, HIVE-22489.4.patch, HIVE-22489.5.patch,
> HIVE-22489.6.patch, HIVE-22489.7.patch, HIVE-22489.8.patch,
> HIVE-22489.9.patch, HIVE-22489.9.patch
>
>
> When the property hive.default.nulls.last is set to true and no null order is
> explicitly specified in the ORDER BY clause of the query null ordering should
> be NULLS LAST.
> But some of the Reduce Sink operators still orders null first.
> {code}
> SET hive.default.nulls.last=true;
> EXPLAIN EXTENDED
> SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key =
> src2.key) ORDER BY src1.key LIMIT 5;
> {code}
> {code}
> PREHOOK: query: EXPLAIN EXTENDED
> SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key =
> src2.key) ORDER BY src1.key
> PREHOOK: type: QUERY
> PREHOOK: Input: default@src
> A masked pattern was here
> POSTHOOK: query: EXPLAIN EXTENDED
> SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key =
> src2.key) ORDER BY src1.key
> POSTHOOK: type: QUERY
> POSTHOOK: Input: default@src
> A masked pattern was here
> OPTIMIZED SQL: SELECT `t0`.`key`, `t2`.`value`
> FROM (SELECT `key`
> FROM `default`.`src`
> WHERE `key` IS NOT NULL) AS `t0`
> INNER JOIN (SELECT `key`, `value`
> FROM `default`.`src`
> WHERE `key` IS NOT NULL) AS `t2` ON `t0`.`key` = `t2`.`key`
> ORDER BY `t0`.`key`
> STAGE DEPENDENCIES:
> Stage-1 is a root stage
> Stage-0 depends on stages: Stage-1
> STAGE PLANS:
> Stage: Stage-1
> Tez
> A masked pattern was here
> Edges:
> Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
> Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
> A masked pattern was here
> Vertices:
> Map 1
> Map Operator Tree:
> TableScan
> alias: src1
> filterExpr: key is not null (type: boolean)
> Statistics: Num rows: 500 Data size: 43500 Basic stats:
> COMPLETE Column stats: COMPLETE
> GatherStats: false
> Filter Operator
> isSamplingPred: false
> predicate: key is not null (type: boolean)
> Statistics: Num rows: 500 Data size: 43500 Basic stats:
> COMPLETE Column stats: COMPLETE
> Select Operator
> expressions: key (type: string)
> outputColumnNames: _col0
> Statistics: Num rows: 500 Data size: 43500 Basic stats:
> COMPLETE Column stats: COMPLETE
> Reduce Output Operator
> key expressions: _col0 (type: string)
> null sort order: a
> sort order: +
> Map-reduce partition columns: _col0 (type: string)
> Statistics: Num rows: 500 Data size: 43500 Basic
> stats: COMPLETE Column stats: COMPLETE
> tag: 0
> auto parallelism: true
> Execution mode: vectorized, llap
> LLAP IO: no inputs
> Path -> Alias:
> A masked pattern was here
> Path -> Partition:
> A masked pattern was here
> Partition
> base file name: src
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> COLUMN_STATS_ACCURATE
> {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
> bucket_count -1
> bucketing_version 2
> column.name.delimiter ,
> columns key,value
> columns.comments 'default','default'
> columns.types string:string
> A masked pattern was here
> name default.src
> numFiles 1
> numRows 500
> rawDataSize 5312
> serialization.ddl struct src { string key, string
[jira] [Updated] (HIVE-22489) Reduce Sink operator should order nulls by parameter
[
https://issues.apache.org/jira/browse/HIVE-22489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22489:
--
Status: Open (was: Patch Available)
> Reduce Sink operator should order nulls by parameter
> -
>
> Key: HIVE-22489
> URL: https://issues.apache.org/jira/browse/HIVE-22489
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22489.1.patch, HIVE-22489.10.patch,
> HIVE-22489.10.patch, HIVE-22489.11.patch, HIVE-22489.12.patch,
> HIVE-22489.13.patch, HIVE-22489.2.patch, HIVE-22489.3.patch,
> HIVE-22489.3.patch, HIVE-22489.4.patch, HIVE-22489.5.patch,
> HIVE-22489.6.patch, HIVE-22489.7.patch, HIVE-22489.8.patch,
> HIVE-22489.9.patch, HIVE-22489.9.patch
>
>
> When the property hive.default.nulls.last is set to true and no null order is
> explicitly specified in the ORDER BY clause of the query null ordering should
> be NULLS LAST.
> But some of the Reduce Sink operators still orders null first.
> {code}
> SET hive.default.nulls.last=true;
> EXPLAIN EXTENDED
> SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key =
> src2.key) ORDER BY src1.key LIMIT 5;
> {code}
> {code}
> PREHOOK: query: EXPLAIN EXTENDED
> SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key =
> src2.key) ORDER BY src1.key
> PREHOOK: type: QUERY
> PREHOOK: Input: default@src
> A masked pattern was here
> POSTHOOK: query: EXPLAIN EXTENDED
> SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key =
> src2.key) ORDER BY src1.key
> POSTHOOK: type: QUERY
> POSTHOOK: Input: default@src
> A masked pattern was here
> OPTIMIZED SQL: SELECT `t0`.`key`, `t2`.`value`
> FROM (SELECT `key`
> FROM `default`.`src`
> WHERE `key` IS NOT NULL) AS `t0`
> INNER JOIN (SELECT `key`, `value`
> FROM `default`.`src`
> WHERE `key` IS NOT NULL) AS `t2` ON `t0`.`key` = `t2`.`key`
> ORDER BY `t0`.`key`
> STAGE DEPENDENCIES:
> Stage-1 is a root stage
> Stage-0 depends on stages: Stage-1
> STAGE PLANS:
> Stage: Stage-1
> Tez
> A masked pattern was here
> Edges:
> Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
> Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
> A masked pattern was here
> Vertices:
> Map 1
> Map Operator Tree:
> TableScan
> alias: src1
> filterExpr: key is not null (type: boolean)
> Statistics: Num rows: 500 Data size: 43500 Basic stats:
> COMPLETE Column stats: COMPLETE
> GatherStats: false
> Filter Operator
> isSamplingPred: false
> predicate: key is not null (type: boolean)
> Statistics: Num rows: 500 Data size: 43500 Basic stats:
> COMPLETE Column stats: COMPLETE
> Select Operator
> expressions: key (type: string)
> outputColumnNames: _col0
> Statistics: Num rows: 500 Data size: 43500 Basic stats:
> COMPLETE Column stats: COMPLETE
> Reduce Output Operator
> key expressions: _col0 (type: string)
> null sort order: a
> sort order: +
> Map-reduce partition columns: _col0 (type: string)
> Statistics: Num rows: 500 Data size: 43500 Basic
> stats: COMPLETE Column stats: COMPLETE
> tag: 0
> auto parallelism: true
> Execution mode: vectorized, llap
> LLAP IO: no inputs
> Path -> Alias:
> A masked pattern was here
> Path -> Partition:
> A masked pattern was here
> Partition
> base file name: src
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> COLUMN_STATS_ACCURATE
> {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
> bucket_count -1
> bucketing_version 2
> column.name.delimiter ,
> columns key,value
> columns.comments 'default','default'
> columns.types string:string
> A masked pattern was here
> name default.src
> numFiles 1
> numRows 500
> rawDataSize 5312
> serialization.ddl struct src { string key, string
[jira] [Updated] (HIVE-22489) Reduce Sink operator should order nulls by parameter
[
https://issues.apache.org/jira/browse/HIVE-22489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22489:
--
Status: Patch Available (was: Open)
> Reduce Sink operator should order nulls by parameter
> -
>
> Key: HIVE-22489
> URL: https://issues.apache.org/jira/browse/HIVE-22489
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22489.1.patch, HIVE-22489.10.patch,
> HIVE-22489.10.patch, HIVE-22489.11.patch, HIVE-22489.12.patch,
> HIVE-22489.13.patch, HIVE-22489.2.patch, HIVE-22489.3.patch,
> HIVE-22489.3.patch, HIVE-22489.4.patch, HIVE-22489.5.patch,
> HIVE-22489.6.patch, HIVE-22489.7.patch, HIVE-22489.8.patch,
> HIVE-22489.9.patch, HIVE-22489.9.patch
>
>
> When the property hive.default.nulls.last is set to true and no null order is
> explicitly specified in the ORDER BY clause of the query null ordering should
> be NULLS LAST.
> But some of the Reduce Sink operators still orders null first.
> {code}
> SET hive.default.nulls.last=true;
> EXPLAIN EXTENDED
> SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key =
> src2.key) ORDER BY src1.key LIMIT 5;
> {code}
> {code}
> PREHOOK: query: EXPLAIN EXTENDED
> SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key =
> src2.key) ORDER BY src1.key
> PREHOOK: type: QUERY
> PREHOOK: Input: default@src
> A masked pattern was here
> POSTHOOK: query: EXPLAIN EXTENDED
> SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key =
> src2.key) ORDER BY src1.key
> POSTHOOK: type: QUERY
> POSTHOOK: Input: default@src
> A masked pattern was here
> OPTIMIZED SQL: SELECT `t0`.`key`, `t2`.`value`
> FROM (SELECT `key`
> FROM `default`.`src`
> WHERE `key` IS NOT NULL) AS `t0`
> INNER JOIN (SELECT `key`, `value`
> FROM `default`.`src`
> WHERE `key` IS NOT NULL) AS `t2` ON `t0`.`key` = `t2`.`key`
> ORDER BY `t0`.`key`
> STAGE DEPENDENCIES:
> Stage-1 is a root stage
> Stage-0 depends on stages: Stage-1
> STAGE PLANS:
> Stage: Stage-1
> Tez
> A masked pattern was here
> Edges:
> Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
> Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
> A masked pattern was here
> Vertices:
> Map 1
> Map Operator Tree:
> TableScan
> alias: src1
> filterExpr: key is not null (type: boolean)
> Statistics: Num rows: 500 Data size: 43500 Basic stats:
> COMPLETE Column stats: COMPLETE
> GatherStats: false
> Filter Operator
> isSamplingPred: false
> predicate: key is not null (type: boolean)
> Statistics: Num rows: 500 Data size: 43500 Basic stats:
> COMPLETE Column stats: COMPLETE
> Select Operator
> expressions: key (type: string)
> outputColumnNames: _col0
> Statistics: Num rows: 500 Data size: 43500 Basic stats:
> COMPLETE Column stats: COMPLETE
> Reduce Output Operator
> key expressions: _col0 (type: string)
> null sort order: a
> sort order: +
> Map-reduce partition columns: _col0 (type: string)
> Statistics: Num rows: 500 Data size: 43500 Basic
> stats: COMPLETE Column stats: COMPLETE
> tag: 0
> auto parallelism: true
> Execution mode: vectorized, llap
> LLAP IO: no inputs
> Path -> Alias:
> A masked pattern was here
> Path -> Partition:
> A masked pattern was here
> Partition
> base file name: src
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> COLUMN_STATS_ACCURATE
> {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
> bucket_count -1
> bucketing_version 2
> column.name.delimiter ,
> columns key,value
> columns.comments 'default','default'
> columns.types string:string
> A masked pattern was here
> name default.src
> numFiles 1
> numRows 500
> rawDataSize 5312
> serialization.ddl struct src { string key, string
[jira] [Commented] (HIVE-21341) Sensible defaults : hive.server2.idle.operation.timeout and hive.server2.idle.session.timeout are too high
[
https://issues.apache.org/jira/browse/HIVE-21341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015364#comment-17015364
]
Hive QA commented on HIVE-21341:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 11m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
27s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
4s{color} | {color:blue} common in master has 63 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
23s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
25s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 18m 3s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20172/dev-support/hive-personality.sh
|
| git revision | master / 1f541b5 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: common U: common |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20172/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Sensible defaults : hive.server2.idle.operation.timeout and
> hive.server2.idle.session.timeout are too high
> --
>
> Key: HIVE-21341
> URL: https://issues.apache.org/jira/browse/HIVE-21341
> Project: Hive
> Issue Type: Improvement
> Components: Configuration, HiveServer2
>Reporter: Ashutosh Chauhan
>Assignee: Ashutosh Chauhan
>Priority: Major
> Attachments: HIVE-21341.1.patch, HIVE-21341.2.patch, HIVE-21341.patch
>
>
> Defaults are too high, which results in extra resources being held too long
> in HS2 memory.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22727) Add hive db schema changes introduced in HIVE-21884 to the schema upgrade scripts
[ https://issues.apache.org/jira/browse/HIVE-22727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015353#comment-17015353 ] Zoltan Haindrich commented on HIVE-22727: - oh...I've missed updating this part; thank you for fixing it! +1 pending qa run > Add hive db schema changes introduced in HIVE-21884 to the schema upgrade > scripts > - > > Key: HIVE-22727 > URL: https://issues.apache.org/jira/browse/HIVE-22727 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Chovan >Assignee: Zoltan Chovan >Priority: Minor > Labels: pull-request-available > Attachments: HIVE-22727.patch > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22653) Remove commons-lang leftovers
[
https://issues.apache.org/jira/browse/HIVE-22653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015347#comment-17015347
]
Hive QA commented on HIVE-22653:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12990750/HIVE-22653.07.patch
{color:green}SUCCESS:{color} +1 due to 24 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17876 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20171/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20171/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20171/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12990750 - PreCommit-HIVE-Build
> Remove commons-lang leftovers
> -
>
> Key: HIVE-22653
> URL: https://issues.apache.org/jira/browse/HIVE-22653
> Project: Hive
> Issue Type: Bug
>Reporter: David Lavati
>Assignee: David Lavati
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-22653.01.patch, HIVE-22653.01.patch,
> HIVE-22653.02.patch, HIVE-22653.03.patch, HIVE-22653.04.patch,
> HIVE-22653.04.patch, HIVE-22653.04.patch, HIVE-22653.05.patch,
> HIVE-22653.06.patch, HIVE-22653.07.patch, HIVE-22653.07.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> HIVE-7145 removed commons-lang - in favor of commons-lang3 - as a direct
> dependency, however a high number of files still refer to commons-lang, which
> is transitively brought in either way.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22627) Add schema changes introduced in HIVE-21443 to the schema upgrade scripts
[ https://issues.apache.org/jira/browse/HIVE-22627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Chovan updated HIVE-22627: - Status: Patch Available (was: Open) > Add schema changes introduced in HIVE-21443 to the schema upgrade scripts > - > > Key: HIVE-22627 > URL: https://issues.apache.org/jira/browse/HIVE-22627 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Chovan >Assignee: Zoltan Chovan >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22627.patch > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22727) Add hive db schema changes introduced in HIVE-21884 to the schema upgrade scripts
[ https://issues.apache.org/jira/browse/HIVE-22727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Chovan updated HIVE-22727: - Status: Patch Available (was: Open) > Add hive db schema changes introduced in HIVE-21884 to the schema upgrade > scripts > - > > Key: HIVE-22727 > URL: https://issues.apache.org/jira/browse/HIVE-22727 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Chovan >Assignee: Zoltan Chovan >Priority: Minor > Labels: pull-request-available > Attachments: HIVE-22727.patch > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22726) TopN Key optimizer should use array instead of priority queue
[ https://issues.apache.org/jira/browse/HIVE-22726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015267#comment-17015267 ] Attila Magyar commented on HIVE-22726: -- cc: [~rzhappy] > TopN Key optimizer should use array instead of priority queue > - > > Key: HIVE-22726 > URL: https://issues.apache.org/jira/browse/HIVE-22726 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Attila Magyar >Assignee: Attila Magyar >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22726.1.patch > > > The TopN key optimizer currently uses a priority queue for keeping track of > the largest/smallest rows. Its max size is the same as the user specified > limit. This should be replaced a more cache line friendly array with a small > (128) maximum size and see how much performance is gained. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22663) Quote all table and column names or do not quote any
[
https://issues.apache.org/jira/browse/HIVE-22663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015261#comment-17015261
]
Hive QA commented on HIVE-22663:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12990742/HIVE-22663.3.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 29 failed/errored test(s), 17876 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.parse.TestReplTableMigrationWithJsonFormat.testIncrementalLoadMigrationManagedToAcid
(batchId=271)
org.apache.hadoop.hive.ql.parse.TestReplTableMigrationWithJsonFormat.testIncrementalLoadMigrationManagedToAcidAllOp
(batchId=271)
org.apache.hadoop.hive.ql.parse.TestReplTableMigrationWithJsonFormat.testIncrementalLoadMigrationManagedToAcidFailure
(batchId=271)
org.apache.hadoop.hive.ql.parse.TestReplTableMigrationWithJsonFormat.testIncrementalLoadMigrationManagedToAcidFailurePart
(batchId=271)
org.apache.hadoop.hive.ql.parse.TestReplTableMigrationWithJsonFormat.testIncrementalLoadMigrationToAcidWithMoveOptimization
(batchId=271)
org.apache.hadoop.hive.ql.parse.TestReplTableMigrationWithJsonFormat.testMigrationWithUpgrade
(batchId=271)
org.apache.hadoop.hive.ql.parse.TestReplicationWithTableMigration.testIncrementalLoadMigrationManagedToAcid
(batchId=260)
org.apache.hadoop.hive.ql.parse.TestReplicationWithTableMigration.testIncrementalLoadMigrationManagedToAcidAllOp
(batchId=260)
org.apache.hadoop.hive.ql.parse.TestReplicationWithTableMigration.testIncrementalLoadMigrationManagedToAcidFailure
(batchId=260)
org.apache.hadoop.hive.ql.parse.TestReplicationWithTableMigration.testIncrementalLoadMigrationManagedToAcidFailurePart
(batchId=260)
org.apache.hadoop.hive.ql.parse.TestReplicationWithTableMigration.testIncrementalLoadMigrationToAcidWithMoveOptimization
(batchId=260)
org.apache.hadoop.hive.ql.parse.TestReplicationWithTableMigration.testMigrationWithUpgrade
(batchId=260)
org.apache.hadoop.hive.ql.parse.TestReplicationWithTableMigrationEx.testConcurrentOpDuringBootStrapDumpCreateTableReplay
(batchId=270)
org.apache.hadoop.hive.ql.parse.TestReplicationWithTableMigrationEx.testConcurrentOpDuringBootStrapDumpInsertOverwrite
(batchId=270)
org.apache.hadoop.hive.ql.parse.TestReplicationWithTableMigrationEx.testConcurrentOpDuringBootStrapDumpInsertReplay
(batchId=270)
org.apache.hadoop.hive.ql.parse.TestReplicationWithTableMigrationEx.testIncLoadPenFlagPropAlterDB
(batchId=270)
org.apache.hadoop.hive.ql.parse.TestReplicationWithTableMigrationEx.testIncLoadPenFlagWithMoveOptimization
(batchId=270)
org.apache.hadoop.hive.ql.parse.TestReplicationWithTableMigrationEx.testOnwerPropagation
(batchId=270)
org.apache.hadoop.hive.ql.parse.TestReplicationWithTableMigrationEx.testOnwerPropagationInc
(batchId=270)
org.apache.hadoop.hive.ql.parse.TestReplicationWithTableMigrationEx.testTableLevelDumpMigration
(batchId=270)
org.apache.hadoop.hive.ql.parse.TestStatsReplicationScenariosMigration.testForParallelBootstrapLoad
(batchId=262)
org.apache.hadoop.hive.ql.parse.TestStatsReplicationScenariosMigration.testMetadataOnlyDump
(batchId=262)
org.apache.hadoop.hive.ql.parse.TestStatsReplicationScenariosMigration.testNonParallelBootstrapLoad
(batchId=262)
org.apache.hadoop.hive.ql.parse.TestStatsReplicationScenariosMigration.testRetryFailure
(batchId=262)
org.apache.hadoop.hive.ql.parse.TestStatsReplicationScenariosMigrationNoAutogather.testForParallelBootstrapLoad
(batchId=272)
org.apache.hadoop.hive.ql.parse.TestStatsReplicationScenariosMigrationNoAutogather.testMetadataOnlyDump
(batchId=272)
org.apache.hadoop.hive.ql.parse.TestStatsReplicationScenariosMigrationNoAutogather.testNonParallelBootstrapLoad
(batchId=272)
org.apache.hadoop.hive.ql.parse.TestStatsReplicationScenariosMigrationNoAutogather.testRetryFailure
(batchId=272)
org.apache.hadoop.hive.ql.security.TestStorageBasedMetastoreAuthorizationProviderWithACL.testSimplePrivileges
(batchId=296)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20170/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20170/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20170/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 29 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12990742 - PreCommit-HIVE-Build
> Quote all table and column names or do not quote any
>
>
> Key: HIVE-22663
> URL:
[jira] [Commented] (HIVE-22712) ReExec Driver execute submit the query in default queue irrespective of user defined queue
[ https://issues.apache.org/jira/browse/HIVE-22712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015236#comment-17015236 ] Rajkumar Singh commented on HIVE-22712: --- Thanks [~kgyrtkirk] for review, uploaded the new patch with the suggested change. > ReExec Driver execute submit the query in default queue irrespective of user > defined queue > -- > > Key: HIVE-22712 > URL: https://issues.apache.org/jira/browse/HIVE-22712 > Project: Hive > Issue Type: Bug > Components: Hive, HiveServer2 >Affects Versions: 3.1.2 > Environment: Hive-3 >Reporter: Rajkumar Singh >Assignee: Rajkumar Singh >Priority: Major > Attachments: HIVE-22712.01.patch, HIVE-22712.02.patch, > HIVE-22712.03.patch, HIVE-22712.patch > > > we unset the queue name intentionally in > TezSessionState#startSessionAndContainers, > as a result reexec create a new session in the default queue and create a > problem, its a cumbersome to add reexec.overlay.tez.queue.name at session > level. > I could not find a better way of setting the queue name (I am open for the > suggestion here) since it can create a conflict with the Global queue name > vs user-defined queue that's why setting while initialization of > ReExecutionOverlayPlugin. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22712) ReExec Driver execute submit the query in default queue irrespective of user defined queue
[ https://issues.apache.org/jira/browse/HIVE-22712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajkumar Singh updated HIVE-22712: -- Status: Open (was: Patch Available) > ReExec Driver execute submit the query in default queue irrespective of user > defined queue > -- > > Key: HIVE-22712 > URL: https://issues.apache.org/jira/browse/HIVE-22712 > Project: Hive > Issue Type: Bug > Components: Hive, HiveServer2 >Affects Versions: 3.1.2 > Environment: Hive-3 >Reporter: Rajkumar Singh >Assignee: Rajkumar Singh >Priority: Major > Attachments: HIVE-22712.01.patch, HIVE-22712.02.patch, > HIVE-22712.03.patch, HIVE-22712.patch > > > we unset the queue name intentionally in > TezSessionState#startSessionAndContainers, > as a result reexec create a new session in the default queue and create a > problem, its a cumbersome to add reexec.overlay.tez.queue.name at session > level. > I could not find a better way of setting the queue name (I am open for the > suggestion here) since it can create a conflict with the Global queue name > vs user-defined queue that's why setting while initialization of > ReExecutionOverlayPlugin. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22712) ReExec Driver execute submit the query in default queue irrespective of user defined queue
[ https://issues.apache.org/jira/browse/HIVE-22712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajkumar Singh updated HIVE-22712: -- Attachment: HIVE-22712.03.patch Status: Patch Available (was: Open) > ReExec Driver execute submit the query in default queue irrespective of user > defined queue > -- > > Key: HIVE-22712 > URL: https://issues.apache.org/jira/browse/HIVE-22712 > Project: Hive > Issue Type: Bug > Components: Hive, HiveServer2 >Affects Versions: 3.1.2 > Environment: Hive-3 >Reporter: Rajkumar Singh >Assignee: Rajkumar Singh >Priority: Major > Attachments: HIVE-22712.01.patch, HIVE-22712.02.patch, > HIVE-22712.03.patch, HIVE-22712.patch > > > we unset the queue name intentionally in > TezSessionState#startSessionAndContainers, > as a result reexec create a new session in the default queue and create a > problem, its a cumbersome to add reexec.overlay.tez.queue.name at session > level. > I could not find a better way of setting the queue name (I am open for the > suggestion here) since it can create a conflict with the Global queue name > vs user-defined queue that's why setting while initialization of > ReExecutionOverlayPlugin. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22727) Add hive db schema changes introduced in HIVE-21884 to the schema upgrade scripts
[ https://issues.apache.org/jira/browse/HIVE-22727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-22727: -- Labels: pull-request-available (was: ) > Add hive db schema changes introduced in HIVE-21884 to the schema upgrade > scripts > - > > Key: HIVE-22727 > URL: https://issues.apache.org/jira/browse/HIVE-22727 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Chovan >Assignee: Zoltan Chovan >Priority: Minor > Labels: pull-request-available > Attachments: HIVE-22727.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-22727) Add hive db schema changes introduced in HIVE-21884 to the schema upgrade scripts
[ https://issues.apache.org/jira/browse/HIVE-22727?focusedWorklogId=371725=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-371725 ] ASF GitHub Bot logged work on HIVE-22727: - Author: ASF GitHub Bot Created on: 14/Jan/20 17:32 Start Date: 14/Jan/20 17:32 Worklog Time Spent: 10m Work Description: zchovan commented on pull request #879: HIVE-22727: Add hive db schema changes introduced in HIVE-21884 to th… URL: https://github.com/apache/hive/pull/879 …e schema upgrade scripts Change-Id: I086ff4380d638fb748734cd9e5cb3ba45ccf5d5e This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 371725) Remaining Estimate: 0h Time Spent: 10m > Add hive db schema changes introduced in HIVE-21884 to the schema upgrade > scripts > - > > Key: HIVE-22727 > URL: https://issues.apache.org/jira/browse/HIVE-22727 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Chovan >Assignee: Zoltan Chovan >Priority: Minor > Labels: pull-request-available > Attachments: HIVE-22727.patch > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22400) UDF minute with time returns NULL
[
https://issues.apache.org/jira/browse/HIVE-22400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015233#comment-17015233
]
Jesus Camacho Rodriguez commented on HIVE-22400:
+1
> UDF minute with time returns NULL
> -
>
> Key: HIVE-22400
> URL: https://issues.apache.org/jira/browse/HIVE-22400
> Project: Hive
> Issue Type: Bug
> Components: UDF
>Affects Versions: 3.1.0
>Reporter: Nikunj
>Assignee: Naresh P R
>Priority: Minor
> Attachments: HIVE-22400.1.patch, HIVE-22400.2.patch,
> HIVE-22400.3.patch, HIVE-22400.patch
>
>
> [impadmin@impetus-g031 ~]$ beeline
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in
> [jar:file:/usr/hdp/3.1.0.0-78/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/usr/hdp/3.1.0.0-78/hadoop/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
> Connecting to
> jdbc:hive2://impetus-dsrv11.impetus.co.in:2181,ct-n0066.impetus.co.in:2181,ct-n0092.impetus.co.in:2181/default;principal=hive/_h...@impetus.co.in;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2
> 19/10/24 19:03:42 [main]: INFO jdbc.HiveConnection: Connected to
> ct-n0092.impetus.co.in:1
> Connected to: Apache Hive (version 3.1.0.3.1.0.0-78)
> Driver: Hive JDBC (version 3.1.0.3.1.0.0-78)
> Transaction isolation: TRANSACTION_REPEATABLE_READ
> Beeline version 3.1.0.3.1.0.0-78 by Apache Hive
> 0: jdbc:hive2://impetus-dsrv11.impetus.co.in:> select minute('12:58:59');
> INFO : Compiling
> command(queryId=hive_20191024190401_bc517191-bd20-4f5a-b5f5-44f762c2d395):
> select minute('12:58:59')
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:_c0,
> type:int, comment:null)], properties:null)
> INFO : Completed compiling
> command(queryId=hive_20191024190401_bc517191-bd20-4f5a-b5f5-44f762c2d395);
> Time taken: 0.427 seconds
> INFO : Executing
> command(queryId=hive_20191024190401_bc517191-bd20-4f5a-b5f5-44f762c2d395):
> select minute('12:58:59')
> INFO : Completed executing
> command(queryId=hive_20191024190401_bc517191-bd20-4f5a-b5f5-44f762c2d395);
> Time taken: 0.003 seconds
> INFO : OK
> +---+
> | _c0 |
> +---+
> | NULL |
> +---+
> 1 row selected (0.739 seconds)
> 0: jdbc:hive2://impetus-dsrv11.impetus.co.in:>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22727) Add hive db schema changes introduced in HIVE-21884 to the schema upgrade scripts
[ https://issues.apache.org/jira/browse/HIVE-22727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Chovan updated HIVE-22727: - Attachment: HIVE-22727.patch > Add hive db schema changes introduced in HIVE-21884 to the schema upgrade > scripts > - > > Key: HIVE-22727 > URL: https://issues.apache.org/jira/browse/HIVE-22727 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Chovan >Assignee: Zoltan Chovan >Priority: Minor > Attachments: HIVE-22727.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22486) Send only accessed columns for masking policies request
[ https://issues.apache.org/jira/browse/HIVE-22486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-22486: --- Attachment: HIVE-22486.91.patch > Send only accessed columns for masking policies request > --- > > Key: HIVE-22486 > URL: https://issues.apache.org/jira/browse/HIVE-22486 > Project: Hive > Issue Type: Improvement > Components: CBO >Affects Versions: 4.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-22486.01.patch, HIVE-22486.02.patch, > HIVE-22486.03.patch, HIVE-22486.05.patch, HIVE-22486.06.patch, > HIVE-22486.90.patch, HIVE-22486.90.patch, HIVE-22486.91.patch, > HIVE-22486.91.patch, HIVE-22486.91.patch, HIVE-22486.patch, > HIVE-22486.patchaddendum > > Time Spent: 1h > Remaining Estimate: 0h > > Currently, we send all columns for masking request, even if they are not > accessed by the given query. We could send only those columns for which the > masking policy will be necessary. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-22727) Add hive db schema changes introduced in HIVE-21884 to the schema upgrade scripts
[ https://issues.apache.org/jira/browse/HIVE-22727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Chovan reassigned HIVE-22727: > Add hive db schema changes introduced in HIVE-21884 to the schema upgrade > scripts > - > > Key: HIVE-22727 > URL: https://issues.apache.org/jira/browse/HIVE-22727 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Chovan >Assignee: Zoltan Chovan >Priority: Minor > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22685) TestHiveSqlDateTimeFormatter Now Broken with New Year 2020
[
https://issues.apache.org/jira/browse/HIVE-22685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015230#comment-17015230
]
David Mollitor commented on HIVE-22685:
---
[~klcopp] Thanks. Love the discussion.
I do really believe that hard-coded date is the way to go.
{quote}
testing with dates around 1970 – they aren't as likely to be actually used.
{quote}
If you would like to suggest other dates that you feel are more likely to be
used, let me know and we can update. I think that's kind of the point though,
that there is no such thing as a good date or bad date. However, if for
example there is some sort of edge case, the edge-case should get its own test.
One cannot introduce this edge case test if the date changes with every test
run. For example, there is some comment about {{Thursday of the first week in
an ISO year always matches the Gregorian year.}}. If this is something that
needs testing, I would ask that you put that test back in, after commit, with
the specific date that might cause an issue.
I also think that it's important that the expected outcome doesn't change. For
example:
{code:none}
-checkParseTimestampIso("IY-IW-ID", "0-01-04", "iw, ", "01, " +
thisYearString.substring(0, 3) + "0");
-checkParseTimestampIso("I-IW-ID", "0-01-04", "iw, ", "01, " +
thisYearString.substring(0, 3) + "0");
+checkParseTimestampIso("IY-IW-ID", "0-01-04", "iw, ", "01, 1970");
+checkParseTimestampIso("I-IW-ID", "0-01-04", "iw, ", "01, 1970");
//time patterns are allowed; date patterns are not
{code}
Is there a unit test now testing this unit test that
{{thisYearString.substring(0, 3)}} is correct? The cost of introducing this
extra capability to make testing possible is well worth it.
> TestHiveSqlDateTimeFormatter Now Broken with New Year 2020
> --
>
> Key: HIVE-22685
> URL: https://issues.apache.org/jira/browse/HIVE-22685
> Project: Hive
> Issue Type: Bug
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-22685.1.patch, HIVE-22685.2.patch,
> HIVE-22685.3.patch
>
>
> Unit test is now broken (n)(n):(
> {code:java}
> //Tests for these patterns would need changing every decade if done in
> the above way.
> //Thursday of the first week in an ISO year always matches the Gregorian
> year.
> checkParseTimestampIso("IY-IW-ID", "0-01-04", "iw, ", "01, " +
> thisYearString.substring(0, 3) + "0");
> checkParseTimestampIso("I-IW-ID", "0-01-04", "iw, ", "01, " +
> thisYearString.substring(0, 3) + "0");
> {code}
> {code}
> org.junit.ComparisonFailure: expected:<01, 20[1]0> but was:<01, 20[2]0>
> at org.junit.Assert.assertEquals(Assert.java:115)
> at org.junit.Assert.assertEquals(Assert.java:144)
> at
> org.apache.hadoop.hive.common.format.datetime.TestHiveSqlDateTimeFormatter.checkParseTimestampIso(TestHiveSqlDateTimeFormatter.java:313)
> at
> org.apache.hadoop.hive.common.format.datetime.TestHiveSqlDateTimeFormatter.testParseTimestamp(TestHiveSqlDateTimeFormatter.java:287)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22663) Quote all table and column names or do not quote any
[
https://issues.apache.org/jira/browse/HIVE-22663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015228#comment-17015228
]
Hive QA commented on HIVE-22663:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
47s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
20s{color} | {color:blue} standalone-metastore/metastore-server in master has
181 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
0s{color} | {color:blue} ql in master has 1531 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
22s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
22s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 98 new + 486 unchanged - 79 fixed = 584 total (was 565) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
23s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 32m 4s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20170/dev-support/hive-personality.sh
|
| git revision | master / 1f541b5 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20170/yetus/diff-checkstyle-standalone-metastore_metastore-server.txt
|
| modules | C: standalone-metastore/metastore-server ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20170/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Quote all table and column names or do not quote any
>
>
> Key: HIVE-22663
> URL: https://issues.apache.org/jira/browse/HIVE-22663
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2, Standalone Metastore
>Affects Versions: 4.0.0
>Reporter: Ashutosh Bapat
>Assignee: Zoltan Chovan
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22663.2.patch, HIVE-22663.3.patch, HIVE-22663.patch
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> The change in HIVE-22546 is causing following stack trace when I run Hive
> with PostgreSQL as backend db for the metastore.
> 0: jdbc:hive2://localhost:1> create database dumpdb with
> ('repl.source.for'='1,2,3');0: jdbc:hive2://localhost:1> create database
> dumpdb with ('repl.source.for'='1,2,3');Error: Error
[jira] [Updated] (HIVE-22518) SQLStdHiveAuthorizerFactoryForTest doesn't work correctly for llap tests
[ https://issues.apache.org/jira/browse/HIVE-22518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-22518: Attachment: HIVE-22518.02.patch > SQLStdHiveAuthorizerFactoryForTest doesn't work correctly for llap tests > > > Key: HIVE-22518 > URL: https://issues.apache.org/jira/browse/HIVE-22518 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-22518.01.patch, HIVE-22518.01.patch, > HIVE-22518.02.patch, HIVE-22518.02.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22627) Add schema changes introduced in HIVE-21443 to the schema upgrade scripts
[ https://issues.apache.org/jira/browse/HIVE-22627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-22627: -- Labels: pull-request-available (was: ) > Add schema changes introduced in HIVE-21443 to the schema upgrade scripts > - > > Key: HIVE-22627 > URL: https://issues.apache.org/jira/browse/HIVE-22627 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Chovan >Assignee: Zoltan Chovan >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22627.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-22627) Add schema changes introduced in HIVE-21443 to the schema upgrade scripts
[ https://issues.apache.org/jira/browse/HIVE-22627?focusedWorklogId=371711=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-371711 ] ASF GitHub Bot logged work on HIVE-22627: - Author: ASF GitHub Bot Created on: 14/Jan/20 17:06 Start Date: 14/Jan/20 17:06 Worklog Time Spent: 10m Work Description: zchovan commented on pull request #878: HIVE-22627: Add schema changes introduced in HIVE-21443 to the schema… URL: https://github.com/apache/hive/pull/878 … upgrade scripts Change-Id: Ie395caccfcd78f1d58a37c104f93c4cc8f7e1b5f This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 371711) Remaining Estimate: 0h Time Spent: 10m > Add schema changes introduced in HIVE-21443 to the schema upgrade scripts > - > > Key: HIVE-22627 > URL: https://issues.apache.org/jira/browse/HIVE-22627 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Chovan >Assignee: Zoltan Chovan >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22627.patch > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22627) Add schema changes introduced in HIVE-21443 to the schema upgrade scripts
[ https://issues.apache.org/jira/browse/HIVE-22627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Chovan updated HIVE-22627: - Attachment: HIVE-22627.patch > Add schema changes introduced in HIVE-21443 to the schema upgrade scripts > - > > Key: HIVE-22627 > URL: https://issues.apache.org/jira/browse/HIVE-22627 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Chovan >Assignee: Zoltan Chovan >Priority: Major > Attachments: HIVE-22627.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22538) RS deduplication does not always enforce hive.optimize.reducededuplication.min.reducer
[
https://issues.apache.org/jira/browse/HIVE-22538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015208#comment-17015208
]
Hive QA commented on HIVE-22538:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12990872/HIVE-22538.3.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 15 failed/errored test(s), 17877 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_nullscan]
(batchId=76)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_table_directories_test]
(batchId=40)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_4]
(batchId=14)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[clusterctas] (batchId=55)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[check_constraint]
(batchId=168)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dynamic_semijoin_reduction_3]
(batchId=186)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[enforce_constraint_notnull]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_into_default_keyword]
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[materialized_view_create_rewrite_4]
(batchId=168)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[materialized_view_rewrite_window]
(batchId=186)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[semijoin_hint]
(batchId=172)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[semijoin_reddedup]
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sqlmerge]
(batchId=188)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sqlmerge_stats]
(batchId=187)
org.apache.hadoop.hive.ql.TestAcidOnTez.testBucketedAcidInsertWithRemoveUnion
(batchId=250)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20169/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20169/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20169/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 15 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12990872 - PreCommit-HIVE-Build
> RS deduplication does not always enforce
> hive.optimize.reducededuplication.min.reducer
> --
>
> Key: HIVE-22538
> URL: https://issues.apache.org/jira/browse/HIVE-22538
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Reporter: Jesus Camacho Rodriguez
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22538.2.patch, HIVE-22538.3.patch, HIVE-22538.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> For transactional tables, that property might be overriden to 1, which can
> lead to merging final aggregation into a single stage (hence leading to
> performance degradation). For instance, when autogather column stats is
> enabled, this can happen for the following query:
> {code}
> set hive.support.concurrency=true;
> set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
> EXPLAIN
> CREATE TABLE x STORED AS ORC TBLPROPERTIES('transactional'='true') AS
> SELECT * FROM SRC x CLUSTER BY x.key;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22685) TestHiveSqlDateTimeFormatter Now Broken with New Year 2020
[
https://issues.apache.org/jira/browse/HIVE-22685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015198#comment-17015198
]
Karen Coppage commented on HIVE-22685:
--
Hi David, ok, using the guava Optional works. But I still think that
changing/exposing the class in the interest of testing is a bad practice. I
also worry about the usefulness of testing with dates around 1970 – they aren't
as likely to be actually used. If we test with constantly changing, relevant
dates then there's a higher chance of catching any issues with the source code.
For clarity, we can include details about the expected value (and the current
date) as the "message" parameter of the assert.
That's my opinion. If you really really really care about using hard-coding
dates for testing then I'll give a +1, but I think it's more elegant to let
source code be source code and not test code.
> TestHiveSqlDateTimeFormatter Now Broken with New Year 2020
> --
>
> Key: HIVE-22685
> URL: https://issues.apache.org/jira/browse/HIVE-22685
> Project: Hive
> Issue Type: Bug
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-22685.1.patch, HIVE-22685.2.patch,
> HIVE-22685.3.patch
>
>
> Unit test is now broken (n)(n):(
> {code:java}
> //Tests for these patterns would need changing every decade if done in
> the above way.
> //Thursday of the first week in an ISO year always matches the Gregorian
> year.
> checkParseTimestampIso("IY-IW-ID", "0-01-04", "iw, ", "01, " +
> thisYearString.substring(0, 3) + "0");
> checkParseTimestampIso("I-IW-ID", "0-01-04", "iw, ", "01, " +
> thisYearString.substring(0, 3) + "0");
> {code}
> {code}
> org.junit.ComparisonFailure: expected:<01, 20[1]0> but was:<01, 20[2]0>
> at org.junit.Assert.assertEquals(Assert.java:115)
> at org.junit.Assert.assertEquals(Assert.java:144)
> at
> org.apache.hadoop.hive.common.format.datetime.TestHiveSqlDateTimeFormatter.checkParseTimestampIso(TestHiveSqlDateTimeFormatter.java:313)
> at
> org.apache.hadoop.hive.common.format.datetime.TestHiveSqlDateTimeFormatter.testParseTimestamp(TestHiveSqlDateTimeFormatter.java:287)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22538) RS deduplication does not always enforce hive.optimize.reducededuplication.min.reducer
[
https://issues.apache.org/jira/browse/HIVE-22538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015173#comment-17015173
]
Hive QA commented on HIVE-22538:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
46s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
1s{color} | {color:blue} ql in master has 1531 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
45s{color} | {color:red} ql: The patch generated 6 new + 463 unchanged - 3
fixed = 469 total (was 466) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 7s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20169/dev-support/hive-personality.sh
|
| git revision | master / 1f541b5 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20169/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20169/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> RS deduplication does not always enforce
> hive.optimize.reducededuplication.min.reducer
> --
>
> Key: HIVE-22538
> URL: https://issues.apache.org/jira/browse/HIVE-22538
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Reporter: Jesus Camacho Rodriguez
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22538.2.patch, HIVE-22538.3.patch, HIVE-22538.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> For transactional tables, that property might be overriden to 1, which can
> lead to merging final aggregation into a single stage (hence leading to
> performance degradation). For instance, when autogather column stats is
> enabled, this can happen for the following query:
> {code}
> set hive.support.concurrency=true;
> set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
> EXPLAIN
> CREATE TABLE x STORED AS ORC TBLPROPERTIES('transactional'='true') AS
> SELECT * FROM SRC x CLUSTER BY x.key;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22726) TopN Key optimizer should use array instead of priority queue
[ https://issues.apache.org/jira/browse/HIVE-22726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Magyar updated HIVE-22726: - Status: Patch Available (was: Open) > TopN Key optimizer should use array instead of priority queue > - > > Key: HIVE-22726 > URL: https://issues.apache.org/jira/browse/HIVE-22726 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Attila Magyar >Assignee: Attila Magyar >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22726.1.patch > > > The TopN key optimizer currently uses a priority queue for keeping track of > the largest/smallest rows. Its max size is the same as the user specified > limit. This should be replaced a more cache line friendly array with a small > (128) maximum size and see how much performance is gained. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22726) TopN Key optimizer should use array instead of priority queue
[ https://issues.apache.org/jira/browse/HIVE-22726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Magyar updated HIVE-22726: - Attachment: HIVE-22726.1.patch > TopN Key optimizer should use array instead of priority queue > - > > Key: HIVE-22726 > URL: https://issues.apache.org/jira/browse/HIVE-22726 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Attila Magyar >Assignee: Attila Magyar >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22726.1.patch > > > The TopN key optimizer currently uses a priority queue for keeping track of > the largest/smallest rows. Its max size is the same as the user specified > limit. This should be replaced a more cache line friendly array with a small > (128) maximum size and see how much performance is gained. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22518) SQLStdHiveAuthorizerFactoryForTest doesn't work correctly for llap tests
[ https://issues.apache.org/jira/browse/HIVE-22518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-22518: Attachment: HIVE-22518.02.patch > SQLStdHiveAuthorizerFactoryForTest doesn't work correctly for llap tests > > > Key: HIVE-22518 > URL: https://issues.apache.org/jira/browse/HIVE-22518 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-22518.01.patch, HIVE-22518.01.patch, > HIVE-22518.02.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22518) SQLStdHiveAuthorizerFactoryForTest doesn't work correctly for llap tests
[
https://issues.apache.org/jira/browse/HIVE-22518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015147#comment-17015147
]
Hive QA commented on HIVE-22518:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12990876/HIVE-22518.01.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20168/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20168/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20168/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2020-01-14 15:06:17.732
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-20168/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2020-01-14 15:06:17.735
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 1f541b5 HIVE-22631: Avoid deep copying partition list in
listPartitionsByExpr (Vineet Garg, reviewed by Jesus Camacho Rodriguez)
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 1f541b5 HIVE-22631: Avoid deep copying partition list in
listPartitionsByExpr (Vineet Garg, reviewed by Jesus Camacho Rodriguez)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2020-01-14 15:06:19.630
+ rm -rf ../yetus_PreCommit-HIVE-Build-20168
+ mkdir ../yetus_PreCommit-HIVE-Build-20168
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-20168
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-20168/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
Trying to apply the patch with -p0
Going to apply patch with: git apply -p0
+ [[ maven == \m\a\v\e\n ]]
+ rm -rf /data/hiveptest/working/maven/org/apache/hive
+ mvn -B clean install -DskipTests -T 4 -q
-Dmaven.repo.local=/data/hiveptest/working/maven
protoc-jar: executing: [/tmp/protoc3067934565328029017.exe, --version]
libprotoc 2.5.0
protoc-jar: executing: [/tmp/protoc3067934565328029017.exe,
-I/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/src/main/protobuf/org/apache/hadoop/hive/metastore,
--java_out=/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/target/generated-sources,
/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/src/main/protobuf/org/apache/hadoop/hive/metastore/metastore.proto]
ANTLR Parser Generator Version 3.5.2
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-remote-resources-plugin:1.5:process
(process-resource-bundles) on project hive-shims: Execution
process-resource-bundles of goal
org.apache.maven.plugins:maven-remote-resources-plugin:1.5:process failed.
ConcurrentModificationException -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please
read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn -rf :hive-shims
+ result=1
+ '[' 1 -ne 0 ']'
+ rm -rf yetus_PreCommit-HIVE-Build-20168
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12990876 - PreCommit-HIVE-Build
> SQLStdHiveAuthorizerFactoryForTest
[jira] [Commented] (HIVE-22400) UDF minute with time returns NULL
[
https://issues.apache.org/jira/browse/HIVE-22400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015145#comment-17015145
]
Hive QA commented on HIVE-22400:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12990736/HIVE-22400.3.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17876 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20167/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20167/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20167/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12990736 - PreCommit-HIVE-Build
> UDF minute with time returns NULL
> -
>
> Key: HIVE-22400
> URL: https://issues.apache.org/jira/browse/HIVE-22400
> Project: Hive
> Issue Type: Bug
> Components: UDF
>Affects Versions: 3.1.0
>Reporter: Nikunj
>Assignee: Naresh P R
>Priority: Minor
> Attachments: HIVE-22400.1.patch, HIVE-22400.2.patch,
> HIVE-22400.3.patch, HIVE-22400.patch
>
>
> [impadmin@impetus-g031 ~]$ beeline
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in
> [jar:file:/usr/hdp/3.1.0.0-78/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/usr/hdp/3.1.0.0-78/hadoop/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
> Connecting to
> jdbc:hive2://impetus-dsrv11.impetus.co.in:2181,ct-n0066.impetus.co.in:2181,ct-n0092.impetus.co.in:2181/default;principal=hive/_h...@impetus.co.in;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2
> 19/10/24 19:03:42 [main]: INFO jdbc.HiveConnection: Connected to
> ct-n0092.impetus.co.in:1
> Connected to: Apache Hive (version 3.1.0.3.1.0.0-78)
> Driver: Hive JDBC (version 3.1.0.3.1.0.0-78)
> Transaction isolation: TRANSACTION_REPEATABLE_READ
> Beeline version 3.1.0.3.1.0.0-78 by Apache Hive
> 0: jdbc:hive2://impetus-dsrv11.impetus.co.in:> select minute('12:58:59');
> INFO : Compiling
> command(queryId=hive_20191024190401_bc517191-bd20-4f5a-b5f5-44f762c2d395):
> select minute('12:58:59')
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:_c0,
> type:int, comment:null)], properties:null)
> INFO : Completed compiling
> command(queryId=hive_20191024190401_bc517191-bd20-4f5a-b5f5-44f762c2d395);
> Time taken: 0.427 seconds
> INFO : Executing
> command(queryId=hive_20191024190401_bc517191-bd20-4f5a-b5f5-44f762c2d395):
> select minute('12:58:59')
> INFO : Completed executing
> command(queryId=hive_20191024190401_bc517191-bd20-4f5a-b5f5-44f762c2d395);
> Time taken: 0.003 seconds
> INFO : OK
> +---+
> | _c0 |
> +---+
> | NULL |
> +---+
> 1 row selected (0.739 seconds)
> 0: jdbc:hive2://impetus-dsrv11.impetus.co.in:>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22712) ReExec Driver execute submit the query in default queue irrespective of user defined queue
[
https://issues.apache.org/jira/browse/HIVE-22712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015144#comment-17015144
]
Zoltan Haindrich commented on HIVE-22712:
-
I never expected an [unset like
this|https://github.com/apache/hive/blob/1f541b52314519e206dcb0f1aa2ebceb8e44d7a6/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionState.java#L473-L481]
fixing the immutabiliy of the HiveConf is longer term goal - and it's very
hard to achieve...
this quick fix will hold until we get to that...
+1
to make sure that we don't lose this fix: could you please add the following
test to 'retry_failure.q'?
{code}
-- HIVE-22712
set hive.fetch.task.conversion=none;
set tez.queue.name=default;
select assert_true(${hiveconf:zzz} > a),
assert_true("${hiveconf:tez.queue.name}" = 'default') from tx_n1;
{code}
> ReExec Driver execute submit the query in default queue irrespective of user
> defined queue
> --
>
> Key: HIVE-22712
> URL: https://issues.apache.org/jira/browse/HIVE-22712
> Project: Hive
> Issue Type: Bug
> Components: Hive, HiveServer2
>Affects Versions: 3.1.2
> Environment: Hive-3
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-22712.01.patch, HIVE-22712.02.patch,
> HIVE-22712.patch
>
>
> we unset the queue name intentionally in
> TezSessionState#startSessionAndContainers,
> as a result reexec create a new session in the default queue and create a
> problem, its a cumbersome to add reexec.overlay.tez.queue.name at session
> level.
> I could not find a better way of setting the queue name (I am open for the
> suggestion here) since it can create a conflict with the Global queue name
> vs user-defined queue that's why setting while initialization of
> ReExecutionOverlayPlugin.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22685) TestHiveSqlDateTimeFormatter Now Broken with New Year 2020
[
https://issues.apache.org/jira/browse/HIVE-22685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015112#comment-17015112
]
David Mollitor commented on HIVE-22685:
---
Added a new patch with proposed changes.
> TestHiveSqlDateTimeFormatter Now Broken with New Year 2020
> --
>
> Key: HIVE-22685
> URL: https://issues.apache.org/jira/browse/HIVE-22685
> Project: Hive
> Issue Type: Bug
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-22685.1.patch, HIVE-22685.2.patch,
> HIVE-22685.3.patch
>
>
> Unit test is now broken (n)(n):(
> {code:java}
> //Tests for these patterns would need changing every decade if done in
> the above way.
> //Thursday of the first week in an ISO year always matches the Gregorian
> year.
> checkParseTimestampIso("IY-IW-ID", "0-01-04", "iw, ", "01, " +
> thisYearString.substring(0, 3) + "0");
> checkParseTimestampIso("I-IW-ID", "0-01-04", "iw, ", "01, " +
> thisYearString.substring(0, 3) + "0");
> {code}
> {code}
> org.junit.ComparisonFailure: expected:<01, 20[1]0> but was:<01, 20[2]0>
> at org.junit.Assert.assertEquals(Assert.java:115)
> at org.junit.Assert.assertEquals(Assert.java:144)
> at
> org.apache.hadoop.hive.common.format.datetime.TestHiveSqlDateTimeFormatter.checkParseTimestampIso(TestHiveSqlDateTimeFormatter.java:313)
> at
> org.apache.hadoop.hive.common.format.datetime.TestHiveSqlDateTimeFormatter.testParseTimestamp(TestHiveSqlDateTimeFormatter.java:287)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22685) TestHiveSqlDateTimeFormatter Now Broken with New Year 2020
[
https://issues.apache.org/jira/browse/HIVE-22685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-22685:
--
Attachment: HIVE-22685.3.patch
> TestHiveSqlDateTimeFormatter Now Broken with New Year 2020
> --
>
> Key: HIVE-22685
> URL: https://issues.apache.org/jira/browse/HIVE-22685
> Project: Hive
> Issue Type: Bug
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-22685.1.patch, HIVE-22685.2.patch,
> HIVE-22685.3.patch
>
>
> Unit test is now broken (n)(n):(
> {code:java}
> //Tests for these patterns would need changing every decade if done in
> the above way.
> //Thursday of the first week in an ISO year always matches the Gregorian
> year.
> checkParseTimestampIso("IY-IW-ID", "0-01-04", "iw, ", "01, " +
> thisYearString.substring(0, 3) + "0");
> checkParseTimestampIso("I-IW-ID", "0-01-04", "iw, ", "01, " +
> thisYearString.substring(0, 3) + "0");
> {code}
> {code}
> org.junit.ComparisonFailure: expected:<01, 20[1]0> but was:<01, 20[2]0>
> at org.junit.Assert.assertEquals(Assert.java:115)
> at org.junit.Assert.assertEquals(Assert.java:144)
> at
> org.apache.hadoop.hive.common.format.datetime.TestHiveSqlDateTimeFormatter.checkParseTimestampIso(TestHiveSqlDateTimeFormatter.java:313)
> at
> org.apache.hadoop.hive.common.format.datetime.TestHiveSqlDateTimeFormatter.testParseTimestamp(TestHiveSqlDateTimeFormatter.java:287)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22400) UDF minute with time returns NULL
[
https://issues.apache.org/jira/browse/HIVE-22400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015111#comment-17015111
]
Hive QA commented on HIVE-22400:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
48s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
33s{color} | {color:blue} common in master has 63 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
3s{color} | {color:blue} ql in master has 1531 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
13s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
15s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 29m 56s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20167/dev-support/hive-personality.sh
|
| git revision | master / 1f541b5 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20167/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> UDF minute with time returns NULL
> -
>
> Key: HIVE-22400
> URL: https://issues.apache.org/jira/browse/HIVE-22400
> Project: Hive
> Issue Type: Bug
> Components: UDF
>Affects Versions: 3.1.0
>Reporter: Nikunj
>Assignee: Naresh P R
>Priority: Minor
> Attachments: HIVE-22400.1.patch, HIVE-22400.2.patch,
> HIVE-22400.3.patch, HIVE-22400.patch
>
>
> [impadmin@impetus-g031 ~]$ beeline
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in
> [jar:file:/usr/hdp/3.1.0.0-78/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/usr/hdp/3.1.0.0-78/hadoop/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
> Connecting to
> jdbc:hive2://impetus-dsrv11.impetus.co.in:2181,ct-n0066.impetus.co.in:2181,ct-n0092.impetus.co.in:2181/default;principal=hive/_h...@impetus.co.in;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2
> 19/10/24
[jira] [Updated] (HIVE-22518) SQLStdHiveAuthorizerFactoryForTest doesn't work correctly for llap tests
[ https://issues.apache.org/jira/browse/HIVE-22518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-22518: Attachment: HIVE-22518.01.patch > SQLStdHiveAuthorizerFactoryForTest doesn't work correctly for llap tests > > > Key: HIVE-22518 > URL: https://issues.apache.org/jira/browse/HIVE-22518 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-22518.01.patch, HIVE-22518.01.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22706) Jdbc storage handler incorrectly interprets boolean column value in derby
[ https://issues.apache.org/jira/browse/HIVE-22706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015109#comment-17015109 ] Zoltan Haindrich commented on HIVE-22706: - [~srahman] well...it's not nice to have this fixed here in hive's jdbc-handler...but yes; it makes sense to also cover the write in case it's needed at some point I've found traces that at some point derby wrote 0/1 into CHAR types and not true/false https://issues.apache.org/jira/browse/DERBY-4965 note: there is some very flexible error handling in the [DBOutputFormat|https://github.com/apache/hadoop/blob/1c51f36be79924489ca4a2e5ca7e96ac75a6ec18/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/db/DBOutputFormat.java#L125] ...it may make resultsets disappear in some cases - when there is a jdbc issue I fixed the write path - and extended the test to cover this as well. > Jdbc storage handler incorrectly interprets boolean column value in derby > - > > Key: HIVE-22706 > URL: https://issues.apache.org/jira/browse/HIVE-22706 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-22706.01.patch, HIVE-22706.01.patch, > HIVE-22706.01.patch, HIVE-22706.01.patch, HIVE-22706.02.patch > > > in case the column value is false ; the storage handler interprets it as true -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22706) Jdbc storage handler incorrectly interprets boolean column value in derby
[ https://issues.apache.org/jira/browse/HIVE-22706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-22706: Attachment: HIVE-22706.02.patch > Jdbc storage handler incorrectly interprets boolean column value in derby > - > > Key: HIVE-22706 > URL: https://issues.apache.org/jira/browse/HIVE-22706 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-22706.01.patch, HIVE-22706.01.patch, > HIVE-22706.01.patch, HIVE-22706.01.patch, HIVE-22706.02.patch > > > in case the column value is false ; the storage handler interprets it as true -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22538) RS deduplication does not always enforce hive.optimize.reducededuplication.min.reducer
[
https://issues.apache.org/jira/browse/HIVE-22538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22538:
--
Status: Patch Available (was: Open)
> RS deduplication does not always enforce
> hive.optimize.reducededuplication.min.reducer
> --
>
> Key: HIVE-22538
> URL: https://issues.apache.org/jira/browse/HIVE-22538
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Reporter: Jesus Camacho Rodriguez
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22538.2.patch, HIVE-22538.3.patch, HIVE-22538.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> For transactional tables, that property might be overriden to 1, which can
> lead to merging final aggregation into a single stage (hence leading to
> performance degradation). For instance, when autogather column stats is
> enabled, this can happen for the following query:
> {code}
> set hive.support.concurrency=true;
> set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
> EXPLAIN
> CREATE TABLE x STORED AS ORC TBLPROPERTIES('transactional'='true') AS
> SELECT * FROM SRC x CLUSTER BY x.key;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22538) RS deduplication does not always enforce hive.optimize.reducededuplication.min.reducer
[
https://issues.apache.org/jira/browse/HIVE-22538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22538:
--
Attachment: HIVE-22538.3.patch
> RS deduplication does not always enforce
> hive.optimize.reducededuplication.min.reducer
> --
>
> Key: HIVE-22538
> URL: https://issues.apache.org/jira/browse/HIVE-22538
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Reporter: Jesus Camacho Rodriguez
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22538.2.patch, HIVE-22538.3.patch, HIVE-22538.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> For transactional tables, that property might be overriden to 1, which can
> lead to merging final aggregation into a single stage (hence leading to
> performance degradation). For instance, when autogather column stats is
> enabled, this can happen for the following query:
> {code}
> set hive.support.concurrency=true;
> set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
> EXPLAIN
> CREATE TABLE x STORED AS ORC TBLPROPERTIES('transactional'='true') AS
> SELECT * FROM SRC x CLUSTER BY x.key;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22538) RS deduplication does not always enforce hive.optimize.reducededuplication.min.reducer
[
https://issues.apache.org/jira/browse/HIVE-22538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22538:
--
Attachment: HIVE-22538.2.patch
> RS deduplication does not always enforce
> hive.optimize.reducededuplication.min.reducer
> --
>
> Key: HIVE-22538
> URL: https://issues.apache.org/jira/browse/HIVE-22538
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Reporter: Jesus Camacho Rodriguez
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22538.2.patch, HIVE-22538.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> For transactional tables, that property might be overriden to 1, which can
> lead to merging final aggregation into a single stage (hence leading to
> performance degradation). For instance, when autogather column stats is
> enabled, this can happen for the following query:
> {code}
> set hive.support.concurrency=true;
> set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
> EXPLAIN
> CREATE TABLE x STORED AS ORC TBLPROPERTIES('transactional'='true') AS
> SELECT * FROM SRC x CLUSTER BY x.key;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22538) RS deduplication does not always enforce hive.optimize.reducededuplication.min.reducer
[
https://issues.apache.org/jira/browse/HIVE-22538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22538:
--
Attachment: (was: HIVE-22538.2.patch)
> RS deduplication does not always enforce
> hive.optimize.reducededuplication.min.reducer
> --
>
> Key: HIVE-22538
> URL: https://issues.apache.org/jira/browse/HIVE-22538
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Reporter: Jesus Camacho Rodriguez
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22538.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> For transactional tables, that property might be overriden to 1, which can
> lead to merging final aggregation into a single stage (hence leading to
> performance degradation). For instance, when autogather column stats is
> enabled, this can happen for the following query:
> {code}
> set hive.support.concurrency=true;
> set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
> EXPLAIN
> CREATE TABLE x STORED AS ORC TBLPROPERTIES('transactional'='true') AS
> SELECT * FROM SRC x CLUSTER BY x.key;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22538) RS deduplication does not always enforce hive.optimize.reducededuplication.min.reducer
[
https://issues.apache.org/jira/browse/HIVE-22538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22538:
--
Status: Open (was: Patch Available)
> RS deduplication does not always enforce
> hive.optimize.reducededuplication.min.reducer
> --
>
> Key: HIVE-22538
> URL: https://issues.apache.org/jira/browse/HIVE-22538
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Reporter: Jesus Camacho Rodriguez
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22538.2.patch, HIVE-22538.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> For transactional tables, that property might be overriden to 1, which can
> lead to merging final aggregation into a single stage (hence leading to
> performance degradation). For instance, when autogather column stats is
> enabled, this can happen for the following query:
> {code}
> set hive.support.concurrency=true;
> set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
> EXPLAIN
> CREATE TABLE x STORED AS ORC TBLPROPERTIES('transactional'='true') AS
> SELECT * FROM SRC x CLUSTER BY x.key;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22685) TestHiveSqlDateTimeFormatter Now Broken with New Year 2020
[
https://issues.apache.org/jira/browse/HIVE-22685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015097#comment-17015097
]
David Mollitor commented on HIVE-22685:
---
[~klcopp] Thanks for the review.
Yes. I see now that it needs to be serialized. Thanks for bringing that to my
attention as I was surprised to see that this is a requirement. OK. Yes, Java
Optional is not Serializeable. While technically they were not intended to be
class variables, I personally think they make a lot of sense as instance
variables because of their idiot-proof-ness. If they are used as intended...
only in Getter/Setter then it adds a lot of boilerplate code that is subject to
fat-fingers. Most importantly, I tend to respect what Google does above all
else:
Their Optional implementation is serializeable so that it can be used as an
instance variable.
[https://guava.dev/releases/19.0/api/docs/com/google/common/base/Optional.html]
But yes, in this case, since it does need to be serializeable, we can use the
Guava optional (my preference) or use native null values.
I still very much encourage that we keep the ability to pass in a
{{LocalDateTime}}. Maybe we can lessen the scope to default package and add
the VisibleForTesting annotation. I think it's not ideal that the values in
the test change every time the test is ran. Sometimes its unavoidable, but in
this case passing in a known value is not too intrusive. If the test fails (as
in this case) the developer has no way to know what the value was at that exact
moment that caused it to fail. Passing in a known value, that does not change,
is ideal for debugging and testing.
> TestHiveSqlDateTimeFormatter Now Broken with New Year 2020
> --
>
> Key: HIVE-22685
> URL: https://issues.apache.org/jira/browse/HIVE-22685
> Project: Hive
> Issue Type: Bug
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-22685.1.patch, HIVE-22685.2.patch
>
>
> Unit test is now broken (n)(n):(
> {code:java}
> //Tests for these patterns would need changing every decade if done in
> the above way.
> //Thursday of the first week in an ISO year always matches the Gregorian
> year.
> checkParseTimestampIso("IY-IW-ID", "0-01-04", "iw, ", "01, " +
> thisYearString.substring(0, 3) + "0");
> checkParseTimestampIso("I-IW-ID", "0-01-04", "iw, ", "01, " +
> thisYearString.substring(0, 3) + "0");
> {code}
> {code}
> org.junit.ComparisonFailure: expected:<01, 20[1]0> but was:<01, 20[2]0>
> at org.junit.Assert.assertEquals(Assert.java:115)
> at org.junit.Assert.assertEquals(Assert.java:144)
> at
> org.apache.hadoop.hive.common.format.datetime.TestHiveSqlDateTimeFormatter.checkParseTimestampIso(TestHiveSqlDateTimeFormatter.java:313)
> at
> org.apache.hadoop.hive.common.format.datetime.TestHiveSqlDateTimeFormatter.testParseTimestamp(TestHiveSqlDateTimeFormatter.java:287)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22585) Clean up catalog/db/table name usage
[
https://issues.apache.org/jira/browse/HIVE-22585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015091#comment-17015091
]
Hive QA commented on HIVE-22585:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12990735/HIVE-22585.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 9 failed/errored test(s), 17876 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_rename_table]
(batchId=35)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input3] (batchId=94)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_3]
(batchId=115)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3]
(batchId=115)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[create_external_transactional]
(batchId=105)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[delete_non_acid_table]
(batchId=106)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[mm_delete]
(batchId=105)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[mm_update]
(batchId=104)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[update_non_acid_table]
(batchId=106)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20166/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20166/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20166/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 9 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12990735 - PreCommit-HIVE-Build
> Clean up catalog/db/table name usage
>
>
> Key: HIVE-22585
> URL: https://issues.apache.org/jira/browse/HIVE-22585
> Project: Hive
> Issue Type: Sub-task
>Reporter: David Lavati
>Assignee: David Lavati
>Priority: Major
> Labels: pull-request-available, refactor
> Attachments: HIVE-22585.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> This is a followup to HIVE-21198 to address some additional improvement ideas
> for the TableName object mentioned in
> [https://github.com/apache/hive/pull/550] and attempt to remove all the fishy
> usages of db/tablenames, as a number of places still rely on certain state
> changes/black magic.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22585) Clean up catalog/db/table name usage
[
https://issues.apache.org/jira/browse/HIVE-22585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015056#comment-17015056
]
Hive QA commented on HIVE-22585:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
3s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
13s{color} | {color:blue} ql in master has 1531 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
10s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
45s{color} | {color:red} ql: The patch generated 4 new + 664 unchanged - 0
fixed = 668 total (was 664) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 28m 50s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20166/dev-support/hive-personality.sh
|
| git revision | master / 1f541b5 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20166/yetus/diff-checkstyle-ql.txt
|
| modules | C: metastore ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20166/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Clean up catalog/db/table name usage
>
>
> Key: HIVE-22585
> URL: https://issues.apache.org/jira/browse/HIVE-22585
> Project: Hive
> Issue Type: Sub-task
>Reporter: David Lavati
>Assignee: David Lavati
>Priority: Major
> Labels: pull-request-available, refactor
> Attachments: HIVE-22585.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> This is a followup to HIVE-21198 to address some additional improvement ideas
> for the TableName object mentioned in
> [https://github.com/apache/hive/pull/550] and attempt to remove all the fishy
> usages of db/tablenames, as a number of places still rely on certain state
> changes/black magic.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-21569) Bump guava version to 28.1-jre
[ https://issues.apache.org/jira/browse/HIVE-21569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015018#comment-17015018 ] David Lavati commented on HIVE-21569: - Looks like this will be taken care of in spark 3.0, see SPARK-30272 > Bump guava version to 28.1-jre > -- > > Key: HIVE-21569 > URL: https://issues.apache.org/jira/browse/HIVE-21569 > Project: Hive > Issue Type: Task > Components: Hive >Reporter: Laszlo Pinter >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-21569.01.patch, HIVE-21569.02.patch, > HIVE-21569.03.patch, HIVE-21569.04.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Changes required to bump guava to 28.1-jre -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22366) Multiple metastore calls for same table and constraints during planning
[
https://issues.apache.org/jira/browse/HIVE-22366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-22366:
---
Status: Patch Available (was: Open)
> Multiple metastore calls for same table and constraints during planning
> ---
>
> Key: HIVE-22366
> URL: https://issues.apache.org/jira/browse/HIVE-22366
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-22366.1.patch, HIVE-22366.2.patch,
> HIVE-22366.3.patch, HIVE-22366.4.patch
>
>
> e.g. for a simple query like
> {code:sql}
> explain select count(i) from test2 group by j limit 3;
> {code}
> planner makes the following calls:
> {code}
> 2019-10-17T22:44:49,892 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr cmd=get_table :
> tbl=hive.default.test2
> 2019-10-17T22:44:49,908 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_primary_keys : tbl=hive.default.test2
> 2019-10-17T22:44:49,910 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_foreign_keys : parentdb=null parenttbl=null foreigndb=default
> foreigntbl=test2
> 2019-10-17T22:44:49,911 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_unique_constraints : tbl=hive.default.test2
> 2019-10-17T22:44:49,913 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_not_null_constraints : tbl=hive.default.test2
> 2019-10-17T22:44:49,979 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_partitions : tbl=hive.default.test2
> 2019-10-17T22:44:49,997 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_aggr_stats_for: table=hive.default.test2
> 2019-10-17T22:44:50,014 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr cmd=get_table :
> tbl=hive.default.test2
> 2019-10-17T22:44:50,023 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_primary_keys : tbl=hive.default.test2
> 2019-10-17T22:44:50,024 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_foreign_keys : parentdb=null parenttbl=null foreigndb=default
> foreigntbl=test2
> 2019-10-17T22:44:50,025 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_unique_constraints : tbl=hive.default.test2
> 2019-10-17T22:44:50,026 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_not_null_constraints : tbl=hive.default.test2
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22366) Multiple metastore calls for same table and constraints during planning
[
https://issues.apache.org/jira/browse/HIVE-22366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015006#comment-17015006
]
Vineet Garg commented on HIVE-22366:
This patch should prevent multiple unnecessary calls
* multiple calls to get_table
* multiple calls to get_table_statistics
> Multiple metastore calls for same table and constraints during planning
> ---
>
> Key: HIVE-22366
> URL: https://issues.apache.org/jira/browse/HIVE-22366
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-22366.1.patch, HIVE-22366.2.patch,
> HIVE-22366.3.patch, HIVE-22366.4.patch
>
>
> e.g. for a simple query like
> {code:sql}
> explain select count(i) from test2 group by j limit 3;
> {code}
> planner makes the following calls:
> {code}
> 2019-10-17T22:44:49,892 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr cmd=get_table :
> tbl=hive.default.test2
> 2019-10-17T22:44:49,908 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_primary_keys : tbl=hive.default.test2
> 2019-10-17T22:44:49,910 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_foreign_keys : parentdb=null parenttbl=null foreigndb=default
> foreigntbl=test2
> 2019-10-17T22:44:49,911 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_unique_constraints : tbl=hive.default.test2
> 2019-10-17T22:44:49,913 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_not_null_constraints : tbl=hive.default.test2
> 2019-10-17T22:44:49,979 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_partitions : tbl=hive.default.test2
> 2019-10-17T22:44:49,997 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_aggr_stats_for: table=hive.default.test2
> 2019-10-17T22:44:50,014 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr cmd=get_table :
> tbl=hive.default.test2
> 2019-10-17T22:44:50,023 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_primary_keys : tbl=hive.default.test2
> 2019-10-17T22:44:50,024 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_foreign_keys : parentdb=null parenttbl=null foreigndb=default
> foreigntbl=test2
> 2019-10-17T22:44:50,025 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_unique_constraints : tbl=hive.default.test2
> 2019-10-17T22:44:50,026 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_not_null_constraints : tbl=hive.default.test2
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22366) Multiple metastore calls for same table and constraints during planning
[
https://issues.apache.org/jira/browse/HIVE-22366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-22366:
---
Attachment: HIVE-22366.4.patch
> Multiple metastore calls for same table and constraints during planning
> ---
>
> Key: HIVE-22366
> URL: https://issues.apache.org/jira/browse/HIVE-22366
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-22366.1.patch, HIVE-22366.2.patch,
> HIVE-22366.3.patch, HIVE-22366.4.patch
>
>
> e.g. for a simple query like
> {code:sql}
> explain select count(i) from test2 group by j limit 3;
> {code}
> planner makes the following calls:
> {code}
> 2019-10-17T22:44:49,892 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr cmd=get_table :
> tbl=hive.default.test2
> 2019-10-17T22:44:49,908 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_primary_keys : tbl=hive.default.test2
> 2019-10-17T22:44:49,910 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_foreign_keys : parentdb=null parenttbl=null foreigndb=default
> foreigntbl=test2
> 2019-10-17T22:44:49,911 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_unique_constraints : tbl=hive.default.test2
> 2019-10-17T22:44:49,913 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_not_null_constraints : tbl=hive.default.test2
> 2019-10-17T22:44:49,979 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_partitions : tbl=hive.default.test2
> 2019-10-17T22:44:49,997 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_aggr_stats_for: table=hive.default.test2
> 2019-10-17T22:44:50,014 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr cmd=get_table :
> tbl=hive.default.test2
> 2019-10-17T22:44:50,023 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_primary_keys : tbl=hive.default.test2
> 2019-10-17T22:44:50,024 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_foreign_keys : parentdb=null parenttbl=null foreigndb=default
> foreigntbl=test2
> 2019-10-17T22:44:50,025 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_unique_constraints : tbl=hive.default.test2
> 2019-10-17T22:44:50,026 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_not_null_constraints : tbl=hive.default.test2
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22366) Multiple metastore calls for same table and constraints during planning
[
https://issues.apache.org/jira/browse/HIVE-22366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-22366:
---
Status: Open (was: Patch Available)
> Multiple metastore calls for same table and constraints during planning
> ---
>
> Key: HIVE-22366
> URL: https://issues.apache.org/jira/browse/HIVE-22366
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-22366.1.patch, HIVE-22366.2.patch,
> HIVE-22366.3.patch, HIVE-22366.4.patch
>
>
> e.g. for a simple query like
> {code:sql}
> explain select count(i) from test2 group by j limit 3;
> {code}
> planner makes the following calls:
> {code}
> 2019-10-17T22:44:49,892 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr cmd=get_table :
> tbl=hive.default.test2
> 2019-10-17T22:44:49,908 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_primary_keys : tbl=hive.default.test2
> 2019-10-17T22:44:49,910 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_foreign_keys : parentdb=null parenttbl=null foreigndb=default
> foreigntbl=test2
> 2019-10-17T22:44:49,911 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_unique_constraints : tbl=hive.default.test2
> 2019-10-17T22:44:49,913 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_not_null_constraints : tbl=hive.default.test2
> 2019-10-17T22:44:49,979 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_partitions : tbl=hive.default.test2
> 2019-10-17T22:44:49,997 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_aggr_stats_for: table=hive.default.test2
> 2019-10-17T22:44:50,014 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr cmd=get_table :
> tbl=hive.default.test2
> 2019-10-17T22:44:50,023 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_primary_keys : tbl=hive.default.test2
> 2019-10-17T22:44:50,024 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_foreign_keys : parentdb=null parenttbl=null foreigndb=default
> foreigntbl=test2
> 2019-10-17T22:44:50,025 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_unique_constraints : tbl=hive.default.test2
> 2019-10-17T22:44:50,026 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_not_null_constraints : tbl=hive.default.test2
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22685) TestHiveSqlDateTimeFormatter Now Broken with New Year 2020
[
https://issues.apache.org/jira/browse/HIVE-22685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015004#comment-17015004
]
Karen Coppage commented on HIVE-22685:
--
Hi David,
Thanks for fixing the issue. The flaw in my thinking was that the test was only
guaranteed to pass on Thursdays :) Thanks also for fixing the minor problems
that I didn't catch the first time, it was educational to look through your
changes.
I'm thinking it's a bad idea to introduce a field (LocalDateTime now) in the
main class (HiveSqlDateTimeFormatter) just for testing purposes. Also, if I
understand correctly, Optional objects aren't supposed to be used as class
variables, and they're also not serializable, which is what Findbugs is
complaining about. A simpler solution that doesn't require changing
HiveSqlDateTimeFormatter is to get the week-based year from the java.time
library:
{code:java}
private String getIsoYear() {
return LocalDateTime.now().format(DateTimeFormatter.ofPattern("")); //
week-based year
}
{code}
and change the two offending lines to:
{code:java}
checkParseTimestampIso("IY-IW-ID", "0-01-04", "iw, ", "01, " +
getIsoYear().substring(0, 3) + "0");
checkParseTimestampIso("I-IW-ID", "0-01-04", "iw, ", "01, " +
getIsoYear().substring(0, 3) + "0");
{code}
What do you think?
> TestHiveSqlDateTimeFormatter Now Broken with New Year 2020
> --
>
> Key: HIVE-22685
> URL: https://issues.apache.org/jira/browse/HIVE-22685
> Project: Hive
> Issue Type: Bug
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-22685.1.patch, HIVE-22685.2.patch
>
>
> Unit test is now broken (n)(n):(
> {code:java}
> //Tests for these patterns would need changing every decade if done in
> the above way.
> //Thursday of the first week in an ISO year always matches the Gregorian
> year.
> checkParseTimestampIso("IY-IW-ID", "0-01-04", "iw, ", "01, " +
> thisYearString.substring(0, 3) + "0");
> checkParseTimestampIso("I-IW-ID", "0-01-04", "iw, ", "01, " +
> thisYearString.substring(0, 3) + "0");
> {code}
> {code}
> org.junit.ComparisonFailure: expected:<01, 20[1]0> but was:<01, 20[2]0>
> at org.junit.Assert.assertEquals(Assert.java:115)
> at org.junit.Assert.assertEquals(Assert.java:144)
> at
> org.apache.hadoop.hive.common.format.datetime.TestHiveSqlDateTimeFormatter.checkParseTimestampIso(TestHiveSqlDateTimeFormatter.java:313)
> at
> org.apache.hadoop.hive.common.format.datetime.TestHiveSqlDateTimeFormatter.testParseTimestamp(TestHiveSqlDateTimeFormatter.java:287)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22366) Multiple metastore calls for same table and constraints during planning
[
https://issues.apache.org/jira/browse/HIVE-22366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-22366:
---
Status: Open (was: Patch Available)
> Multiple metastore calls for same table and constraints during planning
> ---
>
> Key: HIVE-22366
> URL: https://issues.apache.org/jira/browse/HIVE-22366
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-22366.1.patch, HIVE-22366.2.patch,
> HIVE-22366.3.patch
>
>
> e.g. for a simple query like
> {code:sql}
> explain select count(i) from test2 group by j limit 3;
> {code}
> planner makes the following calls:
> {code}
> 2019-10-17T22:44:49,892 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr cmd=get_table :
> tbl=hive.default.test2
> 2019-10-17T22:44:49,908 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_primary_keys : tbl=hive.default.test2
> 2019-10-17T22:44:49,910 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_foreign_keys : parentdb=null parenttbl=null foreigndb=default
> foreigntbl=test2
> 2019-10-17T22:44:49,911 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_unique_constraints : tbl=hive.default.test2
> 2019-10-17T22:44:49,913 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_not_null_constraints : tbl=hive.default.test2
> 2019-10-17T22:44:49,979 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_partitions : tbl=hive.default.test2
> 2019-10-17T22:44:49,997 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_aggr_stats_for: table=hive.default.test2
> 2019-10-17T22:44:50,014 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr cmd=get_table :
> tbl=hive.default.test2
> 2019-10-17T22:44:50,023 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_primary_keys : tbl=hive.default.test2
> 2019-10-17T22:44:50,024 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_foreign_keys : parentdb=null parenttbl=null foreigndb=default
> foreigntbl=test2
> 2019-10-17T22:44:50,025 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_unique_constraints : tbl=hive.default.test2
> 2019-10-17T22:44:50,026 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_not_null_constraints : tbl=hive.default.test2
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22366) Multiple metastore calls for same table and constraints during planning
[
https://issues.apache.org/jira/browse/HIVE-22366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-22366:
---
Status: Patch Available (was: Open)
> Multiple metastore calls for same table and constraints during planning
> ---
>
> Key: HIVE-22366
> URL: https://issues.apache.org/jira/browse/HIVE-22366
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-22366.1.patch, HIVE-22366.2.patch,
> HIVE-22366.3.patch
>
>
> e.g. for a simple query like
> {code:sql}
> explain select count(i) from test2 group by j limit 3;
> {code}
> planner makes the following calls:
> {code}
> 2019-10-17T22:44:49,892 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr cmd=get_table :
> tbl=hive.default.test2
> 2019-10-17T22:44:49,908 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_primary_keys : tbl=hive.default.test2
> 2019-10-17T22:44:49,910 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_foreign_keys : parentdb=null parenttbl=null foreigndb=default
> foreigntbl=test2
> 2019-10-17T22:44:49,911 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_unique_constraints : tbl=hive.default.test2
> 2019-10-17T22:44:49,913 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_not_null_constraints : tbl=hive.default.test2
> 2019-10-17T22:44:49,979 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_partitions : tbl=hive.default.test2
> 2019-10-17T22:44:49,997 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_aggr_stats_for: table=hive.default.test2
> 2019-10-17T22:44:50,014 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr cmd=get_table :
> tbl=hive.default.test2
> 2019-10-17T22:44:50,023 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_primary_keys : tbl=hive.default.test2
> 2019-10-17T22:44:50,024 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_foreign_keys : parentdb=null parenttbl=null foreigndb=default
> foreigntbl=test2
> 2019-10-17T22:44:50,025 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_unique_constraints : tbl=hive.default.test2
> 2019-10-17T22:44:50,026 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_not_null_constraints : tbl=hive.default.test2
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22366) Multiple metastore calls for same table and constraints during planning
[
https://issues.apache.org/jira/browse/HIVE-22366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-22366:
---
Attachment: HIVE-22366.3.patch
> Multiple metastore calls for same table and constraints during planning
> ---
>
> Key: HIVE-22366
> URL: https://issues.apache.org/jira/browse/HIVE-22366
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-22366.1.patch, HIVE-22366.2.patch,
> HIVE-22366.3.patch
>
>
> e.g. for a simple query like
> {code:sql}
> explain select count(i) from test2 group by j limit 3;
> {code}
> planner makes the following calls:
> {code}
> 2019-10-17T22:44:49,892 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr cmd=get_table :
> tbl=hive.default.test2
> 2019-10-17T22:44:49,908 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_primary_keys : tbl=hive.default.test2
> 2019-10-17T22:44:49,910 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_foreign_keys : parentdb=null parenttbl=null foreigndb=default
> foreigntbl=test2
> 2019-10-17T22:44:49,911 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_unique_constraints : tbl=hive.default.test2
> 2019-10-17T22:44:49,913 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_not_null_constraints : tbl=hive.default.test2
> 2019-10-17T22:44:49,979 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_partitions : tbl=hive.default.test2
> 2019-10-17T22:44:49,997 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_aggr_stats_for: table=hive.default.test2
> 2019-10-17T22:44:50,014 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr cmd=get_table :
> tbl=hive.default.test2
> 2019-10-17T22:44:50,023 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_primary_keys : tbl=hive.default.test2
> 2019-10-17T22:44:50,024 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_foreign_keys : parentdb=null parenttbl=null foreigndb=default
> foreigntbl=test2
> 2019-10-17T22:44:50,025 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_unique_constraints : tbl=hive.default.test2
> 2019-10-17T22:44:50,026 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_not_null_constraints : tbl=hive.default.test2
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22648) Upgrade Parquet to 1.11.0
[ https://issues.apache.org/jira/browse/HIVE-22648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marta Kuczora updated HIVE-22648: - Attachment: HIVE-22648.3.patch > Upgrade Parquet to 1.11.0 > - > > Key: HIVE-22648 > URL: https://issues.apache.org/jira/browse/HIVE-22648 > Project: Hive > Issue Type: Improvement > Components: File Formats >Reporter: Marta Kuczora >Assignee: Marta Kuczora >Priority: Major > Labels: Parquet, parquet > Attachments: HIVE-22648.1.patch, HIVE-22648.2.patch, > HIVE-22648.3.patch > > > Upgrade the Parquet version to 1.11.0. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22640) Decimal64ColumnVector: ClassCastException when partition column type is Decimal
[
https://issues.apache.org/jira/browse/HIVE-22640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17014934#comment-17014934
]
Denys Kuzmenko commented on HIVE-22640:
---
[~kgyrtkirk], [~rameshkumar] thank you for the review!
I think, decimal64 optimizations should apply for virtual columns as well.
createColumnVectorFromRowColumnTypeInfos() is used also for regular column to
create a proper vector (Decimal64ColumnVector/DecimalColumnVector) depending on
`hive.vectorized.input.format.supports.enabled=decimal_64`.
> Decimal64ColumnVector: ClassCastException when partition column type is
> Decimal
> ---
>
> Key: HIVE-22640
> URL: https://issues.apache.org/jira/browse/HIVE-22640
> Project: Hive
> Issue Type: Bug
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-22640.1.patch, HIVE-22640.2.patch,
> HIVE-22640.3.patch, HIVE-22640.4.patch
>
>
> Problem statement:
> {code}
> CREATE EXTERNAL TABLE output (id DECIMAL(4,0), foo VARCHAR(10))
> PARTITIONED BY (nr_bank *decimal*(4,0))
> STORED AS ORC TBLPROPERTIES ('orc.compress'='ZLIB')
> insert into output PARTITION (nr_bank=88) values (1,'test');
> insert into output PARTITION (nr_bank=8801) values (1,'8801');
> select count(*), nr_bank from sdlint.output group by nr_bank;
> {code}
> The above query fails with:
> {code}
> Caused by: java.lang.ClassCastException:
> org.apache.hadoop.hive.ql.exec.vector.DecimalColumnVector cannot be cast to
> org.apache.hadoop.hive.ql.exec.vector.LongColumnVector
> at
> org.apache.hadoop.hive.ql.exec.vector.expressions.FuncLongToDecimal.evaluate(FuncLongToDecimal.java:58)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22725) Lazy evaluate HiveMetastore::fireReadTablePreEvent table computation
[ https://issues.apache.org/jira/browse/HIVE-22725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17014922#comment-17014922 ] Rajesh Balamohan commented on HIVE-22725: - Added guava's supplier::memoize in patch .2. > Lazy evaluate HiveMetastore::fireReadTablePreEvent table computation > > > Key: HIVE-22725 > URL: https://issues.apache.org/jira/browse/HIVE-22725 > Project: Hive > Issue Type: Improvement >Reporter: Rajesh Balamohan >Assignee: Rajesh Balamohan >Priority: Minor > Attachments: HIVE-22725.1.patch, HIVE-22725.2.patch, > image-2020-01-14-13-22-54-483.png > > > "TransactionalValidationListener" gets added in the pre-event listeners of > HMS by default. > [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java#L559] > This causes issue in short select queries, as table details are computed for > any partition lookups. > [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java#L4984] > > !image-2020-01-14-13-22-54-483.png|width=579,height=202! > > It would be good to lazy evaluate table lookup in this codepath. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22725) Lazy evaluate HiveMetastore::fireReadTablePreEvent table computation
[ https://issues.apache.org/jira/browse/HIVE-22725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HIVE-22725: Attachment: HIVE-22725.2.patch > Lazy evaluate HiveMetastore::fireReadTablePreEvent table computation > > > Key: HIVE-22725 > URL: https://issues.apache.org/jira/browse/HIVE-22725 > Project: Hive > Issue Type: Improvement >Reporter: Rajesh Balamohan >Assignee: Rajesh Balamohan >Priority: Minor > Attachments: HIVE-22725.1.patch, HIVE-22725.2.patch, > image-2020-01-14-13-22-54-483.png > > > "TransactionalValidationListener" gets added in the pre-event listeners of > HMS by default. > [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java#L559] > This causes issue in short select queries, as table details are computed for > any partition lookups. > [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java#L4984] > > !image-2020-01-14-13-22-54-483.png|width=579,height=202! > > It would be good to lazy evaluate table lookup in this codepath. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-22726) TopN Key optimizer should use array instead of priority queue
[ https://issues.apache.org/jira/browse/HIVE-22726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Magyar reassigned HIVE-22726: > TopN Key optimizer should use array instead of priority queue > - > > Key: HIVE-22726 > URL: https://issues.apache.org/jira/browse/HIVE-22726 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Attila Magyar >Assignee: Attila Magyar >Priority: Major > Fix For: 4.0.0 > > > The TopN key optimizer currently uses a priority queue for keeping track of > the largest/smallest rows. Its max size is the same as the user specified > limit. This should be replaced a more cache line friendly array with a small > (128) maximum size and see how much performance is gained. -- This message was sent by Atlassian Jira (v8.3.4#803005)
