[jira] [Comment Edited] (HIVE-23452) Exception occur when a SQL query across data stored in two relational DB by JDBCStorageHandler with Tez
[
https://issues.apache.org/jira/browse/HIVE-23452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17105975#comment-17105975
]
De Li edited comment on HIVE-23452 at 5/13/20, 7:02 AM:
{code:java}
// mysql table
CREATE EXTERNAL TABLE IF NOT EXISTS test_mysqltable_ex (
id int,
name String
)
STORED BY 'org.apache.hive.storage.jdbc.JdbcStorageHandler'
TBLPROPERTIES (
"hive.sql.database.type" = "MYSQL",
"hive.sql.jdbc.driver" = "com.mysql.jdbc.Driver",
"hive.sql.jdbc.url" = "jdbc:mysql://xxx/test_mysql",
"hive.sql.dbcp.username" = "xxx",
"hive.sql.dbcp.password" = "",
"hive.sql.table" = "test_mysqltable"
);
{code}
{code:java}
// postgresql
CREATE EXTERNAL TABLE IF NOT EXISTS gp_test (
id int
)
STORED BY 'org.apache.hive.storage.jdbc.JdbcStorageHandler'
TBLPROPERTIES (
"hive.sql.database.type" = "POSTGRES",
"hive.sql.jdbc.driver" = "org.postgresql.Driver",
"hive.sql.jdbc.url" = "jdbc:postgresql://:5432/pg",
"hive.sql.dbcp.username" = "",
"hive.sql.dbcp.password" = "",
"hive.sql.table" = "test_gp"
);
{code}
{code:java}
// code placeholder
SELECT aa.id,bb.name FROM default.gp_test aa left join
default.test_mysqltable_ex bb on aa.id=bb.id
where bb.id >1;
{code}
was (Author: lide):
{code:java}
// mysql table
CREATE EXTERNAL TABLE IF NOT EXISTS test_mysqltable_ex (
id int,
name String
)
STORED BY 'org.apache.hive.storage.jdbc.JdbcStorageHandler'
TBLPROPERTIES (
"hive.sql.database.type" = "MYSQL",
"hive.sql.jdbc.driver" = "com.mysql.jdbc.Driver",
"hive.sql.jdbc.url" = "jdbc:mysql://xxx/test_mysql",
"hive.sql.dbcp.username" = "xxx",
"hive.sql.dbcp.password" = "",
"hive.sql.table" = "test_mysqltable"
);
{code}
{code:java}
// postgresql
CREATE EXTERNAL TABLE IF NOT EXISTS gp_test (
id int
)
STORED BY 'org.apache.hive.storage.jdbc.JdbcStorageHandler'
TBLPROPERTIES (
"hive.sql.database.type" = "POSTGRES",
"hive.sql.jdbc.driver" = "org.postgresql.Driver",
"hive.sql.jdbc.url" = "jdbc:postgresql://:5432/cdn",
"hive.sql.dbcp.username" = "",
"hive.sql.dbcp.password" = "",
"hive.sql.table" = "test_gp"
);
{code}
{code:java}
// code placeholder
SELECT aa.id,bb.name FROM default.gp_test aa left join
default.test_mysqltable_ex bb on aa.id=bb.id
where bb.id >1;
{code}
> Exception occur when a SQL query across data stored in two relational DB by
> JDBCStorageHandler with Tez
> ---
>
> Key: HIVE-23452
> URL: https://issues.apache.org/jira/browse/HIVE-23452
> Project: Hive
> Issue Type: Bug
> Components: Tez
>Affects Versions: 3.1.0
>Reporter: De Li
>Priority: Major
>
> Exception occur when a SQL query across data stored in two relational DB by
> JDBCStorageHandler with Tez. It seems there is an incorrect JDBC driver by
> Tez and it works when query with MR.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23458) Introduce unified thread pool for scheduled jobs
[ https://issues.apache.org/jira/browse/HIVE-23458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Chung updated HIVE-23458: Attachment: HIVE-23458.01.patch Status: Patch Available (was: Open) > Introduce unified thread pool for scheduled jobs > > > Key: HIVE-23458 > URL: https://issues.apache.org/jira/browse/HIVE-23458 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Reporter: Eugene Chung >Assignee: Eugene Chung >Priority: Major > Attachments: HIVE-23458.01.patch > > > As I mentioned in [the comment of > HIVE-23164|https://issues.apache.org/jira/browse/HIVE-23164?focusedCommentId=17089506=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17089506], > I've made the unified scheduled executor service like > org.apache.hadoop.hive.metastore.ThreadPool. > I think it could help > 1. to minimize the possibility of making non-daemon threads when developers > need ScheduledExecutorService > 2. to achieve the utilization of server resources because the current > situation is all of the modules make its own ScheduledExecutorService and all > of the threads are just using for one job. > 3. an administrator of Hive server by providing > hive.exec.scheduler.num.threads configuration. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23452) Exception occur when a SQL query across data stored in two relational DB by JDBCStorageHandler with Tez
[ https://issues.apache.org/jira/browse/HIVE-23452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106026#comment-17106026 ] De Li commented on HIVE-23452: -- Yes, table 'test_mysql.test_gp' doesn't exist because it is a wrong combination produced by Hive, so it lead to the bug. > Exception occur when a SQL query across data stored in two relational DB by > JDBCStorageHandler with Tez > --- > > Key: HIVE-23452 > URL: https://issues.apache.org/jira/browse/HIVE-23452 > Project: Hive > Issue Type: Bug > Components: Tez >Affects Versions: 3.1.0 >Reporter: De Li >Priority: Major > > Exception occur when a SQL query across data stored in two relational DB by > JDBCStorageHandler with Tez. It seems there is an incorrect JDBC driver by > Tez and it works when query with MR. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23458) Introduce unified thread pool for scheduled jobs
[ https://issues.apache.org/jira/browse/HIVE-23458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Chung updated HIVE-23458: Fix Version/s: 4.0.0 > Introduce unified thread pool for scheduled jobs > > > Key: HIVE-23458 > URL: https://issues.apache.org/jira/browse/HIVE-23458 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Reporter: Eugene Chung >Assignee: Eugene Chung >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-23458.01.patch > > > As I mentioned in [the comment of > HIVE-23164|https://issues.apache.org/jira/browse/HIVE-23164?focusedCommentId=17089506=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17089506], > I've made the unified scheduled executor service like > org.apache.hadoop.hive.metastore.ThreadPool. > I think it could help > 1. to minimize the possibility of making non-daemon threads when developers > need ScheduledExecutorService > 2. to achieve the utilization of server resources because the current > situation is all of the modules make its own ScheduledExecutorService and all > of the threads are just using for one job. > 3. an administrator of Hive server by providing > hive.exec.scheduler.num.threads configuration. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23133) Numeric operations can have different result across hardware archs
[ https://issues.apache.org/jira/browse/HIVE-23133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenyu Zheng updated HIVE-23133: Description: Use java.lang.StrictMath rather than java.lang.Math in UDFs: UDFCos.java UDFExp.java UDFLn.java UDFLog.java Currently, we have set up an ARM CI to test out how Hive works on ARM platform: [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/] Among the failures, we have observed that some numeric operations can have different result across hardware archs, such as: [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/25/testReport/org.apache.hadoop.hive.cli/TestCliDriver/testCliDriver_vector_decimal_udf2_/] [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/25/testReport/org.apache.hadoop.hive.cli/TestSparkCliDriver/testCliDriver_subquery_select_/] [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/25/testReport/org.apache.hadoop.hive.cli/TestSparkCliDriver/testCliDriver_vectorized_math_funcs_/] we can see that the calculation results of log, exp, cos, toRadians etc is slitly different than the .out file results that we are comparing(they are tested and wrote on X86 machines), this is because of we use [Math Library|https://docs.oracle.com/javase/6/docs/api/java/lang/Math.html] for these kind of calculations. and according to the [illustration|https://docs.oracle.com/javase/6/docs/api/java/lang/Math.html]: _Unlike some of the numeric methods of class StrictMath, all implementations of the equivalent functions of class Math are not_ _defined to return the bit-for-bit same results. This relaxation permits better-performing implementations where strict reproducibility_ _is not required._ _By default many of the Math methods simply call the equivalent method in StrictMath for their implementation._ _Code generators are encouraged to use platform-specific native libraries or microprocessor instructions, where available,_ _to provide higher-performance implementations of Math methods._ so the result will have difference across hardware archs. On the other hand, JAVA provided another library [StrictMath|https://docs.oracle.com/javase/6/docs/api/java/lang/StrictMath.html] that will not have this kind of problem as according to its' [reference|https://docs.oracle.com/javase/6/docs/api/java/lang/StrictMath.html]: _To help ensure portability of Java programs, the definitions of some of the numeric functions in this package require that they produce_ _the same results as certain published algorithms._ So in order to fix the above mentioned problem, we have to consider switch to use StrictMath instead of Math. was: Use java.lang.StrictMath rather than java.lang.Math in UDFs: UDF Currently, we have set up an ARM CI to test out how Hive works on ARM platform: [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/] Among the failures, we have observed that some numeric operations can have different result across hardware archs, such as: [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/25/testReport/org.apache.hadoop.hive.cli/TestCliDriver/testCliDriver_vector_decimal_udf2_/] [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/25/testReport/org.apache.hadoop.hive.cli/TestSparkCliDriver/testCliDriver_subquery_select_/] [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/25/testReport/org.apache.hadoop.hive.cli/TestSparkCliDriver/testCliDriver_vectorized_math_funcs_/] we can see that the calculation results of log, exp, cos, toRadians etc is slitly different than the .out file results that we are comparing(they are tested and wrote on X86 machines), this is because of we use [Math Library|https://docs.oracle.com/javase/6/docs/api/java/lang/Math.html] for these kind of calculations. and according to the [illustration|https://docs.oracle.com/javase/6/docs/api/java/lang/Math.html]: _Unlike some of the numeric methods of class StrictMath, all implementations of the equivalent functions of class Math are not_ _defined to return the bit-for-bit same results. This relaxation permits better-performing implementations where strict reproducibility_ _is not required._ _By default many of the Math methods simply call the equivalent method in StrictMath for their implementation._ _Code generators are encouraged to use platform-specific native libraries or microprocessor instructions, where available,_ _to provide higher-performance implementations of Math methods._ so the result will have difference across hardware archs. On the other hand, JAVA provided another library [StrictMath|https://docs.oracle.com/javase/6/docs/api/java/lang/StrictMath.html] that will not have this kind of problem as according to its' [reference|https://docs.oracle.com/javase/6/docs/api/java/lang/StrictMath.html]: _To
[jira] [Updated] (HIVE-23458) Introduce unified thread pool for scheduled jobs
[ https://issues.apache.org/jira/browse/HIVE-23458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Chung updated HIVE-23458: Description: As I mentioned in [the comment of HIVE-23164|https://issues.apache.org/jira/browse/HIVE-23164?focusedCommentId=17089506=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17089506], I've made the unified scheduled executor service like org.apache.hadoop.hive.metastore.ThreadPool. I think it could help 1. to minimize the possibility of making non-daemon threads when developers need ScheduledExecutorService 2. to achieve the utilization of server resources because the current situation is all of the modules make its own ScheduledExecutorService and all of the threads are just using for one job. 3. administrators of Hive servers by providing hive.exec.scheduler.num.threads configuration so that they can predict and set how many threads are used and needed. was: As I mentioned in [the comment of HIVE-23164|https://issues.apache.org/jira/browse/HIVE-23164?focusedCommentId=17089506=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17089506], I've made the unified scheduled executor service like org.apache.hadoop.hive.metastore.ThreadPool. I think it could help 1. to minimize the possibility of making non-daemon threads when developers need ScheduledExecutorService 2. to achieve the utilization of server resources because the current situation is all of the modules make its own ScheduledExecutorService and all of the threads are just using for one job. 3. an administrator of Hive server by providing hive.exec.scheduler.num.threads configuration. > Introduce unified thread pool for scheduled jobs > > > Key: HIVE-23458 > URL: https://issues.apache.org/jira/browse/HIVE-23458 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Reporter: Eugene Chung >Assignee: Eugene Chung >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-23458.01.patch > > > As I mentioned in [the comment of > HIVE-23164|https://issues.apache.org/jira/browse/HIVE-23164?focusedCommentId=17089506=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17089506], > I've made the unified scheduled executor service like > org.apache.hadoop.hive.metastore.ThreadPool. > I think it could help > 1. to minimize the possibility of making non-daemon threads when developers > need ScheduledExecutorService > 2. to achieve the utilization of server resources because the current > situation is all of the modules make its own ScheduledExecutorService and all > of the threads are just using for one job. > 3. administrators of Hive servers by providing > hive.exec.scheduler.num.threads configuration so that they can predict and > set how many threads are used and needed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23452) Exception occur when a SQL query across data stored in two relational DB by JDBCStorageHandler with Tez
[ https://issues.apache.org/jira/browse/HIVE-23452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106028#comment-17106028 ] De Li commented on HIVE-23452: -- Both the table 'pg.test_gp' and 'test_mysql.test_mysqltable' are exist. > Exception occur when a SQL query across data stored in two relational DB by > JDBCStorageHandler with Tez > --- > > Key: HIVE-23452 > URL: https://issues.apache.org/jira/browse/HIVE-23452 > Project: Hive > Issue Type: Bug > Components: Tez >Affects Versions: 3.1.0 >Reporter: De Li >Priority: Major > > Exception occur when a SQL query across data stored in two relational DB by > JDBCStorageHandler with Tez. It seems there is an incorrect JDBC driver by > Tez and it works when query with MR. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23451) FileSinkOperator calls deleteOnExit (hdfs call) twice for the same file
[
https://issues.apache.org/jira/browse/HIVE-23451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106094#comment-17106094
]
Hive QA commented on HIVE-23451:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 11m
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
31s{color} | {color:blue} ql in master has 1527 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
6s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 29m 34s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22304/dev-support/hive-personality.sh
|
| git revision | master / 9ffbbdc |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22304/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> FileSinkOperator calls deleteOnExit (hdfs call) twice for the same file
> ---
>
> Key: HIVE-23451
> URL: https://issues.apache.org/jira/browse/HIVE-23451
> Project: Hive
> Issue Type: Improvement
>Reporter: Rajesh Balamohan
>Assignee: Rajesh Balamohan
>Priority: Minor
> Attachments: HIVE-23451.1.patch, HIVE-23451.2.patch
>
>
> [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java#L826]
> [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java#L797]
> Can avoid a NN call here (i.e, mainly for small queries).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23432) Add Ranger Replication Metrics
[
https://issues.apache.org/jira/browse/HIVE-23432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106004#comment-17106004
]
Hive QA commented on HIVE-23432:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13002775/HIVE-23432.03.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 17266 tests
executed
*Failed tests:*
{noformat}
TestStatsReplicationScenariosACID - did not produce a TEST-*.xml file (likely
timed out) (batchId=184)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druid_materialized_view_rewrite_ssb]
(batchId=128)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22302/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22302/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22302/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13002775 - PreCommit-HIVE-Build
> Add Ranger Replication Metrics
> ---
>
> Key: HIVE-23432
> URL: https://issues.apache.org/jira/browse/HIVE-23432
> Project: Hive
> Issue Type: Task
>Reporter: Aasha Medhi
>Assignee: Aasha Medhi
>Priority: Major
> Attachments: HIVE-23432.01.patch, HIVE-23432.02.patch,
> HIVE-23432.03.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (HIVE-23452) Exception occur when a SQL query across data stored in two relational DB by JDBCStorageHandler with Tez
[
https://issues.apache.org/jira/browse/HIVE-23452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17105975#comment-17105975
]
De Li edited comment on HIVE-23452 at 5/13/20, 6:50 AM:
{code:java}
// code placeholder
CREATE EXTERNAL TABLE IF NOT EXISTS test_gp_ex (
i int
)
STORED BY 'org.apache.hive.storage.jdbc.JdbcStorageHandler'
TBLPROPERTIES (
"hive.sql.database.type" = "POSTGRES",
"hive.sql.jdbc.driver" = "org.postgresql.Driver",
"hive.sql.jdbc.url" = "jdbc:postgresql://:5432/db",
"hive.sql.dbcp.username" = "gpadmin",
"hive.sql.dbcp.password" = "xxx",
"hive.sql.table" = "test_gp"
);
CREATE EXTERNAL TABLE IF NOT EXISTS test_mysqltable_ex (
id int,
name String
)
STORED BY 'org.apache.hive.storage.jdbc.JdbcStorageHandler'
TBLPROPERTIES (
"hive.sql.database.type" = "MYSQL",
"hive.sql.jdbc.driver" = "com.mysql.jdbc.Driver",
"hive.sql.jdbc.url" = "jdbc:mysql://xxx/test_mysql",
"hive.sql.dbcp.username" = "xxx",
"hive.sql.dbcp.password" = "",
"hive.sql.table" = "test_mysqltable"
);
SELECT aa.id,bb.name FROM test_gp_ex aa left join test_mysqltable_ex bb on
aa.id=bb.id
where bb.id >1;
{code}
was (Author: lide):
{code:java}
// code placeholder
CREATE EXTERNAL TABLE IF NOT EXISTS test_gp_ex (
i int
)
STORED BY 'org.apache.hive.storage.jdbc.JdbcStorageHandler'
TBLPROPERTIES (
"hive.sql.database.type" = "POSTGRES",
"hive.sql.jdbc.driver" = "org.postgresql.Driver",
"hive.sql.jdbc.url" = "jdbc:postgresql://:5432/db",
"hive.sql.dbcp.username" = "gpadmin",
"hive.sql.dbcp.password" = "xxx",
"hive.sql.table" = "test_gp"
);
CREATE EXTERNAL TABLE IF NOT EXISTS test_mysql_table_db (
id int,
name String
)
STORED BY 'org.apache.hive.storage.jdbc.JdbcStorageHandler'
TBLPROPERTIES (
"hive.sql.database.type" = "MYSQL",
"hive.sql.jdbc.driver" = "com.mysql.jdbc.Driver",
"hive.sql.jdbc.url" = "jdbc:mysql://xxx/test_mysql",
"hive.sql.dbcp.username" = "",
"hive.sql.dbcp.password" = "*",
"hive.sql.table" = "test_mysql.test_mysqltable"
);
{code}
> Exception occur when a SQL query across data stored in two relational DB by
> JDBCStorageHandler with Tez
> ---
>
> Key: HIVE-23452
> URL: https://issues.apache.org/jira/browse/HIVE-23452
> Project: Hive
> Issue Type: Bug
> Components: Tez
>Affects Versions: 3.1.0
>Reporter: De Li
>Priority: Major
>
> Exception occur when a SQL query across data stored in two relational DB by
> JDBCStorageHandler with Tez. It seems there is an incorrect JDBC driver by
> Tez and it works when query with MR.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23133) Numeric operations can have different result across hardware archs
[ https://issues.apache.org/jira/browse/HIVE-23133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106042#comment-17106042 ] Zhenyu Zheng commented on HIVE-23133: - [~gopalv] Thanks for the review, in previous patches (1-3) we didn't add config, if that is the better way, we can keep it that way. I will add the funcs that we changed to the ticket description. > Numeric operations can have different result across hardware archs > -- > > Key: HIVE-23133 > URL: https://issues.apache.org/jira/browse/HIVE-23133 > Project: Hive > Issue Type: Sub-task >Reporter: Zhenyu Zheng >Assignee: Zhenyu Zheng >Priority: Major > Attachments: HIVE-23133.1.patch, HIVE-23133.2.patch, > HIVE-23133.3.patch, HIVE-23133.4.patch, HIVE-23133.5.patch > > > Currently, we have set up an ARM CI to test out how Hive works on ARM > platform: > [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/] > Among the failures, we have observed that some numeric operations can have > different result across hardware archs, such as: > [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/25/testReport/org.apache.hadoop.hive.cli/TestCliDriver/testCliDriver_vector_decimal_udf2_/] > [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/25/testReport/org.apache.hadoop.hive.cli/TestSparkCliDriver/testCliDriver_subquery_select_/] > [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/25/testReport/org.apache.hadoop.hive.cli/TestSparkCliDriver/testCliDriver_vectorized_math_funcs_/] > we can see that the calculation results of log, exp, cos, toRadians etc is > slitly different than the .out file results that we are > comparing(they are tested and wrote on X86 machines), this is because of we > use [Math > Library|https://docs.oracle.com/javase/6/docs/api/java/lang/Math.html] for > these kind of calculations. > and according to the > [illustration|https://docs.oracle.com/javase/6/docs/api/java/lang/Math.html]: > _Unlike some of the numeric methods of class StrictMath, all implementations > of the equivalent functions of class Math are not_ > _defined to return the bit-for-bit same results. This relaxation permits > better-performing implementations where strict reproducibility_ > _is not required._ > _By default many of the Math methods simply call the equivalent method in > StrictMath for their implementation._ > _Code generators are encouraged to use platform-specific native libraries or > microprocessor instructions, where available,_ > _to provide higher-performance implementations of Math methods._ > so the result will have difference across hardware archs. > On the other hand, JAVA provided another library > [StrictMath|https://docs.oracle.com/javase/6/docs/api/java/lang/StrictMath.html] > that will not have this kind of problem as according to its' > [reference|https://docs.oracle.com/javase/6/docs/api/java/lang/StrictMath.html]: > _To help ensure portability of Java programs, the definitions of some of the > numeric functions in this package require that they produce_ > _the same results as certain published algorithms._ > So in order to fix the above mentioned problem, we have to consider switch to > use StrictMath instead of Math. > > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23133) Numeric operations can have different result across hardware archs
[ https://issues.apache.org/jira/browse/HIVE-23133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenyu Zheng updated HIVE-23133: Description: Use java.lang.StrictMath rather than java.lang.Math in UDFs: UDF Currently, we have set up an ARM CI to test out how Hive works on ARM platform: [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/] Among the failures, we have observed that some numeric operations can have different result across hardware archs, such as: [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/25/testReport/org.apache.hadoop.hive.cli/TestCliDriver/testCliDriver_vector_decimal_udf2_/] [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/25/testReport/org.apache.hadoop.hive.cli/TestSparkCliDriver/testCliDriver_subquery_select_/] [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/25/testReport/org.apache.hadoop.hive.cli/TestSparkCliDriver/testCliDriver_vectorized_math_funcs_/] we can see that the calculation results of log, exp, cos, toRadians etc is slitly different than the .out file results that we are comparing(they are tested and wrote on X86 machines), this is because of we use [Math Library|https://docs.oracle.com/javase/6/docs/api/java/lang/Math.html] for these kind of calculations. and according to the [illustration|https://docs.oracle.com/javase/6/docs/api/java/lang/Math.html]: _Unlike some of the numeric methods of class StrictMath, all implementations of the equivalent functions of class Math are not_ _defined to return the bit-for-bit same results. This relaxation permits better-performing implementations where strict reproducibility_ _is not required._ _By default many of the Math methods simply call the equivalent method in StrictMath for their implementation._ _Code generators are encouraged to use platform-specific native libraries or microprocessor instructions, where available,_ _to provide higher-performance implementations of Math methods._ so the result will have difference across hardware archs. On the other hand, JAVA provided another library [StrictMath|https://docs.oracle.com/javase/6/docs/api/java/lang/StrictMath.html] that will not have this kind of problem as according to its' [reference|https://docs.oracle.com/javase/6/docs/api/java/lang/StrictMath.html]: _To help ensure portability of Java programs, the definitions of some of the numeric functions in this package require that they produce_ _the same results as certain published algorithms._ So in order to fix the above mentioned problem, we have to consider switch to use StrictMath instead of Math. was: Currently, we have set up an ARM CI to test out how Hive works on ARM platform: [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/] Among the failures, we have observed that some numeric operations can have different result across hardware archs, such as: [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/25/testReport/org.apache.hadoop.hive.cli/TestCliDriver/testCliDriver_vector_decimal_udf2_/] [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/25/testReport/org.apache.hadoop.hive.cli/TestSparkCliDriver/testCliDriver_subquery_select_/] [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/25/testReport/org.apache.hadoop.hive.cli/TestSparkCliDriver/testCliDriver_vectorized_math_funcs_/] we can see that the calculation results of log, exp, cos, toRadians etc is slitly different than the .out file results that we are comparing(they are tested and wrote on X86 machines), this is because of we use [Math Library|https://docs.oracle.com/javase/6/docs/api/java/lang/Math.html] for these kind of calculations. and according to the [illustration|https://docs.oracle.com/javase/6/docs/api/java/lang/Math.html]: _Unlike some of the numeric methods of class StrictMath, all implementations of the equivalent functions of class Math are not_ _defined to return the bit-for-bit same results. This relaxation permits better-performing implementations where strict reproducibility_ _is not required._ _By default many of the Math methods simply call the equivalent method in StrictMath for their implementation._ _Code generators are encouraged to use platform-specific native libraries or microprocessor instructions, where available,_ _to provide higher-performance implementations of Math methods._ so the result will have difference across hardware archs. On the other hand, JAVA provided another library [StrictMath|https://docs.oracle.com/javase/6/docs/api/java/lang/StrictMath.html] that will not have this kind of problem as according to its' [reference|https://docs.oracle.com/javase/6/docs/api/java/lang/StrictMath.html]: _To help ensure portability of Java programs, the definitions of some of the numeric functions in this package require that
[jira] [Updated] (HIVE-23444) Concurrent ACID direct inserts may fail with FileNotFoundException
[
https://issues.apache.org/jira/browse/HIVE-23444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marta Kuczora updated HIVE-23444:
-
Attachment: HIVE-23444.1.patch
> Concurrent ACID direct inserts may fail with FileNotFoundException
> --
>
> Key: HIVE-23444
> URL: https://issues.apache.org/jira/browse/HIVE-23444
> Project: Hive
> Issue Type: Bug
>Reporter: Marta Kuczora
>Assignee: Marta Kuczora
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-23444.1.patch, HIVE-23444.1.patch
>
>
> The following exception may occur when concurrently inserting into an ACID
> table with static partitions and the 'hive.acid.direct.insert.enabled'
> parameter is true. This issue occurs intermittently.
> {noformat}
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> java.io.IOException: java.io.FileNotFoundException: File
> hdfs://ns1/warehouse/tablespace/managed/hive/tpch_unbucketed.db/concurrent_insert_partitioned/l_tax=0.0/_tmp.delta_001_001_
> does not exist.
> at
> org.apache.hadoop.hive.ql.metadata.Hive.loadPartitionInternal(Hive.java:2465)
> ~[hive-exec-3.1.3000.7.1.1.0-493.jar:3.1.3000.7.1.1.0-493]
> at
> org.apache.hadoop.hive.ql.metadata.Hive.loadPartition(Hive.java:2228)
> ~[hive-exec-3.1.3000.7.1.1.0-493.jar:3.1.3000.7.1.1.0-493]
> at
> org.apache.hadoop.hive.ql.exec.MoveTask.handleStaticParts(MoveTask.java:522)
> ~[hive-exec-3.1.3000.7.1.1.0-493.jar:3.1.3000.7.1.1.0-493]
> at org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:442)
> ~[hive-exec-3.1.3000.7.1.1.0-493.jar:3.1.3000.7.1.1.0-493]
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:213)
> ~[hive-exec-3.1.3000.7.1.1.0-493.jar:3.1.3000.7.1.1.0-493]
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:105)
> ~[hive-exec-3.1.3000.7.1.1.0-493.jar:3.1.3000.7.1.1.0-493]
> at org.apache.hadoop.hive.ql.Executor.launchTask(Executor.java:359)
> ~[hive-exec-3.1.3000.7.1.1.0-493.jar:3.1.3000.7.1.1.0-493]
> at org.apache.hadoop.hive.ql.Executor.launchTasks(Executor.java:330)
> ~[hive-exec-3.1.3000.7.1.1.0-493.jar:3.1.3000.7.1.1.0-493]
> at org.apache.hadoop.hive.ql.Executor.runTasks(Executor.java:246)
> ~[hive-exec-3.1.3000.7.1.1.0-493.jar:3.1.3000.7.1.1.0-493]
> at org.apache.hadoop.hive.ql.Executor.execute(Executor.java:109)
> ~[hive-exec-3.1.3000.7.1.1.0-493.jar:3.1.3000.7.1.1.0-493]
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:721)
> ~[hive-exec-3.1.3000.7.1.1.0-493.jar:3.1.3000.7.1.1.0-493]
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:488)
> ~[hive-exec-3.1.3000.7.1.1.0-493.jar:3.1.3000.7.1.1.0-493]
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:482)
> ~[hive-exec-3.1.3000.7.1.1.0-493.jar:3.1.3000.7.1.1.0-493]
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:166)
> ~[hive-exec-3.1.3000.7.1.1.0-493.jar:3.1.3000.7.1.1.0-493]
> at
> org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:225)
> ~[hive-service-3.1.3000.7.1.1.0-493.jar:3.1.3000.7.1.1.0-493]
> ... 13 more
> Caused by: java.io.IOException: java.io.FileNotFoundException: File
> hdfs://ns1/warehouse/tablespace/managed/hive/tpch_unbucketed.db/concurrent_insert_partitioned/l_tax=0.0/_tmp.delta_001_001_
> does not exist.
> at
> org.apache.hadoop.hive.ql.io.AcidUtils.getHdfsDirSnapshots(AcidUtils.java:1472)
> ~[hive-exec-3.1.3000.7.1.1.0-493.jar:3.1.3000.7.1.1.0-493]
> at
> org.apache.hadoop.hive.ql.io.AcidUtils.getAcidState(AcidUtils.java:1297)
> ~[hive-exec-3.1.3000.7.1.1.0-493.jar:3.1.3000.7.1.1.0-493]
> at
> org.apache.hadoop.hive.ql.io.AcidUtils.getAcidFilesForStats(AcidUtils.java:2695)
> ~[hive-exec-3.1.3000.7.1.1.0-493.jar:3.1.3000.7.1.1.0-493]
> at
> org.apache.hadoop.hive.ql.metadata.Hive.loadPartitionInternal(Hive.java:2448)
> ~[hive-exec-3.1.3000.7.1.1.0-493.jar:3.1.3000.7.1.1.0-493]
> at
> org.apache.hadoop.hive.ql.metadata.Hive.loadPartition(Hive.java:2228)
> ~[hive-exec-3.1.3000.7.1.1.0-493.jar:3.1.3000.7.1.1.0-493]
> at
> org.apache.hadoop.hive.ql.exec.MoveTask.handleStaticParts(MoveTask.java:522)
> ~[hive-exec-3.1.3000.7.1.1.0-493.jar:3.1.3000.7.1.1.0-493]
> at org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:442)
> ~[hive-exec-3.1.3000.7.1.1.0-493.jar:3.1.3000.7.1.1.0-493]
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:213)
> ~[hive-exec-3.1.3000.7.1.1.0-493.jar:3.1.3000.7.1.1.0-493]
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:105)
> ~[hive-exec-3.1.3000.7.1.1.0-493.jar:3.1.3000.7.1.1.0-493]
> at
[jira] [Commented] (HIVE-23432) Add Ranger Replication Metrics
[
https://issues.apache.org/jira/browse/HIVE-23432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17105996#comment-17105996
]
Hive QA commented on HIVE-23432:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
19s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
10s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
35s{color} | {color:blue} ql in master has 1527 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
44s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
27s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
33s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
47s{color} | {color:green} ql: The patch generated 0 new + 7 unchanged - 2
fixed = 7 total (was 9) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
24s{color} | {color:green} The patch hive-unit passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 36m 17s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile
xml |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22302/dev-support/hive-personality.sh
|
| git revision | master / 9ffbbdc |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22302/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Add Ranger Replication Metrics
> ---
>
> Key: HIVE-23432
> URL: https://issues.apache.org/jira/browse/HIVE-23432
> Project: Hive
> Issue Type: Task
>Reporter: Aasha Medhi
>Assignee: Aasha Medhi
>Priority: Major
> Attachments: HIVE-23432.01.patch, HIVE-23432.02.patch,
> HIVE-23432.03.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (HIVE-23452) Exception occur when a SQL query across data stored in two relational DB by JDBCStorageHandler with Tez
[
https://issues.apache.org/jira/browse/HIVE-23452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17105975#comment-17105975
]
De Li edited comment on HIVE-23452 at 5/13/20, 6:58 AM:
{code:java}
// mysql table
CREATE EXTERNAL TABLE IF NOT EXISTS test_mysqltable_ex (
id int,
name String
)
STORED BY 'org.apache.hive.storage.jdbc.JdbcStorageHandler'
TBLPROPERTIES (
"hive.sql.database.type" = "MYSQL",
"hive.sql.jdbc.driver" = "com.mysql.jdbc.Driver",
"hive.sql.jdbc.url" = "jdbc:mysql://xxx/test_mysql",
"hive.sql.dbcp.username" = "xxx",
"hive.sql.dbcp.password" = "",
"hive.sql.table" = "test_mysqltable"
);
{code}
{code:java}
// postgresql
CREATE EXTERNAL TABLE IF NOT EXISTS gp_test (
id int
)
STORED BY 'org.apache.hive.storage.jdbc.JdbcStorageHandler'
TBLPROPERTIES (
"hive.sql.database.type" = "POSTGRES",
"hive.sql.jdbc.driver" = "org.postgresql.Driver",
"hive.sql.jdbc.url" = "jdbc:postgresql://:5432/cdn",
"hive.sql.dbcp.username" = "",
"hive.sql.dbcp.password" = "",
"hive.sql.table" = "test_gp"
);
{code}
{code:java}
// code placeholder
SELECT aa.id,bb.name FROM default.gp_test aa left join
default.test_mysqltable_ex bb on aa.id=bb.id
where bb.id >1;
{code}
was (Author: lide):
{code:java}
// code placeholder
CREATE EXTERNAL TABLE IF NOT EXISTS test_gp_ex (
i int
)
STORED BY 'org.apache.hive.storage.jdbc.JdbcStorageHandler'
TBLPROPERTIES (
"hive.sql.database.type" = "POSTGRES",
"hive.sql.jdbc.driver" = "org.postgresql.Driver",
"hive.sql.jdbc.url" = "jdbc:postgresql://:5432/db",
"hive.sql.dbcp.username" = "gpadmin",
"hive.sql.dbcp.password" = "xxx",
"hive.sql.table" = "test_gp"
);
CREATE EXTERNAL TABLE IF NOT EXISTS test_mysqltable_ex (
id int,
name String
)
STORED BY 'org.apache.hive.storage.jdbc.JdbcStorageHandler'
TBLPROPERTIES (
"hive.sql.database.type" = "MYSQL",
"hive.sql.jdbc.driver" = "com.mysql.jdbc.Driver",
"hive.sql.jdbc.url" = "jdbc:mysql://xxx/test_mysql",
"hive.sql.dbcp.username" = "xxx",
"hive.sql.dbcp.password" = "",
"hive.sql.table" = "test_mysqltable"
);
SELECT aa.id,bb.name FROM test_gp_ex aa left join test_mysqltable_ex bb on
aa.id=bb.id
where bb.id >1;
{code}
> Exception occur when a SQL query across data stored in two relational DB by
> JDBCStorageHandler with Tez
> ---
>
> Key: HIVE-23452
> URL: https://issues.apache.org/jira/browse/HIVE-23452
> Project: Hive
> Issue Type: Bug
> Components: Tez
>Affects Versions: 3.1.0
>Reporter: De Li
>Priority: Major
>
> Exception occur when a SQL query across data stored in two relational DB by
> JDBCStorageHandler with Tez. It seems there is an incorrect JDBC driver by
> Tez and it works when query with MR.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-23458) Introduce unified thread pool for scheduled jobs
[ https://issues.apache.org/jira/browse/HIVE-23458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Chung reassigned HIVE-23458: --- > Introduce unified thread pool for scheduled jobs > > > Key: HIVE-23458 > URL: https://issues.apache.org/jira/browse/HIVE-23458 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Reporter: Eugene Chung >Assignee: Eugene Chung >Priority: Major > > As I mentioned in [the comment of > HIVE-23164|https://issues.apache.org/jira/browse/HIVE-23164?focusedCommentId=17089506=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17089506], > I've made the unified scheduled executor service like > org.apache.hadoop.hive.metastore.ThreadPool. > I think it could help > 1. to minimize the possibility of making non-daemon threads when developers > need ScheduledExecutorService > 2. to achieve the utilization of server resources because the current > situation is all of the modules make its own ScheduledExecutorService and all > of the threads are just using for one job. > 3. an administrator of Hive server by providing > hive.exec.scheduler.num.threads configuration. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23449) LLAP: Reduce mkdir and config creations in submitWork hotpath
[
https://issues.apache.org/jira/browse/HIVE-23449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106023#comment-17106023
]
Hive QA commented on HIVE-23449:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 11m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
17s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
52s{color} | {color:blue} llap-server in master has 87 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
16s{color} | {color:red} llap-server: The patch generated 2 new + 154 unchanged
- 0 fixed = 156 total (was 154) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 16m 39s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22303/dev-support/hive-personality.sh
|
| git revision | master / 9ffbbdc |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22303/yetus/diff-checkstyle-llap-server.txt
|
| modules | C: llap-server U: llap-server |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22303/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> LLAP: Reduce mkdir and config creations in submitWork hotpath
> -
>
> Key: HIVE-23449
> URL: https://issues.apache.org/jira/browse/HIVE-23449
> Project: Hive
> Issue Type: Improvement
>Reporter: Rajesh Balamohan
>Assignee: Rajesh Balamohan
>Priority: Major
> Attachments: HIVE-23449.1.patch, HIVE-23449.2.patch,
> HIVE-23449.3.patch, Screenshot 2020-05-12 at 1.09.35 PM.png
>
>
> !Screenshot 2020-05-12 at 1.09.35 PM.png|width=885,height=558!
>
> For short jobs, submitWork gets into hotpath. This can lazy load conf and can
> get rid of dir creations (which needs to be enabled only when DirWatcher is
> enabled)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23449) LLAP: Reduce mkdir and config creations in submitWork hotpath
[
https://issues.apache.org/jira/browse/HIVE-23449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106059#comment-17106059
]
Hive QA commented on HIVE-23449:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13002779/HIVE-23449.3.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17268 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[special_character_in_tabnames_quotes_1]
(batchId=48)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22303/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22303/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22303/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13002779 - PreCommit-HIVE-Build
> LLAP: Reduce mkdir and config creations in submitWork hotpath
> -
>
> Key: HIVE-23449
> URL: https://issues.apache.org/jira/browse/HIVE-23449
> Project: Hive
> Issue Type: Improvement
>Reporter: Rajesh Balamohan
>Assignee: Rajesh Balamohan
>Priority: Major
> Attachments: HIVE-23449.1.patch, HIVE-23449.2.patch,
> HIVE-23449.3.patch, Screenshot 2020-05-12 at 1.09.35 PM.png
>
>
> !Screenshot 2020-05-12 at 1.09.35 PM.png|width=885,height=558!
>
> For short jobs, submitWork gets into hotpath. This can lazy load conf and can
> get rid of dir creations (which needs to be enabled only when DirWatcher is
> enabled)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23458) Introduce unified thread pool for scheduled jobs
[ https://issues.apache.org/jira/browse/HIVE-23458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Chung updated HIVE-23458: Labels: todoc4.0 (was: ) > Introduce unified thread pool for scheduled jobs > > > Key: HIVE-23458 > URL: https://issues.apache.org/jira/browse/HIVE-23458 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Reporter: Eugene Chung >Assignee: Eugene Chung >Priority: Major > Labels: todoc4.0 > Fix For: 4.0.0 > > Attachments: HIVE-23458.01.patch > > > As I mentioned in [the comment of > HIVE-23164|https://issues.apache.org/jira/browse/HIVE-23164?focusedCommentId=17089506=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17089506], > I've made the unified scheduled executor service like > org.apache.hadoop.hive.metastore.ThreadPool. > I think it could help > 1. to minimize the possibility of making non-daemon threads when developers > need ScheduledExecutorService > 2. to achieve the utilization of server resources because the current > situation is all of the modules make its own ScheduledExecutorService and all > of the threads are just using for one job. > 3. administrators of Hive servers by providing > hive.exec.scheduler.num.threads configuration so that they can predict and > set how many threads are used and needed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23444) Concurrent ACID direct inserts may fail with FileNotFoundException
[
https://issues.apache.org/jira/browse/HIVE-23444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106156#comment-17106156
]
Hive QA commented on HIVE-23444:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
25s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
51s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
26s{color} | {color:blue} ql in master has 1527 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
30s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
49s{color} | {color:red} ql: The patch generated 1 new + 167 unchanged - 0
fixed = 168 total (was 167) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 30m 19s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22306/dev-support/hive-personality.sh
|
| git revision | master / 9ffbbdc |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22306/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql itests U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22306/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Concurrent ACID direct inserts may fail with FileNotFoundException
> --
>
> Key: HIVE-23444
> URL: https://issues.apache.org/jira/browse/HIVE-23444
> Project: Hive
> Issue Type: Bug
>Reporter: Marta Kuczora
>Assignee: Marta Kuczora
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-23444.1.patch, HIVE-23444.1.patch
>
>
> The following exception may occur when concurrently inserting into an ACID
> table with static partitions and the 'hive.acid.direct.insert.enabled'
> parameter is true. This issue occurs intermittently.
> {noformat}
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> java.io.IOException: java.io.FileNotFoundException: File
> hdfs://ns1/warehouse/tablespace/managed/hive/tpch_unbucketed.db/concurrent_insert_partitioned/l_tax=0.0/_tmp.delta_001_001_
> does not exist.
> at
> org.apache.hadoop.hive.ql.metadata.Hive.loadPartitionInternal(Hive.java:2465)
> ~[hive-exec-3.1.3000.7.1.1.0-493.jar:3.1.3000.7.1.1.0-493]
> at
> org.apache.hadoop.hive.ql.metadata.Hive.loadPartition(Hive.java:2228)
>
[jira] [Comment Edited] (HIVE-23133) Numeric operations can have different result across hardware archs
[ https://issues.apache.org/jira/browse/HIVE-23133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106042#comment-17106042 ] Zhenyu Zheng edited comment on HIVE-23133 at 5/13/20, 10:03 AM: [~gopalv] Thanks for the review, in previous patches (1-3) we didn't add config, if that is the better way, we can keep it that way. The modified UDFs includes: UDFCos.java UDFExp.java UDFLn.java UDFLog.java UDFLog2.java UDFLog10.java I will also add the funcs that we changed to the ticket description. was (Author: kevin_zheng): [~gopalv] Thanks for the review, in previous patches (1-3) we didn't add config, if that is the better way, we can keep it that way. The modified UDFs includes: I will also add the funcs that we changed to the ticket description. > Numeric operations can have different result across hardware archs > -- > > Key: HIVE-23133 > URL: https://issues.apache.org/jira/browse/HIVE-23133 > Project: Hive > Issue Type: Sub-task >Reporter: Zhenyu Zheng >Assignee: Zhenyu Zheng >Priority: Major > Attachments: HIVE-23133.1.patch, HIVE-23133.2.patch, > HIVE-23133.3.patch, HIVE-23133.4.patch, HIVE-23133.5.patch, HIVE-23133.6.patch > > > Use java.lang.StrictMath rather than java.lang.Math for compatibility in the > following UDFs: > UDFCos.java > UDFExp.java > UDFLn.java > UDFLog.java > UDFLog2.java > UDFLog10.java > > Currently, we have set up an ARM CI to test out how Hive works on ARM > platform: > [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/] > Among the failures, we have observed that some numeric operations can have > different result across hardware archs, such as: > [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/25/testReport/org.apache.hadoop.hive.cli/TestCliDriver/testCliDriver_vector_decimal_udf2_/] > [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/25/testReport/org.apache.hadoop.hive.cli/TestSparkCliDriver/testCliDriver_subquery_select_/] > [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/25/testReport/org.apache.hadoop.hive.cli/TestSparkCliDriver/testCliDriver_vectorized_math_funcs_/] > we can see that the calculation results of log, exp, cos, toRadians etc is > slitly different than the .out file results that we are > comparing(they are tested and wrote on X86 machines), this is because of we > use [Math > Library|https://docs.oracle.com/javase/6/docs/api/java/lang/Math.html] for > these kind of calculations. > and according to the > [illustration|https://docs.oracle.com/javase/6/docs/api/java/lang/Math.html]: > _Unlike some of the numeric methods of class StrictMath, all implementations > of the equivalent functions of class Math are not_ > _defined to return the bit-for-bit same results. This relaxation permits > better-performing implementations where strict reproducibility_ > _is not required._ > _By default many of the Math methods simply call the equivalent method in > StrictMath for their implementation._ > _Code generators are encouraged to use platform-specific native libraries or > microprocessor instructions, where available,_ > _to provide higher-performance implementations of Math methods._ > so the result will have difference across hardware archs. > On the other hand, JAVA provided another library > [StrictMath|https://docs.oracle.com/javase/6/docs/api/java/lang/StrictMath.html] > that will not have this kind of problem as according to its' > [reference|https://docs.oracle.com/javase/6/docs/api/java/lang/StrictMath.html]: > _To help ensure portability of Java programs, the definitions of some of the > numeric functions in this package require that they produce_ > _the same results as certain published algorithms._ > So in order to fix the above mentioned problem, we have to consider switch to > use StrictMath instead of Math. > > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23444) Concurrent ACID direct inserts may fail with FileNotFoundException
[
https://issues.apache.org/jira/browse/HIVE-23444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106176#comment-17106176
]
Hive QA commented on HIVE-23444:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13002794/HIVE-23444.1.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 17269 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniDruidKafkaCliDriver.testCliDriver[druidkafkamini_delimited]
(batchId=225)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[special_character_in_tabnames_1]
(batchId=78)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22306/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22306/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22306/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13002794 - PreCommit-HIVE-Build
> Concurrent ACID direct inserts may fail with FileNotFoundException
> --
>
> Key: HIVE-23444
> URL: https://issues.apache.org/jira/browse/HIVE-23444
> Project: Hive
> Issue Type: Bug
>Reporter: Marta Kuczora
>Assignee: Marta Kuczora
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-23444.1.patch, HIVE-23444.1.patch
>
>
> The following exception may occur when concurrently inserting into an ACID
> table with static partitions and the 'hive.acid.direct.insert.enabled'
> parameter is true. This issue occurs intermittently.
> {noformat}
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> java.io.IOException: java.io.FileNotFoundException: File
> hdfs://ns1/warehouse/tablespace/managed/hive/tpch_unbucketed.db/concurrent_insert_partitioned/l_tax=0.0/_tmp.delta_001_001_
> does not exist.
> at
> org.apache.hadoop.hive.ql.metadata.Hive.loadPartitionInternal(Hive.java:2465)
> ~[hive-exec-3.1.3000.7.1.1.0-493.jar:3.1.3000.7.1.1.0-493]
> at
> org.apache.hadoop.hive.ql.metadata.Hive.loadPartition(Hive.java:2228)
> ~[hive-exec-3.1.3000.7.1.1.0-493.jar:3.1.3000.7.1.1.0-493]
> at
> org.apache.hadoop.hive.ql.exec.MoveTask.handleStaticParts(MoveTask.java:522)
> ~[hive-exec-3.1.3000.7.1.1.0-493.jar:3.1.3000.7.1.1.0-493]
> at org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:442)
> ~[hive-exec-3.1.3000.7.1.1.0-493.jar:3.1.3000.7.1.1.0-493]
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:213)
> ~[hive-exec-3.1.3000.7.1.1.0-493.jar:3.1.3000.7.1.1.0-493]
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:105)
> ~[hive-exec-3.1.3000.7.1.1.0-493.jar:3.1.3000.7.1.1.0-493]
> at org.apache.hadoop.hive.ql.Executor.launchTask(Executor.java:359)
> ~[hive-exec-3.1.3000.7.1.1.0-493.jar:3.1.3000.7.1.1.0-493]
> at org.apache.hadoop.hive.ql.Executor.launchTasks(Executor.java:330)
> ~[hive-exec-3.1.3000.7.1.1.0-493.jar:3.1.3000.7.1.1.0-493]
> at org.apache.hadoop.hive.ql.Executor.runTasks(Executor.java:246)
> ~[hive-exec-3.1.3000.7.1.1.0-493.jar:3.1.3000.7.1.1.0-493]
> at org.apache.hadoop.hive.ql.Executor.execute(Executor.java:109)
> ~[hive-exec-3.1.3000.7.1.1.0-493.jar:3.1.3000.7.1.1.0-493]
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:721)
> ~[hive-exec-3.1.3000.7.1.1.0-493.jar:3.1.3000.7.1.1.0-493]
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:488)
> ~[hive-exec-3.1.3000.7.1.1.0-493.jar:3.1.3000.7.1.1.0-493]
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:482)
> ~[hive-exec-3.1.3000.7.1.1.0-493.jar:3.1.3000.7.1.1.0-493]
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:166)
> ~[hive-exec-3.1.3000.7.1.1.0-493.jar:3.1.3000.7.1.1.0-493]
> at
> org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:225)
> ~[hive-service-3.1.3000.7.1.1.0-493.jar:3.1.3000.7.1.1.0-493]
> ... 13 more
> Caused by: java.io.IOException: java.io.FileNotFoundException: File
> hdfs://ns1/warehouse/tablespace/managed/hive/tpch_unbucketed.db/concurrent_insert_partitioned/l_tax=0.0/_tmp.delta_001_001_
> does not exist.
> at
> org.apache.hadoop.hive.ql.io.AcidUtils.getHdfsDirSnapshots(AcidUtils.java:1472)
>
[jira] [Comment Edited] (HIVE-23133) Numeric operations can have different result across hardware archs
[ https://issues.apache.org/jira/browse/HIVE-23133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106042#comment-17106042 ] Zhenyu Zheng edited comment on HIVE-23133 at 5/13/20, 10:03 AM: [~gopalv] Thanks for the review, in previous patches (1-3) we didn't add config, if that is the better way, we can keep it that way. The modified UDFs includes: I will also add the funcs that we changed to the ticket description. was (Author: kevin_zheng): [~gopalv] Thanks for the review, in previous patches (1-3) we didn't add config, if that is the better way, we can keep it that way. I will add the funcs that we changed to the ticket description. > Numeric operations can have different result across hardware archs > -- > > Key: HIVE-23133 > URL: https://issues.apache.org/jira/browse/HIVE-23133 > Project: Hive > Issue Type: Sub-task >Reporter: Zhenyu Zheng >Assignee: Zhenyu Zheng >Priority: Major > Attachments: HIVE-23133.1.patch, HIVE-23133.2.patch, > HIVE-23133.3.patch, HIVE-23133.4.patch, HIVE-23133.5.patch, HIVE-23133.6.patch > > > Use java.lang.StrictMath rather than java.lang.Math for compatibility in the > following UDFs: > UDFCos.java > UDFExp.java > UDFLn.java > UDFLog.java > UDFLog2.java > UDFLog10.java > > Currently, we have set up an ARM CI to test out how Hive works on ARM > platform: > [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/] > Among the failures, we have observed that some numeric operations can have > different result across hardware archs, such as: > [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/25/testReport/org.apache.hadoop.hive.cli/TestCliDriver/testCliDriver_vector_decimal_udf2_/] > [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/25/testReport/org.apache.hadoop.hive.cli/TestSparkCliDriver/testCliDriver_subquery_select_/] > [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/25/testReport/org.apache.hadoop.hive.cli/TestSparkCliDriver/testCliDriver_vectorized_math_funcs_/] > we can see that the calculation results of log, exp, cos, toRadians etc is > slitly different than the .out file results that we are > comparing(they are tested and wrote on X86 machines), this is because of we > use [Math > Library|https://docs.oracle.com/javase/6/docs/api/java/lang/Math.html] for > these kind of calculations. > and according to the > [illustration|https://docs.oracle.com/javase/6/docs/api/java/lang/Math.html]: > _Unlike some of the numeric methods of class StrictMath, all implementations > of the equivalent functions of class Math are not_ > _defined to return the bit-for-bit same results. This relaxation permits > better-performing implementations where strict reproducibility_ > _is not required._ > _By default many of the Math methods simply call the equivalent method in > StrictMath for their implementation._ > _Code generators are encouraged to use platform-specific native libraries or > microprocessor instructions, where available,_ > _to provide higher-performance implementations of Math methods._ > so the result will have difference across hardware archs. > On the other hand, JAVA provided another library > [StrictMath|https://docs.oracle.com/javase/6/docs/api/java/lang/StrictMath.html] > that will not have this kind of problem as according to its' > [reference|https://docs.oracle.com/javase/6/docs/api/java/lang/StrictMath.html]: > _To help ensure portability of Java programs, the definitions of some of the > numeric functions in this package require that they produce_ > _the same results as certain published algorithms._ > So in order to fix the above mentioned problem, we have to consider switch to > use StrictMath instead of Math. > > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-23365) Put RS deduplication optimization under cost based decision
[ https://issues.apache.org/jira/browse/HIVE-23365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stamatis Zampetakis reassigned HIVE-23365: -- Assignee: Stamatis Zampetakis > Put RS deduplication optimization under cost based decision > --- > > Key: HIVE-23365 > URL: https://issues.apache.org/jira/browse/HIVE-23365 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Reporter: Jesus Camacho Rodriguez >Assignee: Stamatis Zampetakis >Priority: Major > > Currently, RS deduplication is always executed whenever it is semantically > correct. However, it could be beneficial to leave both RS operators in the > plan, e.g., if the NDV of the second RS is very low. Thus, we would like this > decision to be cost-based. We could use a simple heuristic that would work > fine for most of the cases without introducing regressions for existing > cases, e.g., if NDV for partition column is less than estimated parallelism > in the second RS, do not execute deduplication. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23133) Numeric operations can have different result across hardware archs
[
https://issues.apache.org/jira/browse/HIVE-23133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106197#comment-17106197
]
Hive QA commented on HIVE-23133:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
25s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
35s{color} | {color:blue} ql in master has 1527 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
16s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
31s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
15s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 31m 58s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22307/dev-support/hive-personality.sh
|
| git revision | master / 9ffbbdc |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: vector-code-gen ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22307/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Numeric operations can have different result across hardware archs
> --
>
> Key: HIVE-23133
> URL: https://issues.apache.org/jira/browse/HIVE-23133
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zhenyu Zheng
>Assignee: Zhenyu Zheng
>Priority: Major
> Attachments: HIVE-23133.1.patch, HIVE-23133.2.patch,
> HIVE-23133.3.patch, HIVE-23133.4.patch, HIVE-23133.5.patch, HIVE-23133.6.patch
>
>
> Use java.lang.StrictMath rather than java.lang.Math for compatibility in the
> following UDFs:
> UDFCos.java
> UDFExp.java
> UDFLn.java
> UDFLog.java
> UDFLog2.java
> UDFLog10.java
>
> Currently, we have set up an ARM CI to test out how Hive works on ARM
> platform:
> [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/]
> Among the failures, we have observed that some numeric operations can have
> different result across hardware archs, such as:
> [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/25/testReport/org.apache.hadoop.hive.cli/TestCliDriver/testCliDriver_vector_decimal_udf2_/]
> [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/25/testReport/org.apache.hadoop.hive.cli/TestSparkCliDriver/testCliDriver_subquery_select_/]
>
[jira] [Commented] (HIVE-23458) Introduce unified thread pool for scheduled jobs
[
https://issues.apache.org/jira/browse/HIVE-23458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106216#comment-17106216
]
Hive QA commented on HIVE-23458:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13002800/HIVE-23458.01.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22308/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22308/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22308/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2020-05-13 11:15:13.255
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-22308/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2020-05-13 11:15:13.258
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 9ffbbdc HIVE-23409 : If TezSession application reopen fails for
Timeline service down, default TezSession from SessionPool is closed after a
retry ( Naresh PR via Ashutosh Chauhan)
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 9ffbbdc HIVE-23409 : If TezSession application reopen fails for
Timeline service down, default TezSession from SessionPool is closed after a
retry ( Naresh PR via Ashutosh Chauhan)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2020-05-13 11:15:14.191
+ rm -rf ../yetus_PreCommit-HIVE-Build-22308
+ mkdir ../yetus_PreCommit-HIVE-Build-22308
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-22308
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-22308/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
Trying to apply the patch with -p0
Going to apply patch with: git apply -p0
+ [[ maven == \m\a\v\e\n ]]
+ rm -rf /data/hiveptest/working/maven/org/apache/hive
+ mvn -B clean install -DskipTests -T 4 -q
-Dmaven.repo.local=/data/hiveptest/working/maven
protoc-jar: executing: [/tmp/protoc6207579259697332681.exe, --version]
libprotoc 2.6.1
protoc-jar: executing: [/tmp/protoc6207579259697332681.exe,
-I/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/src/main/protobuf/org/apache/hadoop/hive/metastore,
--java_out=/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/target/generated-sources,
/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/src/main/protobuf/org/apache/hadoop/hive/metastore/metastore.proto]
ANTLR Parser Generator Version 3.5.2
protoc-jar: executing: [/tmp/protoc186711488808264886.exe, --version]
libprotoc 2.6.1
ANTLR Parser Generator Version 3.5.2
Output file
/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-server/target/generated-sources/org/apache/hadoop/hive/metastore/parser/FilterParser.java
does not exist: must build
/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/parser/Filter.g
org/apache/hadoop/hive/metastore/parser/Filter.g
ANTLR Parser Generator Version 3.5.2
Output file
/data/hiveptest/working/apache-github-source-source/parser/target/generated-sources/antlr3/org/apache/hadoop/hive/ql/parse/HiveLexer.java
does not exist: must build
/data/hiveptest/working/apache-github-source-source/parser/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g
org/apache/hadoop/hive/ql/parse/HiveLexer.g
Output file
[jira] [Updated] (HIVE-23434) Add option to rewrite PERCENTILE_CONT to sketch functions
[ https://issues.apache.org/jira/browse/HIVE-23434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-23434: Status: Patch Available (was: Open) > Add option to rewrite PERCENTILE_CONT to sketch functions > - > > Key: HIVE-23434 > URL: https://issues.apache.org/jira/browse/HIVE-23434 > Project: Hive > Issue Type: Sub-task >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-23434.01.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23434) Add option to rewrite PERCENTILE_CONT to sketch functions
[ https://issues.apache.org/jira/browse/HIVE-23434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-23434: Attachment: HIVE-23434.01.patch > Add option to rewrite PERCENTILE_CONT to sketch functions > - > > Key: HIVE-23434 > URL: https://issues.apache.org/jira/browse/HIVE-23434 > Project: Hive > Issue Type: Sub-task >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-23434.01.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23460) Add qoption to disable qtests
[
https://issues.apache.org/jira/browse/HIVE-23460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106241#comment-17106241
]
Hive QA commented on HIVE-23460:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
25s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
39s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
18s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
27s{color} | {color:blue} ql in master has 1527 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
38s{color} | {color:blue} accumulo-handler in master has 20 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
0s{color} | {color:blue} itests/util in master has 54 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
47s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
33s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
46s{color} | {color:green} The patch ql passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} The patch accumulo-handler passed checkstyle {color}
|
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s{color} | {color:green} itests/util: The patch generated 0 new + 59
unchanged - 6 fixed = 59 total (was 65) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
44s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 40m 1s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22309/dev-support/hive-personality.sh
|
| git revision | master / 9ffbbdc |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql accumulo-handler itests itests/util U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22309/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Add qoption to disable qtests
> -
>
> Key: HIVE-23460
> URL: https://issues.apache.org/jira/browse/HIVE-23460
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-23460.01.patch
>
>
> instead other ways to exclude them... (testconfiguration.properties;
> CliConfig#excludeQuery)
> {code}
> --! qt:disabled:reason
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23460) Add qoption to disable qtests
[
https://issues.apache.org/jira/browse/HIVE-23460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106259#comment-17106259
]
Hive QA commented on HIVE-23460:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13002807/HIVE-23460.01.patch
{color:green}SUCCESS:{color} +1 due to 38 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 15 failed/errored test(s), 17274 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_joins]
(batchId=222)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_subq_in]
(batchId=3)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_uri_import]
(batchId=17)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[cbo_query44]
(batchId=228)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[cbo_query45]
(batchId=228)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[cbo_query67]
(batchId=228)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[cbo_query70]
(batchId=228)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[cbo_query86]
(batchId=228)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[cbo_query44]
(batchId=227)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[cbo_query45]
(batchId=227)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[cbo_query67]
(batchId=227)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[cbo_query70]
(batchId=227)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[cbo_query86]
(batchId=227)
org.apache.hadoop.hive.ql.parse.TestReplAcidTablesBootstrapWithJsonMessage.org.apache.hadoop.hive.ql.parse.TestReplAcidTablesBootstrapWithJsonMessage
(batchId=182)
org.apache.hive.beeline.TestBeeLineWithArgs.testRowsAffected (batchId=208)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22309/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22309/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22309/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 15 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13002807 - PreCommit-HIVE-Build
> Add qoption to disable qtests
> -
>
> Key: HIVE-23460
> URL: https://issues.apache.org/jira/browse/HIVE-23460
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-23460.01.patch
>
>
> instead other ways to exclude them... (testconfiguration.properties;
> CliConfig#excludeQuery)
> {code}
> --! qt:disabled:reason
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23434) Add option to rewrite PERCENTILE_CONT to sketch functions
[ https://issues.apache.org/jira/browse/HIVE-23434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106266#comment-17106266 ] Zoltan Haindrich commented on HIVE-23434: - opened an [rb|https://reviews.apache.org/r/72506/] in the hope that it will be able to track the file rename - apparently it's not... there is also a [PR|https://github.com/apache/hive/pull/1017] > Add option to rewrite PERCENTILE_CONT to sketch functions > - > > Key: HIVE-23434 > URL: https://issues.apache.org/jira/browse/HIVE-23434 > Project: Hive > Issue Type: Sub-task >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-23434.01.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23458) Introduce unified thread pool for scheduled jobs
[
https://issues.apache.org/jira/browse/HIVE-23458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106492#comment-17106492
]
Hive QA commented on HIVE-23458:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
27s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
22s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
42s{color} | {color:blue} common in master has 63 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
31s{color} | {color:blue} ql in master has 1527 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
50s{color} | {color:blue} service in master has 51 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
44s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
32s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
2s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
50s{color} | {color:red} ql: The patch generated 4 new + 32 unchanged - 2 fixed
= 36 total (was 34) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
46s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
18s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 39m 24s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22314/dev-support/hive-personality.sh
|
| git revision | master / 472aca8 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22314/yetus/diff-checkstyle-ql.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22314/yetus/patch-asflicense-problems.txt
|
| modules | C: common ql service U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22314/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Introduce unified thread pool for scheduled jobs
>
>
> Key: HIVE-23458
> URL: https://issues.apache.org/jira/browse/HIVE-23458
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Reporter: Eugene Chung
>Assignee: Eugene Chung
>Priority: Major
> Labels: todoc4.0
> Fix For: 4.0.0
>
> Attachments: HIVE-23458.01.patch
>
>
> As I mentioned in [the comment of
> HIVE-23164|https://issues.apache.org/jira/browse/HIVE-23164?focusedCommentId=17089506=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17089506],
> I've made the unified scheduled executor service like
> org.apache.hadoop.hive.metastore.ThreadPool.
> I think
[jira] [Commented] (HIVE-23432) Add Ranger Replication Metrics
[
https://issues.apache.org/jira/browse/HIVE-23432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106615#comment-17106615
]
Hive QA commented on HIVE-23432:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
50s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
11s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
28s{color} | {color:blue} ql in master has 1527 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
44s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
30s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
34s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
3s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
49s{color} | {color:red} ql: The patch generated 4 new + 3 unchanged - 6 fixed
= 7 total (was 9) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 37m 17s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile
xml |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22316/dev-support/hive-personality.sh
|
| git revision | master / 472aca8 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22316/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22316/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Add Ranger Replication Metrics
> ---
>
> Key: HIVE-23432
> URL: https://issues.apache.org/jira/browse/HIVE-23432
> Project: Hive
> Issue Type: Task
>Reporter: Aasha Medhi
>Assignee: Aasha Medhi
>Priority: Major
> Attachments: HIVE-23432.01.patch, HIVE-23432.02.patch,
> HIVE-23432.03.patch, HIVE-23432.04.patch, HIVE-23432.05.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22066) Upgrade Apache parent POM to version 21
[ https://issues.apache.org/jira/browse/HIVE-22066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106616#comment-17106616 ] Ashutosh Chauhan commented on HIVE-22066: - [~belugabehr] you want to refresh the patch and create RB for it? > Upgrade Apache parent POM to version 21 > --- > > Key: HIVE-22066 > URL: https://issues.apache.org/jira/browse/HIVE-22066 > Project: Hive > Issue Type: Improvement >Reporter: Jesus Camacho Rodriguez >Assignee: David Mollitor >Priority: Major > Attachments: HIVE-22066.2.patch, HIVE-22066.3.patch, > HIVE-22066.4.patch, HIVE-22066.5.patch, HIVE-22066.6.patch, > HIVE-22066.6.patch, HIVE-22066.7.patch, HIVE-22066.8.patch, > HIVE-22066.8.patch, HIVE-22066.9.patch, HIVE-22066.999.patch, > HIVE-22066.patch, HIVE-22066.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23375) Track MJ HashTable Load time
[
https://issues.apache.org/jira/browse/HIVE-23375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106434#comment-17106434
]
Hive QA commented on HIVE-23375:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13002167/HIVE-23375.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17268 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.metastore.TestPartitionManagement.testPartitionDiscoveryTransactionalTable
(batchId=152)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22313/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22313/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22313/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13002167 - PreCommit-HIVE-Build
> Track MJ HashTable Load time
>
>
> Key: HIVE-23375
> URL: https://issues.apache.org/jira/browse/HIVE-23375
> Project: Hive
> Issue Type: Improvement
>Reporter: Panagiotis Garefalakis
>Assignee: Panagiotis Garefalakis
>Priority: Minor
> Labels: pull-request-available
> Attachments: HIVE-23375.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Introduce TezCounter to track MJ HashTable Load time
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23458) Introduce unified thread pool for scheduled jobs
[
https://issues.apache.org/jira/browse/HIVE-23458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106509#comment-17106509
]
Hive QA commented on HIVE-23458:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13002823/HIVE-23458.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17268 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22314/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22314/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22314/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13002823 - PreCommit-HIVE-Build
> Introduce unified thread pool for scheduled jobs
>
>
> Key: HIVE-23458
> URL: https://issues.apache.org/jira/browse/HIVE-23458
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Reporter: Eugene Chung
>Assignee: Eugene Chung
>Priority: Major
> Labels: todoc4.0
> Fix For: 4.0.0
>
> Attachments: HIVE-23458.01.patch
>
>
> As I mentioned in [the comment of
> HIVE-23164|https://issues.apache.org/jira/browse/HIVE-23164?focusedCommentId=17089506=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17089506],
> I've made the unified scheduled executor service like
> org.apache.hadoop.hive.metastore.ThreadPool.
> I think it could help
> 1. to minimize the possibility of making non-daemon threads when developers
> need ScheduledExecutorService
> 2. to achieve the utilization of server resources because the current
> situation is all of the modules make its own ScheduledExecutorService and all
> of the threads are just using for one job.
> 3. administrators of Hive servers by providing
> hive.exec.scheduler.num.threads configuration so that they can predict and
> set how many threads are used and needed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23434) Add option to rewrite PERCENTILE_CONT to sketch functions
[ https://issues.apache.org/jira/browse/HIVE-23434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-23434: Attachment: HIVE-23434.03.patch > Add option to rewrite PERCENTILE_CONT to sketch functions > - > > Key: HIVE-23434 > URL: https://issues.apache.org/jira/browse/HIVE-23434 > Project: Hive > Issue Type: Sub-task >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-23434.01.patch, HIVE-23434.02.patch, > HIVE-23434.03.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23432) Add Ranger Replication Metrics
[
https://issues.apache.org/jira/browse/HIVE-23432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106635#comment-17106635
]
Hive QA commented on HIVE-23432:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13002831/HIVE-23432.05.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 17270 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.exec.repl.TestRangerDumpTask.testSuccessNonEmptyRangerPolicies
(batchId=291)
org.apache.hadoop.hive.ql.exec.repl.TestRangerDumpTask.testSuccessRangerDumpMetrics
(batchId=291)
org.apache.hadoop.hive.ql.exec.repl.TestRangerDumpTask.testSuccessValidAuthProviderEndpoint
(batchId=291)
org.apache.hadoop.hive.ql.exec.repl.TestRangerLoadTask.testSuccessNonEmptyRangerPolicies
(batchId=292)
org.apache.hadoop.hive.ql.exec.repl.TestRangerLoadTask.testSuccessRangerDumpMetrics
(batchId=292)
org.apache.hadoop.hive.ql.parse.TestReplAcrossInstancesWithJsonMessageFormat.testRangerReplication
(batchId=191)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcrossInstances.testRangerReplication
(batchId=195)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22316/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22316/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22316/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13002831 - PreCommit-HIVE-Build
> Add Ranger Replication Metrics
> ---
>
> Key: HIVE-23432
> URL: https://issues.apache.org/jira/browse/HIVE-23432
> Project: Hive
> Issue Type: Task
>Reporter: Aasha Medhi
>Assignee: Aasha Medhi
>Priority: Major
> Attachments: HIVE-23432.01.patch, HIVE-23432.02.patch,
> HIVE-23432.03.patch, HIVE-23432.04.patch, HIVE-23432.05.patch,
> HIVE-23432.06.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-23466) ZK registry base should remove only specific instance instead of host
[ https://issues.apache.org/jira/browse/HIVE-23466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran reassigned HIVE-23466: > ZK registry base should remove only specific instance instead of host > - > > Key: HIVE-23466 > URL: https://issues.apache.org/jira/browse/HIVE-23466 > Project: Hive > Issue Type: Bug >Affects Versions: 4.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > > When ZKRegistryBase detects new ZK nodes it maintains path based cache and > host based cache. The host based cached already handles multiple instances > running in same host. But even if single instance is removed all instances > belonging to the host are removed. > Another issue is that, if single host has multiple instances it returns a Set > with no ordering. Ideally, we want the newest instance to be top of the set > (use TreeSet maybe?). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23375) Track MJ HashTable Load time
[ https://issues.apache.org/jira/browse/HIVE-23375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Panagiotis Garefalakis updated HIVE-23375: -- Attachment: HIVE-23375.02.patch > Track MJ HashTable Load time > > > Key: HIVE-23375 > URL: https://issues.apache.org/jira/browse/HIVE-23375 > Project: Hive > Issue Type: Improvement >Reporter: Panagiotis Garefalakis >Assignee: Panagiotis Garefalakis >Priority: Minor > Labels: pull-request-available > Attachments: HIVE-23375.01.patch, HIVE-23375.02.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Introduce TezCounter to track MJ HashTable Load time -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23434) Add option to rewrite PERCENTILE_CONT to sketch functions
[
https://issues.apache.org/jira/browse/HIVE-23434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106570#comment-17106570
]
Hive QA commented on HIVE-23434:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13002826/HIVE-23434.02.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 17265 tests
executed
*Failed tests:*
{noformat}
TestStatsReplicationScenariosACID - did not produce a TEST-*.xml file (likely
timed out) (batchId=184)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sketches_materialized_view_rollup2]
(batchId=45)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sketches_materialized_view_safety]
(batchId=81)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22315/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22315/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22315/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13002826 - PreCommit-HIVE-Build
> Add option to rewrite PERCENTILE_CONT to sketch functions
> -
>
> Key: HIVE-23434
> URL: https://issues.apache.org/jira/browse/HIVE-23434
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-23434.01.patch, HIVE-23434.02.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23099) Improve Logger for Operation Child Classes
[
https://issues.apache.org/jira/browse/HIVE-23099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-23099:
--
Attachment: HIVE-23099.5.patch
> Improve Logger for Operation Child Classes
> --
>
> Key: HIVE-23099
> URL: https://issues.apache.org/jira/browse/HIVE-23099
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HIVE-23099.1.patch, HIVE-23099.2.patch,
> HIVE-23099.3.patch, HIVE-23099.4.patch, HIVE-23099.4.patch,
> HIVE-23099.5.patch, HIVE-23099.5.patch, HIVE-23099.5.patch
>
>
> The {{Operation}} class declares its logger this way:
> {code:java|title=Operation.java}
> public abstract class Operation {
> public static final Logger LOG =
> LoggerFactory.getLogger(Operation.class.getName());
> ...
> }
> {code}
> Notice that this is an {{abstract}} class, but the {{Logger}} is tied to the
> {{Operation.class.getName()}}. This means that logging cannot be controlled
> for each subclass of {{Operation}} independently since they all use the same
> static {{Logger}} instance.
> Make the LOG a {{protected}} instance variable that inherits the name of the
> child class.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22066) Upgrade Apache parent POM to version 21
[
https://issues.apache.org/jira/browse/HIVE-22066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106636#comment-17106636
]
David Mollitor commented on HIVE-22066:
---
[~ashutoshc] Sure.
{quote}
The first rule of moving to Java 11 is you do not talk … to update all the
things. Your IDE, your build tool, its plugins, and, most importantly, your
dependencies. You don’t have to do all of these updates in advance, but if you
can, you absolutely should – it will very likely get you past some hurdles you
can then stay blissfully unaware of.
Maven: 3.5.0
compiler plugin: 3.8.0
surefire and failsafe: 2.22.0
* https://blog.codefx.org/java/java-11-migration-guide/
{quote}
Those three items are all outdated in current Hive POM and covered in the
latest Apache ASF Parent POM.
> Upgrade Apache parent POM to version 21
> ---
>
> Key: HIVE-22066
> URL: https://issues.apache.org/jira/browse/HIVE-22066
> Project: Hive
> Issue Type: Improvement
>Reporter: Jesus Camacho Rodriguez
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-22066.2.patch, HIVE-22066.3.patch,
> HIVE-22066.4.patch, HIVE-22066.5.patch, HIVE-22066.6.patch,
> HIVE-22066.6.patch, HIVE-22066.7.patch, HIVE-22066.8.patch,
> HIVE-22066.8.patch, HIVE-22066.9.patch, HIVE-22066.999.patch,
> HIVE-22066.patch, HIVE-22066.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23434) Add option to rewrite PERCENTILE_CONT to sketch functions
[
https://issues.apache.org/jira/browse/HIVE-23434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106590#comment-17106590
]
Hive QA commented on HIVE-23434:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
22s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
30s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
40s{color} | {color:blue} common in master has 63 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
27s{color} | {color:blue} ql in master has 1527 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 9m
35s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
29s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 11m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 9m
24s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
49s{color} | {color:red} ql: The patch generated 7 new + 107 unchanged - 0
fixed = 114 total (was 107) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m
43s{color} | {color:red} root: The patch generated 7 new + 484 unchanged - 0
fixed = 491 total (was 484) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
1s{color} | {color:red} The patch has 2 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
49s{color} | {color:red} patch/common cannot run setBugDatabaseInfo from
findbugs {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 8m
13s{color} | {color:red} patch/ql cannot run setBugDatabaseInfo from findbugs
{color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 6m
34s{color} | {color:red} root in the patch failed. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
37s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 84m 16s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22315/dev-support/hive-personality.sh
|
| git revision | master / 472aca8 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22315/yetus/diff-checkstyle-ql.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22315/yetus/diff-checkstyle-root.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22315/yetus/whitespace-eol.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22315/yetus/patch-findbugs-common.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22315/yetus/patch-findbugs-ql.txt
|
| javadoc |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22315/yetus/patch-javadoc-root.txt
|
| modules | C: common ql . itests U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22315/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Add option to rewrite PERCENTILE_CONT to sketch functions
>
[jira] [Updated] (HIVE-23432) Add Ranger Replication Metrics
[ https://issues.apache.org/jira/browse/HIVE-23432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-23432: --- Status: In Progress (was: Patch Available) > Add Ranger Replication Metrics > --- > > Key: HIVE-23432 > URL: https://issues.apache.org/jira/browse/HIVE-23432 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Attachments: HIVE-23432.01.patch, HIVE-23432.02.patch, > HIVE-23432.03.patch, HIVE-23432.04.patch, HIVE-23432.05.patch, > HIVE-23432.06.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23432) Add Ranger Replication Metrics
[ https://issues.apache.org/jira/browse/HIVE-23432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-23432: --- Attachment: HIVE-23432.06.patch Status: Patch Available (was: In Progress) > Add Ranger Replication Metrics > --- > > Key: HIVE-23432 > URL: https://issues.apache.org/jira/browse/HIVE-23432 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Attachments: HIVE-23432.01.patch, HIVE-23432.02.patch, > HIVE-23432.03.patch, HIVE-23432.04.patch, HIVE-23432.05.patch, > HIVE-23432.06.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23463) Investigate why the results have changed at udf_mask_show_last_n.q, udf_mask_show_first_n.q, udf_mask_last_n.q, udf_mask_first_n.q, udf_mask.q
[ https://issues.apache.org/jira/browse/HIVE-23463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-23463: -- Description: Check [https://reviews.apache.org/r/72491/] for details. > Investigate why the results have changed at udf_mask_show_last_n.q, > udf_mask_show_first_n.q, udf_mask_last_n.q, udf_mask_first_n.q, udf_mask.q > -- > > Key: HIVE-23463 > URL: https://issues.apache.org/jira/browse/HIVE-23463 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Priority: Major > > Check [https://reviews.apache.org/r/72491/] for details. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23432) Add Ranger Replication Metrics
[ https://issues.apache.org/jira/browse/HIVE-23432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106479#comment-17106479 ] PRAVIN KUMAR SINHA commented on HIVE-23432: --- +1 > Add Ranger Replication Metrics > --- > > Key: HIVE-23432 > URL: https://issues.apache.org/jira/browse/HIVE-23432 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Attachments: HIVE-23432.01.patch, HIVE-23432.02.patch, > HIVE-23432.03.patch, HIVE-23432.04.patch, HIVE-23432.05.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23451) FileSinkOperator calls deleteOnExit (hdfs call) twice for the same file
[
https://issues.apache.org/jira/browse/HIVE-23451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106707#comment-17106707
]
Hive QA commented on HIVE-23451:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13002836/HIVE-23451.3.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17268 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22318/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22318/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22318/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13002836 - PreCommit-HIVE-Build
> FileSinkOperator calls deleteOnExit (hdfs call) twice for the same file
> ---
>
> Key: HIVE-23451
> URL: https://issues.apache.org/jira/browse/HIVE-23451
> Project: Hive
> Issue Type: Improvement
>Reporter: Rajesh Balamohan
>Assignee: Rajesh Balamohan
>Priority: Minor
> Attachments: HIVE-23451.1.patch, HIVE-23451.2.patch,
> HIVE-23451.3.patch
>
>
> [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java#L826]
> [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java#L797]
> Can avoid a NN call here (i.e, mainly for small queries).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-23467) Add a skip.trash config for HMS to skip trash when deleting external table data
[ https://issues.apache.org/jira/browse/HIVE-23467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam An reassigned HIVE-23467: - > Add a skip.trash config for HMS to skip trash when deleting external table > data > --- > > Key: HIVE-23467 > URL: https://issues.apache.org/jira/browse/HIVE-23467 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Sam An >Assignee: Sam An >Priority: Trivial > > We have an auto.purge flag, which means skip trash. It can be confusing as we > have 'external.table.purge'='true' to indicate delete table data when this > tblproperties is set. > > We should make the meaning clearer by introducing a skip trash alias/option. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23099) Improve Logger for Operation Child Classes
[
https://issues.apache.org/jira/browse/HIVE-23099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106748#comment-17106748
]
Hive QA commented on HIVE-23099:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 11m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
50s{color} | {color:blue} service in master has 51 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
21s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
16s{color} | {color:red} service: The patch generated 1 new + 57 unchanged - 2
fixed = 58 total (was 59) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 17m 2s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22320/dev-support/hive-personality.sh
|
| git revision | master / b63c35a |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22320/yetus/diff-checkstyle-service.txt
|
| modules | C: service U: service |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22320/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Improve Logger for Operation Child Classes
> --
>
> Key: HIVE-23099
> URL: https://issues.apache.org/jira/browse/HIVE-23099
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HIVE-23099.1.patch, HIVE-23099.2.patch,
> HIVE-23099.3.patch, HIVE-23099.4.patch, HIVE-23099.4.patch,
> HIVE-23099.5.patch, HIVE-23099.5.patch, HIVE-23099.5.patch
>
>
> The {{Operation}} class declares its logger this way:
> {code:java|title=Operation.java}
> public abstract class Operation {
> public static final Logger LOG =
> LoggerFactory.getLogger(Operation.class.getName());
> ...
> }
> {code}
> Notice that this is an {{abstract}} class, but the {{Logger}} is tied to the
> {{Operation.class.getName()}}. This means that logging cannot be controlled
> for each subclass of {{Operation}} independently since they all use the same
> static {{Logger}} instance.
> Make the LOG a {{protected}} instance variable that inherits the name of the
> child class.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22678) Run Eclipse Cleanup Against hive-accumulo-handler Module
[
https://issues.apache.org/jira/browse/HIVE-22678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106803#comment-17106803
]
Hive QA commented on HIVE-22678:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 15s{color}
| {color:red}
/data/hiveptest/logs/PreCommit-HIVE-Build-22323/patches/PreCommit-HIVE-Build-22323.patch
does not apply to master. Rebase required? Wrong Branch? See
http://cwiki.apache.org/confluence/display/Hive/HowToContribute for help.
{color} |
\\
\\
|| Subsystem || Report/Notes ||
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22323/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Run Eclipse Cleanup Against hive-accumulo-handler Module
>
>
> Key: HIVE-22678
> URL: https://issues.apache.org/jira/browse/HIVE-22678
> Project: Hive
> Issue Type: Sub-task
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HIVE-22678.1.patch, HIVE-22678.2.patch,
> HIVE-22678.2.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22675) Replace Base64 in hive-standalone-metastore Package
[
https://issues.apache.org/jira/browse/HIVE-22675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106737#comment-17106737
]
Hive QA commented on HIVE-22675:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13002837/HIVE-22675.3.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 17265 tests
executed
*Failed tests:*
{noformat}
TestStatsReplicationScenariosACID - did not produce a TEST-*.xml file (likely
timed out) (batchId=184)
org.apache.hadoop.hive.metastore.TestHiveMetaStorePartitionSpecs.org.apache.hadoop.hive.metastore.TestHiveMetaStorePartitionSpecs
(batchId=156)
org.apache.hadoop.hive.metastore.TestHiveMetaStorePartitionSpecs.testAddPartitions
(batchId=156)
org.apache.hadoop.hive.metastore.TestHiveMetaStorePartitionSpecs.testFetchingPartitionsWithDifferentSchemas
(batchId=156)
org.apache.hadoop.hive.metastore.TestHiveMetaStorePartitionSpecs.testGetPartitionSpecs_WithAndWithoutPartitionGrouping
(batchId=156)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22319/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22319/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22319/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13002837 - PreCommit-HIVE-Build
> Replace Base64 in hive-standalone-metastore Package
> ---
>
> Key: HIVE-22675
> URL: https://issues.apache.org/jira/browse/HIVE-22675
> Project: Hive
> Issue Type: Sub-task
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HIVE-22675.1.patch, HIVE-22675.2.patch,
> HIVE-22675.3.patch, HIVE-22675.3.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23440) Move q tests to TestMiniLlapLocal from TestCliDriver where the output is different, batch 4
[ https://issues.apache.org/jira/browse/HIVE-23440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-23440: -- Attachment: HIVE-23440.01.patch > Move q tests to TestMiniLlapLocal from TestCliDriver where the output is > different, batch 4 > --- > > Key: HIVE-23440 > URL: https://issues.apache.org/jira/browse/HIVE-23440 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Attachments: HIVE-23440.01.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23354) Remove file size sanity checking from compareTempOrDuplicateFiles
[ https://issues.apache.org/jira/browse/HIVE-23354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Sherman updated HIVE-23354: Attachment: HIVE-23354.4.patch > Remove file size sanity checking from compareTempOrDuplicateFiles > - > > Key: HIVE-23354 > URL: https://issues.apache.org/jira/browse/HIVE-23354 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: John Sherman >Assignee: John Sherman >Priority: Major > Attachments: HIVE-23354.1.patch, HIVE-23354.2.patch, > HIVE-23354.3.patch, HIVE-23354.4.patch > > > [https://github.com/apache/hive/blob/cdd55aa319a3440963a886ebfff11cd2a240781d/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java#L1952-L2010] > compareTempOrDuplicateFiles uses a combination of attemptId and fileSize to > determine which file(s) to keep. > I've seen instances where this function throws an exception due to the fact > that the newer attemptId file size is less than the older attemptId (thus > failing the query). > I think this assumption is faulty, due to various factors such as file > compression and the order in which values are written. It may be prudent to > trust that the newest attemptId is in fact the best choice. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23354) Remove file size sanity checking from compareTempOrDuplicateFiles
[ https://issues.apache.org/jira/browse/HIVE-23354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106792#comment-17106792 ] John Sherman commented on HIVE-23354: - Newest patch I expect to do much better on the test front, I took a different approach on disabling speculative execution. (Similar to how hive.mapred.reduce.tasks.speculative.execution worked, but in reverse). I also marked hive.mapred.reduce.tasks.speculative.execution as deprecated and removed the usage of it, to prevent foot shooting. > Remove file size sanity checking from compareTempOrDuplicateFiles > - > > Key: HIVE-23354 > URL: https://issues.apache.org/jira/browse/HIVE-23354 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: John Sherman >Assignee: John Sherman >Priority: Major > Attachments: HIVE-23354.1.patch, HIVE-23354.2.patch, > HIVE-23354.3.patch, HIVE-23354.4.patch > > > [https://github.com/apache/hive/blob/cdd55aa319a3440963a886ebfff11cd2a240781d/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java#L1952-L2010] > compareTempOrDuplicateFiles uses a combination of attemptId and fileSize to > determine which file(s) to keep. > I've seen instances where this function throws an exception due to the fact > that the newer attemptId file size is less than the older attemptId (thus > failing the query). > I think this assumption is faulty, due to various factors such as file > compression and the order in which values are written. It may be prudent to > trust that the newest attemptId is in fact the best choice. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23438) Missing Rows When Left Outer Join In N-way HybridGraceHashJoin
[
https://issues.apache.org/jira/browse/HIVE-23438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
范宜臻 updated HIVE-23438:
---
Attachment: HIVE-23438.branch-2.3.patch
Status: Patch Available (was: Open)
> Missing Rows When Left Outer Join In N-way HybridGraceHashJoin
> --
>
> Key: HIVE-23438
> URL: https://issues.apache.org/jira/browse/HIVE-23438
> Project: Hive
> Issue Type: Bug
> Components: SQL, Tez
>Affects Versions: 2.3.4
>Reporter: 范宜臻
>Priority: Major
> Attachments: HIVE-23438.branch-2.3.patch
>
>
> *Run Test in Patch File*
> {code:java}
> mvn test -Dtest=TestMiniTezCliDriver -Dqfile=hybridgrace_hashjoin_2.q{code}
> *Manual Reproduce*
> *STEP 1. Create test data(q_test_init_tez.sql)*
> {code:java}
> //create table src1
> CREATE TABLE src1 (key STRING COMMENT 'default', value STRING COMMENT
> 'default') STORED AS TEXTFILE;
> LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/kv3.txt" INTO TABLE src1;
> //create table src2
> CREATE TABLE src2(key STRING COMMENT 'default', value STRING COMMENT
> 'default') STORED AS TEXTFILE;
> LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/kv11.txt" OVERWRITE INTO
> TABLE src2;
> //create table srcpart
> CREATE TABLE srcpart (key STRING COMMENT 'default', value STRING COMMENT
> 'default')
> PARTITIONED BY (ds STRING, hr STRING)
> STORED AS TEXTFILE;
> LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/kv1.txt"
> OVERWRITE INTO TABLE srcpart PARTITION (ds="2008-04-08", hr="11");
> LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/kv1.txt"
> OVERWRITE INTO TABLE srcpart PARTITION (ds="2008-04-08", hr="12");
> LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/kv1.txt"
> OVERWRITE INTO TABLE srcpart PARTITION (ds="2008-04-09", hr="11");
> LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/kv1.txt"
> OVERWRITE INTO TABLE srcpart PARTITION (ds="2008-04-09", hr="12");{code}
> *STEP 2. Run query*
> {code:java}
> set hive.auto.convert.join=true;
> set hive.auto.convert.join.noconditionaltask=true;
> set hive.auto.convert.join.noconditionaltask.size=1000;
> set hive.cbo.enable=false;
> set hive.mapjoin.hybridgrace.hashtable=true;
> select *
> from
> (
> select key from src1 group by key
> ) x
> left join src2 z on x.key = z.key
> join
> (
> select key from srcpart y group by key
> ) y on y.key = x.key;
> {code}
> *EXPECTED RESULT***
>
> {code:java}
> 128 NULLNULL128
> 146 146 1val_1461 146
> 150 150 1val_1501 150
> 238 NULLNULL238
> 369 NULLNULL369
> 406 406 1val_4061 406
> 273 273 1val_2731 273
> 98NULLNULL98
> 213 213 1val_2131 213
> 255 NULLNULL255
> 401 401 1val_4011 401
> 278 NULLNULL278
> 6666 11val_6611 66
> 224 NULLNULL224
> 311 NULLNULL311
> {code}
>
> *ACTUAL RESULT*
> {code:java}
> 128 NULLNULL128
> 146 146 1val_1461 146
> 150 150 1val_1501 150
> 213 213 1val_2131 213
> 238 NULLNULL238
> 273 273 1val_2731 273
> 369 NULLNULL369
> 406 406 1val_4061 406
> 98NULLNULL98
> 401 401 1val_4011 401
> 6666 11val_6611 66
> {code}
>
> *ROOT CAUSE*
> src1 left join src2, src1 is big table and src2 is small table. Join result
> between big table row and the corresponding hashtable maybe NO_MATCH state,
> however, these NO_MATCH rows is needed because LEFT OUTER JOIN.
> In addition, these big table rows will not spilled into matchfile related to
> this hashtable on disk because only SPILL state can use `spillBigTableRow`.
> Then, these big table rows will be spilled into matchfile in hashtables of
> table `srcpart`(second small table)
> Finally, when reProcessBigTable, big table rows in matchfile are only read
> from `firstSmallTable`, some datum are missing.
>
> *WORKAROUND*
> configure firstSmallTable in completeInitializationOp and only spill big
> table row into firstSmallTable when spill matchfile.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23456) Upgrade Calcite version to 1.23.0
[ https://issues.apache.org/jira/browse/HIVE-23456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stamatis Zampetakis updated HIVE-23456: --- Status: Patch Available (was: Open) Initial patch just to see the status of the tests. > Upgrade Calcite version to 1.23.0 > - > > Key: HIVE-23456 > URL: https://issues.apache.org/jira/browse/HIVE-23456 > Project: Hive > Issue Type: Task >Reporter: Stamatis Zampetakis >Assignee: Stamatis Zampetakis >Priority: Major > Attachments: HIVE-23456.01.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23467) Add a skip.trash config for HMS to skip trash when deleting external table data
[ https://issues.apache.org/jira/browse/HIVE-23467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam An updated HIVE-23467: -- Description: We have an auto.purge flag, which means skip trash. It can be confusing as we have 'external.table.purge'='true' to indicate delete table data when this tblproperties is set. We should make the meaning clearer by introducing a skip trash alias/option. Additionally, we shall add an alias for external.table.purge, and name it external.table.autodelete, and document it more prominently, so as to maintain backward compatibility, and make the meaning of auto deletion of data more obvious. The net effect of these 2 changes will be. If the user sets 'external.table.autodelete'='true' the table data will be removed when table is dropped. and if 'skip.trash'='true' is set, HMS will not move the table data to trash folder when removing the files. This will result in faster removal, especially when underlying FS is S3. was: We have an auto.purge flag, which means skip trash. It can be confusing as we have 'external.table.purge'='true' to indicate delete table data when this tblproperties is set. We should make the meaning clearer by introducing a skip trash alias/option. > Add a skip.trash config for HMS to skip trash when deleting external table > data > --- > > Key: HIVE-23467 > URL: https://issues.apache.org/jira/browse/HIVE-23467 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Sam An >Assignee: Sam An >Priority: Trivial > > We have an auto.purge flag, which means skip trash. It can be confusing as we > have 'external.table.purge'='true' to indicate delete table data when this > tblproperties is set. > We should make the meaning clearer by introducing a skip trash alias/option. > Additionally, we shall add an alias for external.table.purge, and name it > external.table.autodelete, and document it more prominently, so as to > maintain backward compatibility, and make the meaning of auto deletion of > data more obvious. > The net effect of these 2 changes will be. If the user sets > 'external.table.autodelete'='true' > the table data will be removed when table is dropped. and if > 'skip.trash'='true' > is set, HMS will not move the table data to trash folder when removing the > files. This will result in faster removal, especially when underlying FS is > S3. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23440) Move q tests to TestMiniLlapLocal from TestCliDriver where the output is different, batch 4
[ https://issues.apache.org/jira/browse/HIVE-23440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-23440: -- Status: Patch Available (was: Open) > Move q tests to TestMiniLlapLocal from TestCliDriver where the output is > different, batch 4 > --- > > Key: HIVE-23440 > URL: https://issues.apache.org/jira/browse/HIVE-23440 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Attachments: HIVE-23440.01.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23354) Remove file size sanity checking from compareTempOrDuplicateFiles
[ https://issues.apache.org/jira/browse/HIVE-23354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Sherman updated HIVE-23354: Attachment: HIVE-23354.3.patch > Remove file size sanity checking from compareTempOrDuplicateFiles > - > > Key: HIVE-23354 > URL: https://issues.apache.org/jira/browse/HIVE-23354 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: John Sherman >Assignee: John Sherman >Priority: Major > Attachments: HIVE-23354.1.patch, HIVE-23354.2.patch, > HIVE-23354.3.patch > > > [https://github.com/apache/hive/blob/cdd55aa319a3440963a886ebfff11cd2a240781d/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java#L1952-L2010] > compareTempOrDuplicateFiles uses a combination of attemptId and fileSize to > determine which file(s) to keep. > I've seen instances where this function throws an exception due to the fact > that the newer attemptId file size is less than the older attemptId (thus > failing the query). > I think this assumption is faulty, due to various factors such as file > compression and the order in which values are written. It may be prudent to > trust that the newest attemptId is in fact the best choice. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23354) Remove file size sanity checking from compareTempOrDuplicateFiles
[ https://issues.apache.org/jira/browse/HIVE-23354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106758#comment-17106758 ] John Sherman commented on HIVE-23354: - I expect some failures with the new commit. I'm still trying to work out the best way to handle the fact that mapreduce.map.speculative is the default setting from MR config. > Remove file size sanity checking from compareTempOrDuplicateFiles > - > > Key: HIVE-23354 > URL: https://issues.apache.org/jira/browse/HIVE-23354 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: John Sherman >Assignee: John Sherman >Priority: Major > Attachments: HIVE-23354.1.patch, HIVE-23354.2.patch, > HIVE-23354.3.patch > > > [https://github.com/apache/hive/blob/cdd55aa319a3440963a886ebfff11cd2a240781d/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java#L1952-L2010] > compareTempOrDuplicateFiles uses a combination of attemptId and fileSize to > determine which file(s) to keep. > I've seen instances where this function throws an exception due to the fact > that the newer attemptId file size is less than the older attemptId (thus > failing the query). > I think this assumption is faulty, due to various factors such as file > compression and the order in which values are written. It may be prudent to > trust that the newest attemptId is in fact the best choice. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23017) Use Inherited Logger in Tasks
[
https://issues.apache.org/jira/browse/HIVE-23017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106762#comment-17106762
]
Hive QA commented on HIVE-23017:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13002839/HIVE-23017.5.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22321/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22321/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22321/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2020-05-14 00:58:20.922
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-22321/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2020-05-14 00:58:20.925
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at b63c35a HIVE-23451 : FileSinkOperator calls deleteOnExit (hdfs
call) twice for the same file ( Rajesh Balamohan via Ashutosh Chauhan )
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at b63c35a HIVE-23451 : FileSinkOperator calls deleteOnExit (hdfs
call) twice for the same file ( Rajesh Balamohan via Ashutosh Chauhan )
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2020-05-14 00:58:21.949
+ rm -rf ../yetus_PreCommit-HIVE-Build-22321
+ mkdir ../yetus_PreCommit-HIVE-Build-22321
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-22321
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-22321/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
Trying to apply the patch with -p0
error: a/ql/src/java/org/apache/hadoop/hive/ql/ddl/DDLTask.java: does not exist
in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/ColumnStatsUpdateTask.java:
does not exist in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/ExplainSQRewriteTask.java:
does not exist in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/ExplainTask.java: does not
exist in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/ExportTask.java: does not
exist in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/FetchTask.java: does not
exist in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/MoveTask.java: does not
exist in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/ReplCopyTask.java: does not
exist in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/ReplTxnTask.java: does not
exist in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/StatsTask.java: does not
exist in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/Task.java: does not exist
in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/mr/ExecDriver.java: does
not exist in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/mr/MapRedTask.java: does
not exist in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/mr/MapredLocalTask.java:
does not exist in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/repl/DirCopyTask.java: does
not exist in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplDumpTask.java:
does not exist in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplLoadTask.java:
does not exist in index
error:
a/ql/src/java/org/apache/hadoop/hive/ql/exec/schq/ScheduledQueryMaintenanceTask.java:
does not exist in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/spark/SparkTask.java: does
not exist in
[jira] [Commented] (HIVE-22674) Replace Base64 in serde Package
[
https://issues.apache.org/jira/browse/HIVE-22674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106794#comment-17106794
]
Hive QA commented on HIVE-22674:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13002840/HIVE-22674.2.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 17268 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[compute_stats_binary]
(batchId=78)
org.apache.hadoop.hive.metastore.TestPartitionManagement.testPartitionDiscoveryTransactionalTable
(batchId=152)
org.apache.hadoop.hive.ql.parse.TestScheduledReplicationScenarios.testAcidTablesReplLoadBootstrapIncr
(batchId=205)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22322/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22322/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22322/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13002840 - PreCommit-HIVE-Build
> Replace Base64 in serde Package
> ---
>
> Key: HIVE-22674
> URL: https://issues.apache.org/jira/browse/HIVE-22674
> Project: Hive
> Issue Type: Sub-task
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HIVE-22674.1.patch, HIVE-22674.2.patch,
> HIVE-22674.2.patch, HIVE-22674.2.patch, HIVE-22674.2.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23438) Missing Rows When Left Outer Join In N-way HybridGraceHashJoin
[
https://issues.apache.org/jira/browse/HIVE-23438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
范宜臻 updated HIVE-23438:
---
Status: Open (was: Patch Available)
> Missing Rows When Left Outer Join In N-way HybridGraceHashJoin
> --
>
> Key: HIVE-23438
> URL: https://issues.apache.org/jira/browse/HIVE-23438
> Project: Hive
> Issue Type: Bug
> Components: SQL, Tez
>Affects Versions: 2.3.4
>Reporter: 范宜臻
>Priority: Major
>
> *Run Test in Patch File*
> {code:java}
> mvn test -Dtest=TestMiniTezCliDriver -Dqfile=hybridgrace_hashjoin_2.q{code}
> *Manual Reproduce*
> *STEP 1. Create test data(q_test_init_tez.sql)*
> {code:java}
> //create table src1
> CREATE TABLE src1 (key STRING COMMENT 'default', value STRING COMMENT
> 'default') STORED AS TEXTFILE;
> LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/kv3.txt" INTO TABLE src1;
> //create table src2
> CREATE TABLE src2(key STRING COMMENT 'default', value STRING COMMENT
> 'default') STORED AS TEXTFILE;
> LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/kv11.txt" OVERWRITE INTO
> TABLE src2;
> //create table srcpart
> CREATE TABLE srcpart (key STRING COMMENT 'default', value STRING COMMENT
> 'default')
> PARTITIONED BY (ds STRING, hr STRING)
> STORED AS TEXTFILE;
> LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/kv1.txt"
> OVERWRITE INTO TABLE srcpart PARTITION (ds="2008-04-08", hr="11");
> LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/kv1.txt"
> OVERWRITE INTO TABLE srcpart PARTITION (ds="2008-04-08", hr="12");
> LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/kv1.txt"
> OVERWRITE INTO TABLE srcpart PARTITION (ds="2008-04-09", hr="11");
> LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/kv1.txt"
> OVERWRITE INTO TABLE srcpart PARTITION (ds="2008-04-09", hr="12");{code}
> *STEP 2. Run query*
> {code:java}
> set hive.auto.convert.join=true;
> set hive.auto.convert.join.noconditionaltask=true;
> set hive.auto.convert.join.noconditionaltask.size=1000;
> set hive.cbo.enable=false;
> set hive.mapjoin.hybridgrace.hashtable=true;
> select *
> from
> (
> select key from src1 group by key
> ) x
> left join src2 z on x.key = z.key
> join
> (
> select key from srcpart y group by key
> ) y on y.key = x.key;
> {code}
> *EXPECTED RESULT***
>
> {code:java}
> 128 NULLNULL128
> 146 146 1val_1461 146
> 150 150 1val_1501 150
> 238 NULLNULL238
> 369 NULLNULL369
> 406 406 1val_4061 406
> 273 273 1val_2731 273
> 98NULLNULL98
> 213 213 1val_2131 213
> 255 NULLNULL255
> 401 401 1val_4011 401
> 278 NULLNULL278
> 6666 11val_6611 66
> 224 NULLNULL224
> 311 NULLNULL311
> {code}
>
> *ACTUAL RESULT*
> {code:java}
> 128 NULLNULL128
> 146 146 1val_1461 146
> 150 150 1val_1501 150
> 213 213 1val_2131 213
> 238 NULLNULL238
> 273 273 1val_2731 273
> 369 NULLNULL369
> 406 406 1val_4061 406
> 98NULLNULL98
> 401 401 1val_4011 401
> 6666 11val_6611 66
> {code}
>
> *ROOT CAUSE*
> src1 left join src2, src1 is big table and src2 is small table. Join result
> between big table row and the corresponding hashtable maybe NO_MATCH state,
> however, these NO_MATCH rows is needed because LEFT OUTER JOIN.
> In addition, these big table rows will not spilled into matchfile related to
> this hashtable on disk because only SPILL state can use `spillBigTableRow`.
> Then, these big table rows will be spilled into matchfile in hashtables of
> table `srcpart`(second small table)
> Finally, when reProcessBigTable, big table rows in matchfile are only read
> from `firstSmallTable`, some datum are missing.
>
> *WORKAROUND*
> configure firstSmallTable in completeInitializationOp and only spill big
> table row into firstSmallTable when spill matchfile.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23438) Missing Rows When Left Outer Join In N-way HybridGraceHashJoin
[
https://issues.apache.org/jira/browse/HIVE-23438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
范宜臻 updated HIVE-23438:
---
Attachment: (was: HIVE-23438.branch-2.3.patch)
> Missing Rows When Left Outer Join In N-way HybridGraceHashJoin
> --
>
> Key: HIVE-23438
> URL: https://issues.apache.org/jira/browse/HIVE-23438
> Project: Hive
> Issue Type: Bug
> Components: SQL, Tez
>Affects Versions: 2.3.4
>Reporter: 范宜臻
>Priority: Major
>
> *Run Test in Patch File*
> {code:java}
> mvn test -Dtest=TestMiniTezCliDriver -Dqfile=hybridgrace_hashjoin_2.q{code}
> *Manual Reproduce*
> *STEP 1. Create test data(q_test_init_tez.sql)*
> {code:java}
> //create table src1
> CREATE TABLE src1 (key STRING COMMENT 'default', value STRING COMMENT
> 'default') STORED AS TEXTFILE;
> LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/kv3.txt" INTO TABLE src1;
> //create table src2
> CREATE TABLE src2(key STRING COMMENT 'default', value STRING COMMENT
> 'default') STORED AS TEXTFILE;
> LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/kv11.txt" OVERWRITE INTO
> TABLE src2;
> //create table srcpart
> CREATE TABLE srcpart (key STRING COMMENT 'default', value STRING COMMENT
> 'default')
> PARTITIONED BY (ds STRING, hr STRING)
> STORED AS TEXTFILE;
> LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/kv1.txt"
> OVERWRITE INTO TABLE srcpart PARTITION (ds="2008-04-08", hr="11");
> LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/kv1.txt"
> OVERWRITE INTO TABLE srcpart PARTITION (ds="2008-04-08", hr="12");
> LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/kv1.txt"
> OVERWRITE INTO TABLE srcpart PARTITION (ds="2008-04-09", hr="11");
> LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/kv1.txt"
> OVERWRITE INTO TABLE srcpart PARTITION (ds="2008-04-09", hr="12");{code}
> *STEP 2. Run query*
> {code:java}
> set hive.auto.convert.join=true;
> set hive.auto.convert.join.noconditionaltask=true;
> set hive.auto.convert.join.noconditionaltask.size=1000;
> set hive.cbo.enable=false;
> set hive.mapjoin.hybridgrace.hashtable=true;
> select *
> from
> (
> select key from src1 group by key
> ) x
> left join src2 z on x.key = z.key
> join
> (
> select key from srcpart y group by key
> ) y on y.key = x.key;
> {code}
> *EXPECTED RESULT***
>
> {code:java}
> 128 NULLNULL128
> 146 146 1val_1461 146
> 150 150 1val_1501 150
> 238 NULLNULL238
> 369 NULLNULL369
> 406 406 1val_4061 406
> 273 273 1val_2731 273
> 98NULLNULL98
> 213 213 1val_2131 213
> 255 NULLNULL255
> 401 401 1val_4011 401
> 278 NULLNULL278
> 6666 11val_6611 66
> 224 NULLNULL224
> 311 NULLNULL311
> {code}
>
> *ACTUAL RESULT*
> {code:java}
> 128 NULLNULL128
> 146 146 1val_1461 146
> 150 150 1val_1501 150
> 213 213 1val_2131 213
> 238 NULLNULL238
> 273 273 1val_2731 273
> 369 NULLNULL369
> 406 406 1val_4061 406
> 98NULLNULL98
> 401 401 1val_4011 401
> 6666 11val_6611 66
> {code}
>
> *ROOT CAUSE*
> src1 left join src2, src1 is big table and src2 is small table. Join result
> between big table row and the corresponding hashtable maybe NO_MATCH state,
> however, these NO_MATCH rows is needed because LEFT OUTER JOIN.
> In addition, these big table rows will not spilled into matchfile related to
> this hashtable on disk because only SPILL state can use `spillBigTableRow`.
> Then, these big table rows will be spilled into matchfile in hashtables of
> table `srcpart`(second small table)
> Finally, when reProcessBigTable, big table rows in matchfile are only read
> from `firstSmallTable`, some datum are missing.
>
> *WORKAROUND*
> configure firstSmallTable in completeInitializationOp and only spill big
> table row into firstSmallTable when spill matchfile.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23460) Add qoption to disable qtests
[
https://issues.apache.org/jira/browse/HIVE-23460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106659#comment-17106659
]
Hive QA commented on HIVE-23460:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
19s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
17s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
25s{color} | {color:blue} ql in master has 1527 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
37s{color} | {color:blue} accumulo-handler in master has 20 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
57s{color} | {color:blue} itests/util in master has 54 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
45s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
33s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s{color} | {color:green} The patch ql passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} The patch accumulo-handler passed checkstyle {color}
|
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s{color} | {color:green} itests/util: The patch generated 0 new + 63
unchanged - 7 fixed = 63 total (was 70) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
48s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 39m 28s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22317/dev-support/hive-personality.sh
|
| git revision | master / 472aca8 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql accumulo-handler itests itests/util U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22317/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Add qoption to disable qtests
> -
>
> Key: HIVE-23460
> URL: https://issues.apache.org/jira/browse/HIVE-23460
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-23460.01.patch, HIVE-23460.02.patch
>
>
> instead other ways to exclude them... (testconfiguration.properties;
> CliConfig#excludeQuery)
> {code}
> --! qt:disabled:reason
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23451) FileSinkOperator calls deleteOnExit (hdfs call) twice for the same file
[
https://issues.apache.org/jira/browse/HIVE-23451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106696#comment-17106696
]
Hive QA commented on HIVE-23451:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 11m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
32s{color} | {color:blue} ql in master has 1527 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
6s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 29m 1s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22318/dev-support/hive-personality.sh
|
| git revision | master / 472aca8 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22318/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> FileSinkOperator calls deleteOnExit (hdfs call) twice for the same file
> ---
>
> Key: HIVE-23451
> URL: https://issues.apache.org/jira/browse/HIVE-23451
> Project: Hive
> Issue Type: Improvement
>Reporter: Rajesh Balamohan
>Assignee: Rajesh Balamohan
>Priority: Minor
> Attachments: HIVE-23451.1.patch, HIVE-23451.2.patch,
> HIVE-23451.3.patch
>
>
> [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java#L826]
> [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java#L797]
> Can avoid a NN call here (i.e, mainly for small queries).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23099) Improve Logger for Operation Child Classes
[
https://issues.apache.org/jira/browse/HIVE-23099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106759#comment-17106759
]
Hive QA commented on HIVE-23099:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13002845/HIVE-23099.5.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17268 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22320/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22320/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22320/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13002845 - PreCommit-HIVE-Build
> Improve Logger for Operation Child Classes
> --
>
> Key: HIVE-23099
> URL: https://issues.apache.org/jira/browse/HIVE-23099
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HIVE-23099.1.patch, HIVE-23099.2.patch,
> HIVE-23099.3.patch, HIVE-23099.4.patch, HIVE-23099.4.patch,
> HIVE-23099.5.patch, HIVE-23099.5.patch, HIVE-23099.5.patch
>
>
> The {{Operation}} class declares its logger this way:
> {code:java|title=Operation.java}
> public abstract class Operation {
> public static final Logger LOG =
> LoggerFactory.getLogger(Operation.class.getName());
> ...
> }
> {code}
> Notice that this is an {{abstract}} class, but the {{Logger}} is tied to the
> {{Operation.class.getName()}}. This means that logging cannot be controlled
> for each subclass of {{Operation}} independently since they all use the same
> static {{Logger}} instance.
> Make the LOG a {{protected}} instance variable that inherits the name of the
> child class.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23440) Move q tests to TestMiniLlapLocal from TestCliDriver where the output is different, batch 4
[ https://issues.apache.org/jira/browse/HIVE-23440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106751#comment-17106751 ] Jesus Camacho Rodriguez commented on HIVE-23440: +1 (pending tests) > Move q tests to TestMiniLlapLocal from TestCliDriver where the output is > different, batch 4 > --- > > Key: HIVE-23440 > URL: https://issues.apache.org/jira/browse/HIVE-23440 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Attachments: HIVE-23440.01.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22674) Replace Base64 in serde Package
[
https://issues.apache.org/jira/browse/HIVE-22674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106769#comment-17106769
]
Hive QA commented on HIVE-22674:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 11m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
48s{color} | {color:blue} serde in master has 198 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
17s{color} | {color:red} serde: The patch generated 1 new + 33 unchanged - 1
fixed = 34 total (was 34) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 16m 21s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22322/dev-support/hive-personality.sh
|
| git revision | master / b63c35a |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22322/yetus/diff-checkstyle-serde.txt
|
| modules | C: serde U: serde |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22322/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Replace Base64 in serde Package
> ---
>
> Key: HIVE-22674
> URL: https://issues.apache.org/jira/browse/HIVE-22674
> Project: Hive
> Issue Type: Sub-task
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HIVE-22674.1.patch, HIVE-22674.2.patch,
> HIVE-22674.2.patch, HIVE-22674.2.patch, HIVE-22674.2.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23460) Add qoption to disable qtests
[
https://issues.apache.org/jira/browse/HIVE-23460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Haindrich updated HIVE-23460:
Attachment: HIVE-23460.02.patch
> Add qoption to disable qtests
> -
>
> Key: HIVE-23460
> URL: https://issues.apache.org/jira/browse/HIVE-23460
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-23460.01.patch, HIVE-23460.02.patch,
> HIVE-23460.02.patch
>
>
> instead other ways to exclude them... (testconfiguration.properties;
> CliConfig#excludeQuery)
> {code}
> --! qt:disabled:reason
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23456) Upgrade Calcite version to 1.23.0
[ https://issues.apache.org/jira/browse/HIVE-23456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stamatis Zampetakis updated HIVE-23456: --- Attachment: HIVE-23456.01.patch > Upgrade Calcite version to 1.23.0 > - > > Key: HIVE-23456 > URL: https://issues.apache.org/jira/browse/HIVE-23456 > Project: Hive > Issue Type: Task >Reporter: Stamatis Zampetakis >Assignee: Stamatis Zampetakis >Priority: Major > Attachments: HIVE-23456.01.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23451) FileSinkOperator calls deleteOnExit (hdfs call) twice for the same file
[ https://issues.apache.org/jira/browse/HIVE-23451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-23451: Fix Version/s: 4.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to master. Thanks, Rajesh! > FileSinkOperator calls deleteOnExit (hdfs call) twice for the same file > --- > > Key: HIVE-23451 > URL: https://issues.apache.org/jira/browse/HIVE-23451 > Project: Hive > Issue Type: Improvement >Reporter: Rajesh Balamohan >Assignee: Rajesh Balamohan >Priority: Minor > Fix For: 4.0.0 > > Attachments: HIVE-23451.1.patch, HIVE-23451.2.patch, > HIVE-23451.3.patch > > > [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java#L826] > [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java#L797] > Can avoid a NN call here (i.e, mainly for small queries). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22675) Replace Base64 in hive-standalone-metastore Package
[
https://issues.apache.org/jira/browse/HIVE-22675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106719#comment-17106719
]
Hive QA commented on HIVE-22675:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 11m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
21s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
31s{color} | {color:blue} standalone-metastore/metastore-server in master has
186 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
23s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
21s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 1 new + 31 unchanged - 1 fixed = 32 total (was 32) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 18m 33s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22319/dev-support/hive-personality.sh
|
| git revision | master / 472aca8 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22319/yetus/diff-checkstyle-standalone-metastore_metastore-server.txt
|
| modules | C: standalone-metastore/metastore-server U:
standalone-metastore/metastore-server |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22319/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Replace Base64 in hive-standalone-metastore Package
> ---
>
> Key: HIVE-22675
> URL: https://issues.apache.org/jira/browse/HIVE-22675
> Project: Hive
> Issue Type: Sub-task
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HIVE-22675.1.patch, HIVE-22675.2.patch,
> HIVE-22675.3.patch, HIVE-22675.3.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23460) Add qoption to disable qtests
[
https://issues.apache.org/jira/browse/HIVE-23460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106676#comment-17106676
]
Hive QA commented on HIVE-23460:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13002830/HIVE-23460.02.patch
{color:green}SUCCESS:{color} +1 due to 39 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 17266 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.TestWarehouseExternalDir.org.apache.hadoop.hive.ql.TestWarehouseExternalDir
(batchId=180)
org.apache.hadoop.hive.ql.TestWarehouseExternalDir.testExternalDefaultPaths
(batchId=180)
org.apache.hadoop.hive.ql.parse.TestScheduledReplicationScenarios.testAcidTablesReplLoadBootstrapIncr
(batchId=205)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22317/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22317/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22317/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13002830 - PreCommit-HIVE-Build
> Add qoption to disable qtests
> -
>
> Key: HIVE-23460
> URL: https://issues.apache.org/jira/browse/HIVE-23460
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-23460.01.patch, HIVE-23460.02.patch
>
>
> instead other ways to exclude them... (testconfiguration.properties;
> CliConfig#excludeQuery)
> {code}
> --! qt:disabled:reason
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23390) Duplicate entry for a table in TAB_COL_STATS
[
https://issues.apache.org/jira/browse/HIVE-23390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106766#comment-17106766
]
Mithun Antony commented on HIVE-23390:
--
Suggestion is to put a unique constraint in the TAB_COL_STATS on columns
DB_NAME, TABLE_NAME, COLUMN_NAME considering the fact that a there will be only
one record for a column
> Duplicate entry for a table in TAB_COL_STATS
> -
>
> Key: HIVE-23390
> URL: https://issues.apache.org/jira/browse/HIVE-23390
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 2.3.4
>Reporter: Mithun Antony
>Priority: Major
>
> When *_analyze _* command was executed from presto to update the stats
> of a table for the first time from multiple cluster sharing the same Hive
> metastore. Duplicate entry for the same table is inserted to the
> *_TAB_COL_STATS_* table.
> This lead to failure executing further *_analyze _* commands.
> {code:java}
> Query failed: Multiple entries with same key:
> dummy=HiveColumnStatistics{integerStatistics=Optional[IntegerStatistics{min=OptionalLong[1],
> max=OptionalLong[1]}], doubleStatistics=Optional.empty,
> decimalStatistics=Optional.empty, dateStatistics=Optional.empty,
> booleanStatistics=Optional.empty, maxValueSizeInBytes=OptionalLong.empty,
> totalSizeInBytes=OptionalLong.empty, nullsCount=OptionalLong[0],
> distinctValuesCount=OptionalLong[1]} and
> dummy=HiveColumnStatistics{integerStatistics=Optional[IntegerStatistics{min=OptionalLong[1],
> max=OptionalLong[1]}], doubleStatistics=Optional.empty,
> decimalStatistics=Optional.empty, dateStatistics=Optional.empty,
> booleanStatistics=Optional.empty, maxValueSizeInBytes=OptionalLong.empty,
> totalSizeInBytes=OptionalLong.empty, nullsCount=OptionalLong[0],
> distinctValuesCount=OptionalLong[1]}.
> {code}
> Duplicate records in the *_TAB_COL_STATS_*
> {code:java}
> '7','default','dual','dummy','smallint','245671','1','1',NULL,NULL,NULL,NULL,'0','1',NULL,NULL,NULL,NULL,'1588345509'
>
> '11','default','dual','dummy','smallint','245671','1','1',NULL,NULL,NULL,NULL,'0','1',NULL,NULL,NULL,NULL,'1588345509'{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22570) Review of ExprNodeDesc.java
[
https://issues.apache.org/jira/browse/HIVE-22570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106885#comment-17106885
]
Hive QA commented on HIVE-22570:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13002842/HIVE-22570.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17239 tests
executed
*Failed tests:*
{noformat}
TestReplAcidTablesWithJsonMessage - did not produce a TEST-*.xml file (likely
timed out) (batchId=199)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22324/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22324/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22324/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13002842 - PreCommit-HIVE-Build
> Review of ExprNodeDesc.java
> ---
>
> Key: HIVE-22570
> URL: https://issues.apache.org/jira/browse/HIVE-22570
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HIVE-22570.1.patch, HIVE-22570.1.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22570) Review of ExprNodeDesc.java
[
https://issues.apache.org/jira/browse/HIVE-22570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106861#comment-17106861
]
Hive QA commented on HIVE-22570:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 11m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
47s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
26s{color} | {color:blue} ql in master has 1527 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 29m 19s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22324/dev-support/hive-personality.sh
|
| git revision | master / b63c35a |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22324/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Review of ExprNodeDesc.java
> ---
>
> Key: HIVE-22570
> URL: https://issues.apache.org/jira/browse/HIVE-22570
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HIVE-22570.1.patch, HIVE-22570.1.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23446) LLAP: Reduce IPC connection misses to AM for short queries
[
https://issues.apache.org/jira/browse/HIVE-23446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106925#comment-17106925
]
Ashutosh Chauhan commented on HIVE-23446:
-
+1 pending tests.
> LLAP: Reduce IPC connection misses to AM for short queries
> --
>
> Key: HIVE-23446
> URL: https://issues.apache.org/jira/browse/HIVE-23446
> Project: Hive
> Issue Type: Improvement
>Reporter: Rajesh Balamohan
>Assignee: Rajesh Balamohan
>Priority: Major
> Attachments: HIVE-23446.1.patch, HIVE-23446.2.patch
>
>
> [https://github.com/apache/hive/blob/master/llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/QueryInfo.java#L343]
>
> Umbilical UGI pool for is maintained at QueryInfo level. When there are lots
> of short queries, this ends up missing IPC cache and ends up recreating
> threads/connections to the same AM.
> It would be good to maintain this pool in {{ContainerRunnerImpl}} instead and
> recycle as needed.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-18882) Do Not Hide Exception in Hive Metastore Client Connection
[
https://issues.apache.org/jira/browse/HIVE-18882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106930#comment-17106930
]
Hive QA commented on HIVE-18882:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13002844/HIVE-18882.5.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17239 tests
executed
*Failed tests:*
{noformat}
TestReplAcidTablesWithJsonMessage - did not produce a TEST-*.xml file (likely
timed out) (batchId=199)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22325/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22325/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22325/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13002844 - PreCommit-HIVE-Build
> Do Not Hide Exception in Hive Metastore Client Connection
> -
>
> Key: HIVE-18882
> URL: https://issues.apache.org/jira/browse/HIVE-18882
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Affects Versions: 3.0.0
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Labels: noob
> Attachments: HIVE-18882.1.patch, HIVE-18882.2.patch,
> HIVE-18882.3.patch, HIVE-18882.4.patch, HIVE-18882.5.patch, HIVE-18882.patch
>
>
> [https://github.com/apache/hive/blob/4047befe48c8f762c58d8854e058385c1df151c6/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java#L526-L531]
>
> {code:java}
> if (LOG.isDebugEnabled()) {
> LOG.warn("Failed to connect to the MetaStore Server...", e);
> } else {
> // Don't print full exception trace if DEBUG is not on.
> LOG.warn("Failed to connect to the MetaStore Server...");
> }
> {code}
> I do not understand the logic here. I always want to see the reason for the
> failure. Otherwise, I do not know why it is failing unless I restart the
> server with debug logging enabled. By that point, the error may have
> cleared. Please just use the Exception in the WARN output without adding
> confusing logging for debugging. This is never an expected behavior... that
> enabling debug would change a _warn_ level log message.
> Also... please remove the ellipsis, they add no value.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-18882) Do Not Hide Exception in Hive Metastore Client Connection
[
https://issues.apache.org/jira/browse/HIVE-18882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106894#comment-17106894
]
Hive QA commented on HIVE-18882:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 11m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
39s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
2s{color} | {color:blue} standalone-metastore/metastore-common in master has 35
extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 8s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22325/dev-support/hive-personality.sh
|
| git revision | master / 9f40d7c |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: standalone-metastore/metastore-common U:
standalone-metastore/metastore-common |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22325/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Do Not Hide Exception in Hive Metastore Client Connection
> -
>
> Key: HIVE-18882
> URL: https://issues.apache.org/jira/browse/HIVE-18882
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Affects Versions: 3.0.0
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Labels: noob
> Attachments: HIVE-18882.1.patch, HIVE-18882.2.patch,
> HIVE-18882.3.patch, HIVE-18882.4.patch, HIVE-18882.5.patch, HIVE-18882.patch
>
>
> [https://github.com/apache/hive/blob/4047befe48c8f762c58d8854e058385c1df151c6/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java#L526-L531]
>
> {code:java}
> if (LOG.isDebugEnabled()) {
> LOG.warn("Failed to connect to the MetaStore Server...", e);
> } else {
> // Don't print full exception trace if DEBUG is not on.
> LOG.warn("Failed to connect to the MetaStore Server...");
> }
> {code}
> I do not understand the logic here. I always want to see the reason for the
> failure. Otherwise, I do not know why it is failing unless I restart the
> server with debug logging enabled. By that point, the error may have
> cleared. Please just use the Exception in the WARN output without adding
> confusing logging for debugging. This is never an expected behavior... that
> enabling debug would change a _warn_ level log message.
> Also... please remove the ellipsis, they add no value.
--
This message was sent by
[jira] [Updated] (HIVE-23432) Add Ranger Replication Metrics
[ https://issues.apache.org/jira/browse/HIVE-23432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-23432: --- Status: In Progress (was: Patch Available) > Add Ranger Replication Metrics > --- > > Key: HIVE-23432 > URL: https://issues.apache.org/jira/browse/HIVE-23432 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Attachments: HIVE-23432.01.patch, HIVE-23432.02.patch, > HIVE-23432.03.patch, HIVE-23432.04.patch, HIVE-23432.05.patch, > HIVE-23432.06.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23423) Check of disabling hash aggregation ignores grouping set
[ https://issues.apache.org/jira/browse/HIVE-23423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-23423: Fix Version/s: 4.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to master with updated title. Thanks, Gopal! > Check of disabling hash aggregation ignores grouping set > > > Key: HIVE-23423 > URL: https://issues.apache.org/jira/browse/HIVE-23423 > Project: Hive > Issue Type: Bug > Components: Hive, Operators, Query Processor >Affects Versions: 4.0.0 >Reporter: Nita Dembla >Assignee: Gopal Vijayaraghavan >Priority: Major > Labels: Performance > Fix For: 4.0.0 > > Attachments: HIVE-23423.1.patch, HIVE-23423.WIP.patch > > > https://issues.apache.org/jira/browse/HIVE-23356 fixed the issue with > disabling hash aggregation on grouping set queries. Need a fix for > VectorGroupbyOperator operator. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23446) LLAP: Reduce IPC connection misses to AM for short queries
[
https://issues.apache.org/jira/browse/HIVE-23446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rajesh Balamohan updated HIVE-23446:
Attachment: HIVE-23446.2.patch
> LLAP: Reduce IPC connection misses to AM for short queries
> --
>
> Key: HIVE-23446
> URL: https://issues.apache.org/jira/browse/HIVE-23446
> Project: Hive
> Issue Type: Improvement
>Reporter: Rajesh Balamohan
>Assignee: Rajesh Balamohan
>Priority: Major
> Attachments: HIVE-23446.1.patch, HIVE-23446.2.patch
>
>
> [https://github.com/apache/hive/blob/master/llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/QueryInfo.java#L343]
>
> Umbilical UGI pool for is maintained at QueryInfo level. When there are lots
> of short queries, this ends up missing IPC cache and ends up recreating
> threads/connections to the same AM.
> It would be good to maintain this pool in {{ContainerRunnerImpl}} instead and
> recycle as needed.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22678) Run Eclipse Cleanup Against hive-accumulo-handler Module
[
https://issues.apache.org/jira/browse/HIVE-22678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106839#comment-17106839
]
Hive QA commented on HIVE-22678:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13002841/HIVE-22678.2.patch
{color:green}SUCCESS:{color} +1 due to 10 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17268 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22323/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22323/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22323/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13002841 - PreCommit-HIVE-Build
> Run Eclipse Cleanup Against hive-accumulo-handler Module
>
>
> Key: HIVE-22678
> URL: https://issues.apache.org/jira/browse/HIVE-22678
> Project: Hive
> Issue Type: Sub-task
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HIVE-22678.1.patch, HIVE-22678.2.patch,
> HIVE-22678.2.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23423) Check of disabling hash aggregation ignores grouping set
[ https://issues.apache.org/jira/browse/HIVE-23423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-23423: Summary: Check of disabling hash aggregation ignores grouping set (was: Hash aggregation is always disabled in vectorized execution of grouping set queries) > Check of disabling hash aggregation ignores grouping set > > > Key: HIVE-23423 > URL: https://issues.apache.org/jira/browse/HIVE-23423 > Project: Hive > Issue Type: Bug > Components: Hive, Operators, Query Processor >Affects Versions: 4.0.0 >Reporter: Nita Dembla >Assignee: Gopal Vijayaraghavan >Priority: Major > Labels: Performance > Attachments: HIVE-23423.1.patch, HIVE-23423.WIP.patch > > > https://issues.apache.org/jira/browse/HIVE-23356 fixed the issue with > disabling hash aggregation on grouping set queries. Need a fix for > VectorGroupbyOperator operator. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23423) Hash aggregation is always disabled in vectorized execution of grouping set queries
[ https://issues.apache.org/jira/browse/HIVE-23423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106886#comment-17106886 ] Ashutosh Chauhan commented on HIVE-23423: - +1 > Hash aggregation is always disabled in vectorized execution of grouping set > queries > --- > > Key: HIVE-23423 > URL: https://issues.apache.org/jira/browse/HIVE-23423 > Project: Hive > Issue Type: Bug > Components: Hive, Operators, Query Processor >Affects Versions: 4.0.0 >Reporter: Nita Dembla >Assignee: Gopal Vijayaraghavan >Priority: Major > Labels: Performance > Attachments: HIVE-23423.1.patch, HIVE-23423.WIP.patch > > > https://issues.apache.org/jira/browse/HIVE-23356 fixed the issue with > disabling hash aggregation on grouping set queries. Need a fix for > VectorGroupbyOperator operator. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23432) Add Ranger Replication Metrics
[ https://issues.apache.org/jira/browse/HIVE-23432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-23432: --- Attachment: HIVE-23432.07.patch Status: Patch Available (was: In Progress) > Add Ranger Replication Metrics > --- > > Key: HIVE-23432 > URL: https://issues.apache.org/jira/browse/HIVE-23432 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Attachments: HIVE-23432.01.patch, HIVE-23432.02.patch, > HIVE-23432.03.patch, HIVE-23432.04.patch, HIVE-23432.05.patch, > HIVE-23432.06.patch, HIVE-23432.07.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23133) Numeric operations can have different result across hardware archs
[ https://issues.apache.org/jira/browse/HIVE-23133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenyu Zheng updated HIVE-23133: Attachment: HIVE-23133.6.patch Status: Patch Available (was: Open) > Numeric operations can have different result across hardware archs > -- > > Key: HIVE-23133 > URL: https://issues.apache.org/jira/browse/HIVE-23133 > Project: Hive > Issue Type: Sub-task >Reporter: Zhenyu Zheng >Assignee: Zhenyu Zheng >Priority: Major > Attachments: HIVE-23133.1.patch, HIVE-23133.2.patch, > HIVE-23133.3.patch, HIVE-23133.4.patch, HIVE-23133.5.patch, HIVE-23133.6.patch > > > Use java.lang.StrictMath rather than java.lang.Math in UDFs: > UDFCos.java > UDFExp.java > UDFLn.java > UDFLog.java > > Currently, we have set up an ARM CI to test out how Hive works on ARM > platform: > [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/] > Among the failures, we have observed that some numeric operations can have > different result across hardware archs, such as: > [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/25/testReport/org.apache.hadoop.hive.cli/TestCliDriver/testCliDriver_vector_decimal_udf2_/] > [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/25/testReport/org.apache.hadoop.hive.cli/TestSparkCliDriver/testCliDriver_subquery_select_/] > [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/25/testReport/org.apache.hadoop.hive.cli/TestSparkCliDriver/testCliDriver_vectorized_math_funcs_/] > we can see that the calculation results of log, exp, cos, toRadians etc is > slitly different than the .out file results that we are > comparing(they are tested and wrote on X86 machines), this is because of we > use [Math > Library|https://docs.oracle.com/javase/6/docs/api/java/lang/Math.html] for > these kind of calculations. > and according to the > [illustration|https://docs.oracle.com/javase/6/docs/api/java/lang/Math.html]: > _Unlike some of the numeric methods of class StrictMath, all implementations > of the equivalent functions of class Math are not_ > _defined to return the bit-for-bit same results. This relaxation permits > better-performing implementations where strict reproducibility_ > _is not required._ > _By default many of the Math methods simply call the equivalent method in > StrictMath for their implementation._ > _Code generators are encouraged to use platform-specific native libraries or > microprocessor instructions, where available,_ > _to provide higher-performance implementations of Math methods._ > so the result will have difference across hardware archs. > On the other hand, JAVA provided another library > [StrictMath|https://docs.oracle.com/javase/6/docs/api/java/lang/StrictMath.html] > that will not have this kind of problem as according to its' > [reference|https://docs.oracle.com/javase/6/docs/api/java/lang/StrictMath.html]: > _To help ensure portability of Java programs, the definitions of some of the > numeric functions in this package require that they produce_ > _the same results as certain published algorithms._ > So in order to fix the above mentioned problem, we have to consider switch to > use StrictMath instead of Math. > > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23133) Numeric operations can have different result across hardware archs
[ https://issues.apache.org/jira/browse/HIVE-23133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenyu Zheng updated HIVE-23133: Status: Open (was: Patch Available) > Numeric operations can have different result across hardware archs > -- > > Key: HIVE-23133 > URL: https://issues.apache.org/jira/browse/HIVE-23133 > Project: Hive > Issue Type: Sub-task >Reporter: Zhenyu Zheng >Assignee: Zhenyu Zheng >Priority: Major > Attachments: HIVE-23133.1.patch, HIVE-23133.2.patch, > HIVE-23133.3.patch, HIVE-23133.4.patch, HIVE-23133.5.patch, HIVE-23133.6.patch > > > Use java.lang.StrictMath rather than java.lang.Math in UDFs: > UDFCos.java > UDFExp.java > UDFLn.java > UDFLog.java > > Currently, we have set up an ARM CI to test out how Hive works on ARM > platform: > [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/] > Among the failures, we have observed that some numeric operations can have > different result across hardware archs, such as: > [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/25/testReport/org.apache.hadoop.hive.cli/TestCliDriver/testCliDriver_vector_decimal_udf2_/] > [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/25/testReport/org.apache.hadoop.hive.cli/TestSparkCliDriver/testCliDriver_subquery_select_/] > [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/25/testReport/org.apache.hadoop.hive.cli/TestSparkCliDriver/testCliDriver_vectorized_math_funcs_/] > we can see that the calculation results of log, exp, cos, toRadians etc is > slitly different than the .out file results that we are > comparing(they are tested and wrote on X86 machines), this is because of we > use [Math > Library|https://docs.oracle.com/javase/6/docs/api/java/lang/Math.html] for > these kind of calculations. > and according to the > [illustration|https://docs.oracle.com/javase/6/docs/api/java/lang/Math.html]: > _Unlike some of the numeric methods of class StrictMath, all implementations > of the equivalent functions of class Math are not_ > _defined to return the bit-for-bit same results. This relaxation permits > better-performing implementations where strict reproducibility_ > _is not required._ > _By default many of the Math methods simply call the equivalent method in > StrictMath for their implementation._ > _Code generators are encouraged to use platform-specific native libraries or > microprocessor instructions, where available,_ > _to provide higher-performance implementations of Math methods._ > so the result will have difference across hardware archs. > On the other hand, JAVA provided another library > [StrictMath|https://docs.oracle.com/javase/6/docs/api/java/lang/StrictMath.html] > that will not have this kind of problem as according to its' > [reference|https://docs.oracle.com/javase/6/docs/api/java/lang/StrictMath.html]: > _To help ensure portability of Java programs, the definitions of some of the > numeric functions in this package require that they produce_ > _the same results as certain published algorithms._ > So in order to fix the above mentioned problem, we have to consider switch to > use StrictMath instead of Math. > > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23133) Numeric operations can have different result across hardware archs
[ https://issues.apache.org/jira/browse/HIVE-23133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenyu Zheng updated HIVE-23133: Description: Use java.lang.StrictMath rather than java.lang.Math in UDFs: UDFCos.java UDFExp.java UDFLn.java UDFLog.java UDFLog2.java UDFLog10.java Currently, we have set up an ARM CI to test out how Hive works on ARM platform: [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/] Among the failures, we have observed that some numeric operations can have different result across hardware archs, such as: [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/25/testReport/org.apache.hadoop.hive.cli/TestCliDriver/testCliDriver_vector_decimal_udf2_/] [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/25/testReport/org.apache.hadoop.hive.cli/TestSparkCliDriver/testCliDriver_subquery_select_/] [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/25/testReport/org.apache.hadoop.hive.cli/TestSparkCliDriver/testCliDriver_vectorized_math_funcs_/] we can see that the calculation results of log, exp, cos, toRadians etc is slitly different than the .out file results that we are comparing(they are tested and wrote on X86 machines), this is because of we use [Math Library|https://docs.oracle.com/javase/6/docs/api/java/lang/Math.html] for these kind of calculations. and according to the [illustration|https://docs.oracle.com/javase/6/docs/api/java/lang/Math.html]: _Unlike some of the numeric methods of class StrictMath, all implementations of the equivalent functions of class Math are not_ _defined to return the bit-for-bit same results. This relaxation permits better-performing implementations where strict reproducibility_ _is not required._ _By default many of the Math methods simply call the equivalent method in StrictMath for their implementation._ _Code generators are encouraged to use platform-specific native libraries or microprocessor instructions, where available,_ _to provide higher-performance implementations of Math methods._ so the result will have difference across hardware archs. On the other hand, JAVA provided another library [StrictMath|https://docs.oracle.com/javase/6/docs/api/java/lang/StrictMath.html] that will not have this kind of problem as according to its' [reference|https://docs.oracle.com/javase/6/docs/api/java/lang/StrictMath.html]: _To help ensure portability of Java programs, the definitions of some of the numeric functions in this package require that they produce_ _the same results as certain published algorithms._ So in order to fix the above mentioned problem, we have to consider switch to use StrictMath instead of Math. was: Use java.lang.StrictMath rather than java.lang.Math in UDFs: UDFCos.java UDFExp.java UDFLn.java UDFLog.java Currently, we have set up an ARM CI to test out how Hive works on ARM platform: [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/] Among the failures, we have observed that some numeric operations can have different result across hardware archs, such as: [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/25/testReport/org.apache.hadoop.hive.cli/TestCliDriver/testCliDriver_vector_decimal_udf2_/] [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/25/testReport/org.apache.hadoop.hive.cli/TestSparkCliDriver/testCliDriver_subquery_select_/] [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/25/testReport/org.apache.hadoop.hive.cli/TestSparkCliDriver/testCliDriver_vectorized_math_funcs_/] we can see that the calculation results of log, exp, cos, toRadians etc is slitly different than the .out file results that we are comparing(they are tested and wrote on X86 machines), this is because of we use [Math Library|https://docs.oracle.com/javase/6/docs/api/java/lang/Math.html] for these kind of calculations. and according to the [illustration|https://docs.oracle.com/javase/6/docs/api/java/lang/Math.html]: _Unlike some of the numeric methods of class StrictMath, all implementations of the equivalent functions of class Math are not_ _defined to return the bit-for-bit same results. This relaxation permits better-performing implementations where strict reproducibility_ _is not required._ _By default many of the Math methods simply call the equivalent method in StrictMath for their implementation._ _Code generators are encouraged to use platform-specific native libraries or microprocessor instructions, where available,_ _to provide higher-performance implementations of Math methods._ so the result will have difference across hardware archs. On the other hand, JAVA provided another library [StrictMath|https://docs.oracle.com/javase/6/docs/api/java/lang/StrictMath.html] that will not have this kind of problem as according to its'
[jira] [Updated] (HIVE-23458) Introduce unified thread pool for scheduled jobs
[ https://issues.apache.org/jira/browse/HIVE-23458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Chung updated HIVE-23458: Status: Open (was: Patch Available) > Introduce unified thread pool for scheduled jobs > > > Key: HIVE-23458 > URL: https://issues.apache.org/jira/browse/HIVE-23458 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Reporter: Eugene Chung >Assignee: Eugene Chung >Priority: Major > Labels: todoc4.0 > Fix For: 4.0.0 > > Attachments: HIVE-23458.01.patch > > > As I mentioned in [the comment of > HIVE-23164|https://issues.apache.org/jira/browse/HIVE-23164?focusedCommentId=17089506=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17089506], > I've made the unified scheduled executor service like > org.apache.hadoop.hive.metastore.ThreadPool. > I think it could help > 1. to minimize the possibility of making non-daemon threads when developers > need ScheduledExecutorService > 2. to achieve the utilization of server resources because the current > situation is all of the modules make its own ScheduledExecutorService and all > of the threads are just using for one job. > 3. administrators of Hive servers by providing > hive.exec.scheduler.num.threads configuration so that they can predict and > set how many threads are used and needed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23451) FileSinkOperator calls deleteOnExit (hdfs call) twice for the same file
[
https://issues.apache.org/jira/browse/HIVE-23451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106121#comment-17106121
]
Hive QA commented on HIVE-23451:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13002780/HIVE-23451.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 17268 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_llap_counters1]
(batchId=31)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_llap_counters]
(batchId=34)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_basic]
(batchId=29)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_schema_evol_3a]
(batchId=32)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22304/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22304/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22304/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13002780 - PreCommit-HIVE-Build
> FileSinkOperator calls deleteOnExit (hdfs call) twice for the same file
> ---
>
> Key: HIVE-23451
> URL: https://issues.apache.org/jira/browse/HIVE-23451
> Project: Hive
> Issue Type: Improvement
>Reporter: Rajesh Balamohan
>Assignee: Rajesh Balamohan
>Priority: Minor
> Attachments: HIVE-23451.1.patch, HIVE-23451.2.patch
>
>
> [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java#L826]
> [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java#L797]
> Can avoid a NN call here (i.e, mainly for small queries).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23458) Introduce unified thread pool for scheduled jobs
[
https://issues.apache.org/jira/browse/HIVE-23458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106123#comment-17106123
]
Hive QA commented on HIVE-23458:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13002787/HIVE-23458.01.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22305/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22305/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22305/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2020-05-13 08:53:12.303
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-22305/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2020-05-13 08:53:12.307
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 9ffbbdc HIVE-23409 : If TezSession application reopen fails for
Timeline service down, default TezSession from SessionPool is closed after a
retry ( Naresh PR via Ashutosh Chauhan)
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 9ffbbdc HIVE-23409 : If TezSession application reopen fails for
Timeline service down, default TezSession from SessionPool is closed after a
retry ( Naresh PR via Ashutosh Chauhan)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2020-05-13 08:53:13.294
+ rm -rf ../yetus_PreCommit-HIVE-Build-22305
+ mkdir ../yetus_PreCommit-HIVE-Build-22305
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-22305
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-22305/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
Trying to apply the patch with -p0
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/exec/tez/PerPoolTriggerValidatorRunnable.java:18
Falling back to three-way merge...
Applied patch to
'ql/src/java/org/apache/hadoop/hive/ql/exec/tez/PerPoolTriggerValidatorRunnable.java'
with conflicts.
error: patch failed:
service/src/java/org/apache/hive/service/server/HiveServer2.java:41
Falling back to three-way merge...
Applied patch to
'service/src/java/org/apache/hive/service/server/HiveServer2.java' cleanly.
Going to apply patch with: git apply -p0
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/exec/tez/PerPoolTriggerValidatorRunnable.java:18
Falling back to three-way merge...
Applied patch to
'ql/src/java/org/apache/hadoop/hive/ql/exec/tez/PerPoolTriggerValidatorRunnable.java'
with conflicts.
error: patch failed:
service/src/java/org/apache/hive/service/server/HiveServer2.java:41
Falling back to three-way merge...
Applied patch to
'service/src/java/org/apache/hive/service/server/HiveServer2.java' cleanly.
U
ql/src/java/org/apache/hadoop/hive/ql/exec/tez/PerPoolTriggerValidatorRunnable.java
+ result=1
+ '[' 1 -ne 0 ']'
+ rm -rf yetus_PreCommit-HIVE-Build-22305
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13002787 - PreCommit-HIVE-Build
> Introduce unified thread pool for scheduled jobs
>
>
> Key: HIVE-23458
> URL: https://issues.apache.org/jira/browse/HIVE-23458
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Reporter: Eugene Chung
>Assignee: Eugene Chung
>Priority: Major
> Labels: todoc4.0
> Fix For: 4.0.0
>
>
[jira] [Updated] (HIVE-23133) Numeric operations can have different result across hardware archs
[ https://issues.apache.org/jira/browse/HIVE-23133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenyu Zheng updated HIVE-23133: Description: Use java.lang.StrictMath rather than java.lang.Math in UDFs for compatibility: UDFCos.java UDFExp.java UDFLn.java UDFLog.java UDFLog2.java UDFLog10.java Currently, we have set up an ARM CI to test out how Hive works on ARM platform: [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/] Among the failures, we have observed that some numeric operations can have different result across hardware archs, such as: [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/25/testReport/org.apache.hadoop.hive.cli/TestCliDriver/testCliDriver_vector_decimal_udf2_/] [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/25/testReport/org.apache.hadoop.hive.cli/TestSparkCliDriver/testCliDriver_subquery_select_/] [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/25/testReport/org.apache.hadoop.hive.cli/TestSparkCliDriver/testCliDriver_vectorized_math_funcs_/] we can see that the calculation results of log, exp, cos, toRadians etc is slitly different than the .out file results that we are comparing(they are tested and wrote on X86 machines), this is because of we use [Math Library|https://docs.oracle.com/javase/6/docs/api/java/lang/Math.html] for these kind of calculations. and according to the [illustration|https://docs.oracle.com/javase/6/docs/api/java/lang/Math.html]: _Unlike some of the numeric methods of class StrictMath, all implementations of the equivalent functions of class Math are not_ _defined to return the bit-for-bit same results. This relaxation permits better-performing implementations where strict reproducibility_ _is not required._ _By default many of the Math methods simply call the equivalent method in StrictMath for their implementation._ _Code generators are encouraged to use platform-specific native libraries or microprocessor instructions, where available,_ _to provide higher-performance implementations of Math methods._ so the result will have difference across hardware archs. On the other hand, JAVA provided another library [StrictMath|https://docs.oracle.com/javase/6/docs/api/java/lang/StrictMath.html] that will not have this kind of problem as according to its' [reference|https://docs.oracle.com/javase/6/docs/api/java/lang/StrictMath.html]: _To help ensure portability of Java programs, the definitions of some of the numeric functions in this package require that they produce_ _the same results as certain published algorithms._ So in order to fix the above mentioned problem, we have to consider switch to use StrictMath instead of Math. was: Use java.lang.StrictMath rather than java.lang.Math in UDFs: UDFCos.java UDFExp.java UDFLn.java UDFLog.java UDFLog2.java UDFLog10.java Currently, we have set up an ARM CI to test out how Hive works on ARM platform: [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/] Among the failures, we have observed that some numeric operations can have different result across hardware archs, such as: [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/25/testReport/org.apache.hadoop.hive.cli/TestCliDriver/testCliDriver_vector_decimal_udf2_/] [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/25/testReport/org.apache.hadoop.hive.cli/TestSparkCliDriver/testCliDriver_subquery_select_/] [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/25/testReport/org.apache.hadoop.hive.cli/TestSparkCliDriver/testCliDriver_vectorized_math_funcs_/] we can see that the calculation results of log, exp, cos, toRadians etc is slitly different than the .out file results that we are comparing(they are tested and wrote on X86 machines), this is because of we use [Math Library|https://docs.oracle.com/javase/6/docs/api/java/lang/Math.html] for these kind of calculations. and according to the [illustration|https://docs.oracle.com/javase/6/docs/api/java/lang/Math.html]: _Unlike some of the numeric methods of class StrictMath, all implementations of the equivalent functions of class Math are not_ _defined to return the bit-for-bit same results. This relaxation permits better-performing implementations where strict reproducibility_ _is not required._ _By default many of the Math methods simply call the equivalent method in StrictMath for their implementation._ _Code generators are encouraged to use platform-specific native libraries or microprocessor instructions, where available,_ _to provide higher-performance implementations of Math methods._ so the result will have difference across hardware archs. On the other hand, JAVA provided another library [StrictMath|https://docs.oracle.com/javase/6/docs/api/java/lang/StrictMath.html] that will not have this
[jira] [Updated] (HIVE-23133) Numeric operations can have different result across hardware archs
[ https://issues.apache.org/jira/browse/HIVE-23133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenyu Zheng updated HIVE-23133: Description: Use java.lang.StrictMath rather than java.lang.Math for compatibility in the following UDFs: UDFCos.java UDFExp.java UDFLn.java UDFLog.java UDFLog2.java UDFLog10.java Currently, we have set up an ARM CI to test out how Hive works on ARM platform: [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/] Among the failures, we have observed that some numeric operations can have different result across hardware archs, such as: [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/25/testReport/org.apache.hadoop.hive.cli/TestCliDriver/testCliDriver_vector_decimal_udf2_/] [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/25/testReport/org.apache.hadoop.hive.cli/TestSparkCliDriver/testCliDriver_subquery_select_/] [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/25/testReport/org.apache.hadoop.hive.cli/TestSparkCliDriver/testCliDriver_vectorized_math_funcs_/] we can see that the calculation results of log, exp, cos, toRadians etc is slitly different than the .out file results that we are comparing(they are tested and wrote on X86 machines), this is because of we use [Math Library|https://docs.oracle.com/javase/6/docs/api/java/lang/Math.html] for these kind of calculations. and according to the [illustration|https://docs.oracle.com/javase/6/docs/api/java/lang/Math.html]: _Unlike some of the numeric methods of class StrictMath, all implementations of the equivalent functions of class Math are not_ _defined to return the bit-for-bit same results. This relaxation permits better-performing implementations where strict reproducibility_ _is not required._ _By default many of the Math methods simply call the equivalent method in StrictMath for their implementation._ _Code generators are encouraged to use platform-specific native libraries or microprocessor instructions, where available,_ _to provide higher-performance implementations of Math methods._ so the result will have difference across hardware archs. On the other hand, JAVA provided another library [StrictMath|https://docs.oracle.com/javase/6/docs/api/java/lang/StrictMath.html] that will not have this kind of problem as according to its' [reference|https://docs.oracle.com/javase/6/docs/api/java/lang/StrictMath.html]: _To help ensure portability of Java programs, the definitions of some of the numeric functions in this package require that they produce_ _the same results as certain published algorithms._ So in order to fix the above mentioned problem, we have to consider switch to use StrictMath instead of Math. was: Use java.lang.StrictMath rather than java.lang.Math in UDFs for compatibility: UDFCos.java UDFExp.java UDFLn.java UDFLog.java UDFLog2.java UDFLog10.java Currently, we have set up an ARM CI to test out how Hive works on ARM platform: [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/] Among the failures, we have observed that some numeric operations can have different result across hardware archs, such as: [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/25/testReport/org.apache.hadoop.hive.cli/TestCliDriver/testCliDriver_vector_decimal_udf2_/] [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/25/testReport/org.apache.hadoop.hive.cli/TestSparkCliDriver/testCliDriver_subquery_select_/] [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/25/testReport/org.apache.hadoop.hive.cli/TestSparkCliDriver/testCliDriver_vectorized_math_funcs_/] we can see that the calculation results of log, exp, cos, toRadians etc is slitly different than the .out file results that we are comparing(they are tested and wrote on X86 machines), this is because of we use [Math Library|https://docs.oracle.com/javase/6/docs/api/java/lang/Math.html] for these kind of calculations. and according to the [illustration|https://docs.oracle.com/javase/6/docs/api/java/lang/Math.html]: _Unlike some of the numeric methods of class StrictMath, all implementations of the equivalent functions of class Math are not_ _defined to return the bit-for-bit same results. This relaxation permits better-performing implementations where strict reproducibility_ _is not required._ _By default many of the Math methods simply call the equivalent method in StrictMath for their implementation._ _Code generators are encouraged to use platform-specific native libraries or microprocessor instructions, where available,_ _to provide higher-performance implementations of Math methods._ so the result will have difference across hardware archs. On the other hand, JAVA provided another library
[jira] [Updated] (HIVE-23460) Add qoption to disable qtests
[
https://issues.apache.org/jira/browse/HIVE-23460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Haindrich updated HIVE-23460:
Status: Patch Available (was: Open)
> Add qoption to disable qtests
> -
>
> Key: HIVE-23460
> URL: https://issues.apache.org/jira/browse/HIVE-23460
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-23460.01.patch
>
>
> instead other ways to exclude them... (testconfiguration.properties;
> CliConfig#excludeQuery)
> {code}
> --! qt:disabled:reason
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-23460) Add qoption to disable qtests
[
https://issues.apache.org/jira/browse/HIVE-23460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Haindrich reassigned HIVE-23460:
---
> Add qoption to disable qtests
> -
>
> Key: HIVE-23460
> URL: https://issues.apache.org/jira/browse/HIVE-23460
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-23460.01.patch
>
>
> instead other ways to exclude them... (testconfiguration.properties;
> CliConfig#excludeQuery)
> {code}
> --! qt:disabled:reason
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23460) Add qoption to disable qtests
[
https://issues.apache.org/jira/browse/HIVE-23460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Haindrich updated HIVE-23460:
Attachment: HIVE-23460.01.patch
> Add qoption to disable qtests
> -
>
> Key: HIVE-23460
> URL: https://issues.apache.org/jira/browse/HIVE-23460
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-23460.01.patch
>
>
> instead other ways to exclude them... (testconfiguration.properties;
> CliConfig#excludeQuery)
> {code}
> --! qt:disabled:reason
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
