[jira] [Commented] (HIVE-21954) QTest: support for running qtests on various metastore DBs
[
https://issues.apache.org/jira/browse/HIVE-21954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16985196#comment-16985196
]
László Bodor commented on HIVE-21954:
-
pushed to master, thanks [~kgyrtkirk] for the review!
a couple of examples using the new feature:
{code}
mvn test -Dtest.output.overwrite=true -Pitests -pl itests/qtest
-Dtest=TestCliDriver -Dqfile=partition_params_postgres.q
-Dtest.metastore.db=mssql
mvn test -Dtest.output.overwrite=true -Pitests -pl itests/qtest
-Dtest=TestCliDriver -Dqfile=partition_params_postgres.q
-Dtest.metastore.db=mysql
mvn test -Dtest.output.overwrite=true -Pitests -pl itests/qtest
-Dtest=TestCliDriver -Dqfile=partition_params_postgres.q
-Dtest.metastore.db=postgres
mvn test -Dtest.output.overwrite=true -Pitests -pl itests/qtest
-Dtest=TestCliDriver -Dqfile=partition_params_postgres.q
-Dtest.metastore.db=oracle
-Ditest.jdbc.jars=/path/to/your/god/damn/oracle/jdbc/driver/ojdbc6.jar
{code}
the data can be checked in metastore if you prevent it from cleaning up:
{code}
export QTEST_LEAVE_FILES=true
mvn ... -Dmetastore.itest.no.stop.container=true
{code}
remove all metastore docker containers (e.g. for making sure that you have a
clean environment for running new qtests with the feature)
{code}
docker ps -a -q --filter="name=metastore-test-.*-install" | xargs docker rm -f
{code}
> QTest: support for running qtests on various metastore DBs
> --
>
> Key: HIVE-21954
> URL: https://issues.apache.org/jira/browse/HIVE-21954
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore, Testing Infrastructure
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-21954.01.patch, HIVE-21954.02.patch,
> HIVE-21954.03.patch, HIVE-21954.03.patch, HIVE-21954.03.patch,
> HIVE-21954.04.patch, HIVE-21954.05.patch, HIVE-21954.07.patch,
> HIVE-21954.07.patch, HIVE-21954.08.patch, HIVE-21954.09.patch,
> HIVE-21954.10.patch, HIVE-21954.10.patch
>
>
> In HIVE-21940, a postgres metastore related issue has been fixed, and a local
> reproduction has been provided.
> {code}
> export QTEST_LEAVE_FILES=true
> docker kill metastore-test-postgres-install
> docker rm metastore-test-postgres-install
> cd standalone-metastore
> mvn verify -DskipITests=false -Dit.test=ITestPostgres#install -Dtest=nosuch
> -Dmetastore.itest.no.stop.container=true
> cd ..
> mvn test -Dtest.output.overwrite=true -Pitests,hadoop-2 -pl itests/qtest
> -Dtest=TestCliDriver -Dqfile=partition_params_postgres.q
> -Dhive.metastore.rawstore.impl=org.apache.hadoop.hive.metastore.ObjectStore
> {code}
> The problem with this solution is that data/conf/hive-site.xml has to be
> edited manually. My proposal is to introduce a property
> (-Dmetastore.db=postgres), which can take care of the parameters on the fly.
> 2 supported solutions could be:
> 1. simple parameters: -Dmetastore.db=postgres
> In this case, tests depend on settings from ITestPostgres class (password,
> db, etc.)
> 2. verbose but flexible parameters: [see hive-site.xml HIVE-21940's repro
> patch|https://issues.apache.org/jira/secure/attachment/12973534/HIVE-21940.repro.patch]
>
>
> In the long term, we should consider running this kind of tests in precommit
> phase, so maybe -Dmetastore.db=postgres could start metastore db
> automatically. Also we should consider running some qtests on various
> metastores. I would not pick randomly, but choose some "metastore-heavy" ones
> instead.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-21954) QTest: support for running qtests on various metastore DBs
[
https://issues.apache.org/jira/browse/HIVE-21954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16985133#comment-16985133
]
Hive QA commented on HIVE-21954:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
4s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m
31s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
15s{color} | {color:blue} standalone-metastore/metastore-server in master has
179 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
18s{color} | {color:blue} ql in master has 1534 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
43s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
54s{color} | {color:blue} itests/util in master has 53 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 11m
16s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 11m
38s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
24s{color} | {color:red} standalone-metastore: The patch generated 1 new + 104
unchanged - 6 fixed = 105 total (was 110) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
19s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 1 new + 104 unchanged - 6 fixed = 105 total (was 110) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m
19s{color} | {color:red} root: The patch generated 1 new + 800 unchanged - 6
fixed = 801 total (was 806) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
17s{color} | {color:red} patch/standalone-metastore/metastore-server cannot run
setBugDatabaseInfo from findbugs {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 5m
43s{color} | {color:red} patch/ql cannot run setBugDatabaseInfo from findbugs
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
28s{color} | {color:red} patch/itests/hive-unit cannot run setBugDatabaseInfo
from findbugs {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
49s{color} | {color:red} patch/itests/util cannot run setBugDatabaseInfo from
findbugs {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 11m
13s{color} | {color:red} root generated 9 new + 337 unchanged - 0 fixed = 346
total (was 337) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
24s{color} | {color:red} itests_util generated 9 new + 9 unchanged - 0 fixed =
18 total (was 9) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
22s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}104m 59s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile
xml |
| uname | Linux
[jira] [Commented] (HIVE-21954) QTest: support for running qtests on various metastore DBs
[
https://issues.apache.org/jira/browse/HIVE-21954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16985128#comment-16985128
]
Hive QA commented on HIVE-21954:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12987152/HIVE-21954.10.patch
{color:green}SUCCESS:{color} +1 due to 12 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17752 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19677/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19677/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19677/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12987152 - PreCommit-HIVE-Build
> QTest: support for running qtests on various metastore DBs
> --
>
> Key: HIVE-21954
> URL: https://issues.apache.org/jira/browse/HIVE-21954
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore, Testing Infrastructure
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-21954.01.patch, HIVE-21954.02.patch,
> HIVE-21954.03.patch, HIVE-21954.03.patch, HIVE-21954.03.patch,
> HIVE-21954.04.patch, HIVE-21954.05.patch, HIVE-21954.07.patch,
> HIVE-21954.07.patch, HIVE-21954.08.patch, HIVE-21954.09.patch,
> HIVE-21954.10.patch, HIVE-21954.10.patch
>
>
> In HIVE-21940, a postgres metastore related issue has been fixed, and a local
> reproduction has been provided.
> {code}
> export QTEST_LEAVE_FILES=true
> docker kill metastore-test-postgres-install
> docker rm metastore-test-postgres-install
> cd standalone-metastore
> mvn verify -DskipITests=false -Dit.test=ITestPostgres#install -Dtest=nosuch
> -Dmetastore.itest.no.stop.container=true
> cd ..
> mvn test -Dtest.output.overwrite=true -Pitests,hadoop-2 -pl itests/qtest
> -Dtest=TestCliDriver -Dqfile=partition_params_postgres.q
> -Dhive.metastore.rawstore.impl=org.apache.hadoop.hive.metastore.ObjectStore
> {code}
> The problem with this solution is that data/conf/hive-site.xml has to be
> edited manually. My proposal is to introduce a property
> (-Dmetastore.db=postgres), which can take care of the parameters on the fly.
> 2 supported solutions could be:
> 1. simple parameters: -Dmetastore.db=postgres
> In this case, tests depend on settings from ITestPostgres class (password,
> db, etc.)
> 2. verbose but flexible parameters: [see hive-site.xml HIVE-21940's repro
> patch|https://issues.apache.org/jira/secure/attachment/12973534/HIVE-21940.repro.patch]
>
> In the first implementation, I would not start metastore db automatically
> (which is done be 'mvn verify ...'), but it's still under planning.
> In the long term, we should consider running this kind of tests in precommit
> phase, so maybe -Dmetastore.db=postgres could start metastore db
> automatically. Also we should consider running some qtests on various
> metastores. I would not pick randomly, but choose some "metastore-heavy" ones
> instead.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-21954) QTest: support for running qtests on various metastore DBs
[
https://issues.apache.org/jira/browse/HIVE-21954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984731#comment-16984731
]
Hive QA commented on HIVE-21954:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12987052/HIVE-21954.10.patch
{color:green}SUCCESS:{color} +1 due to 12 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 13 failed/errored test(s), 17815 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_insert1_overwrite_partitions]
(batchId=2)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_insert2_overwrite_partitions]
(batchId=69)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition2]
(batchId=35)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition3]
(batchId=20)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition4]
(batchId=28)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition5]
(batchId=20)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition]
(batchId=66)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_partition_boolexpr]
(batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_partition_ctas]
(batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_partition_multilevels]
(batchId=100)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[temp_table_llap_partitioned]
(batchId=167)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[delete_non_acid_table]
(batchId=106)
org.apache.hive.service.server.TestHS2HttpServerPam.org.apache.hive.service.server.TestHS2HttpServerPam
(batchId=243)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19664/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19664/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19664/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 13 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12987052 - PreCommit-HIVE-Build
> QTest: support for running qtests on various metastore DBs
> --
>
> Key: HIVE-21954
> URL: https://issues.apache.org/jira/browse/HIVE-21954
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore, Testing Infrastructure
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-21954.01.patch, HIVE-21954.02.patch,
> HIVE-21954.03.patch, HIVE-21954.03.patch, HIVE-21954.03.patch,
> HIVE-21954.04.patch, HIVE-21954.05.patch, HIVE-21954.07.patch,
> HIVE-21954.07.patch, HIVE-21954.08.patch, HIVE-21954.09.patch,
> HIVE-21954.10.patch
>
>
> In HIVE-21940, a postgres metastore related issue has been fixed, and a local
> reproduction has been provided.
> {code}
> export QTEST_LEAVE_FILES=true
> docker kill metastore-test-postgres-install
> docker rm metastore-test-postgres-install
> cd standalone-metastore
> mvn verify -DskipITests=false -Dit.test=ITestPostgres#install -Dtest=nosuch
> -Dmetastore.itest.no.stop.container=true
> cd ..
> mvn test -Dtest.output.overwrite=true -Pitests,hadoop-2 -pl itests/qtest
> -Dtest=TestCliDriver -Dqfile=partition_params_postgres.q
> -Dhive.metastore.rawstore.impl=org.apache.hadoop.hive.metastore.ObjectStore
> {code}
> The problem with this solution is that data/conf/hive-site.xml has to be
> edited manually. My proposal is to introduce a property
> (-Dmetastore.db=postgres), which can take care of the parameters on the fly.
> 2 supported solutions could be:
> 1. simple parameters: -Dmetastore.db=postgres
> In this case, tests depend on settings from ITestPostgres class (password,
> db, etc.)
> 2. verbose but flexible parameters: [see hive-site.xml HIVE-21940's repro
> patch|https://issues.apache.org/jira/secure/attachment/12973534/HIVE-21940.repro.patch]
>
> In the first implementation, I would not start metastore db automatically
> (which is done be 'mvn verify ...'), but it's still under planning.
> In the long term, we should consider running this kind of tests in precommit
> phase, so maybe -Dmetastore.db=postgres could start metastore db
> automatically. Also we should consider running some qtests on various
> metastores. I would not pick
[jira] [Commented] (HIVE-21954) QTest: support for running qtests on various metastore DBs
[
https://issues.apache.org/jira/browse/HIVE-21954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984293#comment-16984293
]
Zoltan Haindrich commented on HIVE-21954:
-
+1
> QTest: support for running qtests on various metastore DBs
> --
>
> Key: HIVE-21954
> URL: https://issues.apache.org/jira/browse/HIVE-21954
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore, Testing Infrastructure
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-21954.01.patch, HIVE-21954.02.patch,
> HIVE-21954.03.patch, HIVE-21954.03.patch, HIVE-21954.03.patch,
> HIVE-21954.04.patch, HIVE-21954.05.patch, HIVE-21954.07.patch,
> HIVE-21954.07.patch, HIVE-21954.08.patch, HIVE-21954.09.patch
>
>
> In HIVE-21940, a postgres metastore related issue has been fixed, and a local
> reproduction has been provided.
> {code}
> export QTEST_LEAVE_FILES=true
> docker kill metastore-test-postgres-install
> docker rm metastore-test-postgres-install
> cd standalone-metastore
> mvn verify -DskipITests=false -Dit.test=ITestPostgres#install -Dtest=nosuch
> -Dmetastore.itest.no.stop.container=true
> cd ..
> mvn test -Dtest.output.overwrite=true -Pitests,hadoop-2 -pl itests/qtest
> -Dtest=TestCliDriver -Dqfile=partition_params_postgres.q
> -Dhive.metastore.rawstore.impl=org.apache.hadoop.hive.metastore.ObjectStore
> {code}
> The problem with this solution is that data/conf/hive-site.xml has to be
> edited manually. My proposal is to introduce a property
> (-Dmetastore.db=postgres), which can take care of the parameters on the fly.
> 2 supported solutions could be:
> 1. simple parameters: -Dmetastore.db=postgres
> In this case, tests depend on settings from ITestPostgres class (password,
> db, etc.)
> 2. verbose but flexible parameters: [see hive-site.xml HIVE-21940's repro
> patch|https://issues.apache.org/jira/secure/attachment/12973534/HIVE-21940.repro.patch]
>
> In the first implementation, I would not start metastore db automatically
> (which is done be 'mvn verify ...'), but it's still under planning.
> In the long term, we should consider running this kind of tests in precommit
> phase, so maybe -Dmetastore.db=postgres could start metastore db
> automatically. Also we should consider running some qtests on various
> metastores. I would not pick randomly, but choose some "metastore-heavy" ones
> instead.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-21954) QTest: support for running qtests on various metastore DBs
[
https://issues.apache.org/jira/browse/HIVE-21954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16970812#comment-16970812
]
Hive QA commented on HIVE-21954:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
0s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
10s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m
24s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
10s{color} | {color:blue} standalone-metastore/metastore-server in master has
173 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
16s{color} | {color:blue} ql in master has 1549 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
42s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
51s{color} | {color:blue} itests/util in master has 53 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 10m
28s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 12m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 10m
52s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
24s{color} | {color:red} standalone-metastore: The patch generated 1 new + 104
unchanged - 6 fixed = 105 total (was 110) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
18s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 1 new + 104 unchanged - 6 fixed = 105 total (was 110) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m
10s{color} | {color:red} root: The patch generated 1 new + 800 unchanged - 6
fixed = 801 total (was 806) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
4s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 7m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 10m
43s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 92m 9s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense xml javac javadoc findbugs checkstyle
compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19367/dev-support/hive-personality.sh
|
| git revision | master / 469ef5c |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19367/yetus/diff-checkstyle-standalone-metastore.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19367/yetus/diff-checkstyle-standalone-metastore_metastore-server.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19367/yetus/diff-checkstyle-root.txt
|
| modules | C: standalone-metastore standalone-metastore/metastore-server ql .
itests/hive-unit itests/qtest
[jira] [Commented] (HIVE-21954) QTest: support for running qtests on various metastore DBs
[
https://issues.apache.org/jira/browse/HIVE-21954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16970807#comment-16970807
]
Hive QA commented on HIVE-21954:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12985420/HIVE-21954.09.patch
{color:green}SUCCESS:{color} +1 due to 12 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17669 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19367/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19367/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19367/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12985420 - PreCommit-HIVE-Build
> QTest: support for running qtests on various metastore DBs
> --
>
> Key: HIVE-21954
> URL: https://issues.apache.org/jira/browse/HIVE-21954
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore, Testing Infrastructure
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-21954.01.patch, HIVE-21954.02.patch,
> HIVE-21954.03.patch, HIVE-21954.03.patch, HIVE-21954.03.patch,
> HIVE-21954.04.patch, HIVE-21954.05.patch, HIVE-21954.07.patch,
> HIVE-21954.07.patch, HIVE-21954.08.patch, HIVE-21954.09.patch
>
>
> In HIVE-21940, a postgres metastore related issue has been fixed, and a local
> reproduction has been provided.
> {code}
> export QTEST_LEAVE_FILES=true
> docker kill metastore-test-postgres-install
> docker rm metastore-test-postgres-install
> cd standalone-metastore
> mvn verify -DskipITests=false -Dit.test=ITestPostgres#install -Dtest=nosuch
> -Dmetastore.itest.no.stop.container=true
> cd ..
> mvn test -Dtest.output.overwrite=true -Pitests,hadoop-2 -pl itests/qtest
> -Dtest=TestCliDriver -Dqfile=partition_params_postgres.q
> -Dhive.metastore.rawstore.impl=org.apache.hadoop.hive.metastore.ObjectStore
> {code}
> The problem with this solution is that data/conf/hive-site.xml has to be
> edited manually. My proposal is to introduce a property
> (-Dmetastore.db=postgres), which can take care of the parameters on the fly.
> 2 supported solutions could be:
> 1. simple parameters: -Dmetastore.db=postgres
> In this case, tests depend on settings from ITestPostgres class (password,
> db, etc.)
> 2. verbose but flexible parameters: [see hive-site.xml HIVE-21940's repro
> patch|https://issues.apache.org/jira/secure/attachment/12973534/HIVE-21940.repro.patch]
>
> In the first implementation, I would not start metastore db automatically
> (which is done be 'mvn verify ...'), but it's still under planning.
> In the long term, we should consider running this kind of tests in precommit
> phase, so maybe -Dmetastore.db=postgres could start metastore db
> automatically. Also we should consider running some qtests on various
> metastores. I would not pick randomly, but choose some "metastore-heavy" ones
> instead.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-21954) QTest: support for running qtests on various metastore DBs
[
https://issues.apache.org/jira/browse/HIVE-21954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16965365#comment-16965365
]
László Bodor commented on HIVE-21954:
-
[~kgyrtkirk], [~pvary]: could you please take a look? the patch contains the
following:
1. Database rule refactoring in standalone-metastore project
2. Qtest infrastructure: preparing qtest infra for being able to run any qtest
against any kind of metastores
3. A bunch of cleanups in the current qtest mess
With the patch, you can run the following (if your docker is set up properly,
meaning that you can run the metastore integration tests, already described in
standalone-metastore/DEV-README):
{code}
mvn test -Dtest.output.overwrite=true -Pitests -pl itests/qtest
-Dtest=TestCliDriver -Dqfile=partition_params_postgres.q
-Dtest.metastore.db=mssql
mvn test -Dtest.output.overwrite=true -Pitests -pl itests/qtest
-Dtest=TestCliDriver -Dqfile=partition_params_postgres.q
-Dtest.metastore.db=mysql
mvn test -Dtest.output.overwrite=true -Pitests -pl itests/qtest
-Dtest=TestCliDriver -Dqfile=partition_params_postgres.q
-Dtest.metastore.db=postgres
mvn test -Dtest.output.overwrite=true -Pitests -pl itests/qtest
-Dtest=TestCliDriver -Dqfile=partition_params_postgres.q
-Dtest.metastore.db=oracle
-Ditest.jdbc.jars=/path/to/your/god/damn/oracle/jdbc/driver/ojdbc6.jar
{code}
> QTest: support for running qtests on various metastore DBs
> --
>
> Key: HIVE-21954
> URL: https://issues.apache.org/jira/browse/HIVE-21954
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore, Testing Infrastructure
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-21954.01.patch, HIVE-21954.02.patch,
> HIVE-21954.03.patch, HIVE-21954.03.patch, HIVE-21954.03.patch,
> HIVE-21954.04.patch, HIVE-21954.05.patch, HIVE-21954.07.patch,
> HIVE-21954.07.patch, HIVE-21954.08.patch
>
>
> In HIVE-21940, a postgres metastore related issue has been fixed, and a local
> reproduction has been provided.
> {code}
> export QTEST_LEAVE_FILES=true
> docker kill metastore-test-postgres-install
> docker rm metastore-test-postgres-install
> cd standalone-metastore
> mvn verify -DskipITests=false -Dit.test=ITestPostgres#install -Dtest=nosuch
> -Dmetastore.itest.no.stop.container=true
> cd ..
> mvn test -Dtest.output.overwrite=true -Pitests,hadoop-2 -pl itests/qtest
> -Dtest=TestCliDriver -Dqfile=partition_params_postgres.q
> -Dhive.metastore.rawstore.impl=org.apache.hadoop.hive.metastore.ObjectStore
> {code}
> The problem with this solution is that data/conf/hive-site.xml has to be
> edited manually. My proposal is to introduce a property
> (-Dmetastore.db=postgres), which can take care of the parameters on the fly.
> 2 supported solutions could be:
> 1. simple parameters: -Dmetastore.db=postgres
> In this case, tests depend on settings from ITestPostgres class (password,
> db, etc.)
> 2. verbose but flexible parameters: [see hive-site.xml HIVE-21940's repro
> patch|https://issues.apache.org/jira/secure/attachment/12973534/HIVE-21940.repro.patch]
>
> In the first implementation, I would not start metastore db automatically
> (which is done be 'mvn verify ...'), but it's still under planning.
> In the long term, we should consider running this kind of tests in precommit
> phase, so maybe -Dmetastore.db=postgres could start metastore db
> automatically. Also we should consider running some qtests on various
> metastores. I would not pick randomly, but choose some "metastore-heavy" ones
> instead.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-21954) QTest: support for running qtests on various metastore DBs
[
https://issues.apache.org/jira/browse/HIVE-21954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16965362#comment-16965362
]
Hive QA commented on HIVE-21954:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
48s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m
38s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
17s{color} | {color:blue} standalone-metastore/metastore-server in master has
171 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
11s{color} | {color:blue} ql in master has 1545 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
46s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
50s{color} | {color:blue} itests/util in master has 53 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 11m
17s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 12m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 11m
51s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
25s{color} | {color:red} standalone-metastore: The patch generated 1 new + 104
unchanged - 6 fixed = 105 total (was 110) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
18s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 1 new + 104 unchanged - 6 fixed = 105 total (was 110) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m
24s{color} | {color:red} root: The patch generated 1 new + 801 unchanged - 6
fixed = 802 total (was 807) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
4s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 7m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 11m
18s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 95m 34s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense xml javac javadoc findbugs checkstyle
compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19265/dev-support/hive-personality.sh
|
| git revision | master / 1c2b7fa |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19265/yetus/diff-checkstyle-standalone-metastore.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19265/yetus/diff-checkstyle-standalone-metastore_metastore-server.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19265/yetus/diff-checkstyle-root.txt
|
| modules | C: standalone-metastore standalone-metastore/metastore-server ql .
itests/hive-unit itests/qtest
[jira] [Commented] (HIVE-21954) QTest: support for running qtests on various metastore DBs
[
https://issues.apache.org/jira/browse/HIVE-21954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16965350#comment-16965350
]
Hive QA commented on HIVE-21954:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12984680/HIVE-21954.08.patch
{color:green}SUCCESS:{color} +1 due to 12 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17547 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19265/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19265/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19265/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12984680 - PreCommit-HIVE-Build
> QTest: support for running qtests on various metastore DBs
> --
>
> Key: HIVE-21954
> URL: https://issues.apache.org/jira/browse/HIVE-21954
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore, Testing Infrastructure
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-21954.01.patch, HIVE-21954.02.patch,
> HIVE-21954.03.patch, HIVE-21954.03.patch, HIVE-21954.03.patch,
> HIVE-21954.04.patch, HIVE-21954.05.patch, HIVE-21954.07.patch,
> HIVE-21954.07.patch, HIVE-21954.08.patch
>
>
> In HIVE-21940, a postgres metastore related issue has been fixed, and a local
> reproduction has been provided.
> {code}
> export QTEST_LEAVE_FILES=true
> docker kill metastore-test-postgres-install
> docker rm metastore-test-postgres-install
> cd standalone-metastore
> mvn verify -DskipITests=false -Dit.test=ITestPostgres#install -Dtest=nosuch
> -Dmetastore.itest.no.stop.container=true
> cd ..
> mvn test -Dtest.output.overwrite=true -Pitests,hadoop-2 -pl itests/qtest
> -Dtest=TestCliDriver -Dqfile=partition_params_postgres.q
> -Dhive.metastore.rawstore.impl=org.apache.hadoop.hive.metastore.ObjectStore
> {code}
> The problem with this solution is that data/conf/hive-site.xml has to be
> edited manually. My proposal is to introduce a property
> (-Dmetastore.db=postgres), which can take care of the parameters on the fly.
> 2 supported solutions could be:
> 1. simple parameters: -Dmetastore.db=postgres
> In this case, tests depend on settings from ITestPostgres class (password,
> db, etc.)
> 2. verbose but flexible parameters: [see hive-site.xml HIVE-21940's repro
> patch|https://issues.apache.org/jira/secure/attachment/12973534/HIVE-21940.repro.patch]
>
> In the first implementation, I would not start metastore db automatically
> (which is done be 'mvn verify ...'), but it's still under planning.
> In the long term, we should consider running this kind of tests in precommit
> phase, so maybe -Dmetastore.db=postgres could start metastore db
> automatically. Also we should consider running some qtests on various
> metastores. I would not pick randomly, but choose some "metastore-heavy" ones
> instead.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-21954) QTest: support for running qtests on various metastore DBs
[
https://issues.apache.org/jira/browse/HIVE-21954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16965166#comment-16965166
]
Hive QA commented on HIVE-21954:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
3s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
54s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m
27s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
10s{color} | {color:blue} standalone-metastore/metastore-server in master has
171 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
15s{color} | {color:blue} ql in master has 1545 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
45s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
51s{color} | {color:blue} itests/util in master has 53 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 10m
51s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 12m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 11m
31s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
23s{color} | {color:red} standalone-metastore: The patch generated 7 new + 104
unchanged - 6 fixed = 111 total (was 110) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
18s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 7 new + 104 unchanged - 6 fixed = 111 total (was 110) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m
17s{color} | {color:red} root: The patch generated 7 new + 801 unchanged - 6
fixed = 808 total (was 807) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
4s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
10s{color} | {color:red} patch/standalone-metastore/metastore-server cannot run
setBugDatabaseInfo from findbugs {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 5m
22s{color} | {color:red} patch/ql cannot run setBugDatabaseInfo from findbugs
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
42s{color} | {color:red} patch/itests/hive-unit cannot run setBugDatabaseInfo
from findbugs {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
50s{color} | {color:red} patch/itests/util cannot run setBugDatabaseInfo from
findbugs {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 10m
9s{color} | {color:red} root generated 9 new + 337 unchanged - 0 fixed = 346
total (was 337) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
24s{color} | {color:red} itests_util generated 9 new + 9 unchanged - 0 fixed =
18 total (was 9) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
25s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}103m 5s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense xml javac javadoc findbugs checkstyle
compile |
| uname | Linux
[jira] [Commented] (HIVE-21954) QTest: support for running qtests on various metastore DBs
[
https://issues.apache.org/jira/browse/HIVE-21954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16965159#comment-16965159
]
Hive QA commented on HIVE-21954:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12984628/HIVE-21954.07.patch
{color:green}SUCCESS:{color} +1 due to 12 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17547 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=171)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19255/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19255/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19255/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12984628 - PreCommit-HIVE-Build
> QTest: support for running qtests on various metastore DBs
> --
>
> Key: HIVE-21954
> URL: https://issues.apache.org/jira/browse/HIVE-21954
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore, Testing Infrastructure
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-21954.01.patch, HIVE-21954.02.patch,
> HIVE-21954.03.patch, HIVE-21954.03.patch, HIVE-21954.03.patch,
> HIVE-21954.04.patch, HIVE-21954.05.patch, HIVE-21954.07.patch,
> HIVE-21954.07.patch
>
>
> In HIVE-21940, a postgres metastore related issue has been fixed, and a local
> reproduction has been provided.
> {code}
> export QTEST_LEAVE_FILES=true
> docker kill metastore-test-postgres-install
> docker rm metastore-test-postgres-install
> cd standalone-metastore
> mvn verify -DskipITests=false -Dit.test=ITestPostgres#install -Dtest=nosuch
> -Dmetastore.itest.no.stop.container=true
> cd ..
> mvn test -Dtest.output.overwrite=true -Pitests,hadoop-2 -pl itests/qtest
> -Dtest=TestCliDriver -Dqfile=partition_params_postgres.q
> -Dhive.metastore.rawstore.impl=org.apache.hadoop.hive.metastore.ObjectStore
> {code}
> The problem with this solution is that data/conf/hive-site.xml has to be
> edited manually. My proposal is to introduce a property
> (-Dmetastore.db=postgres), which can take care of the parameters on the fly.
> 2 supported solutions could be:
> 1. simple parameters: -Dmetastore.db=postgres
> In this case, tests depend on settings from ITestPostgres class (password,
> db, etc.)
> 2. verbose but flexible parameters: [see hive-site.xml HIVE-21940's repro
> patch|https://issues.apache.org/jira/secure/attachment/12973534/HIVE-21940.repro.patch]
>
> In the first implementation, I would not start metastore db automatically
> (which is done be 'mvn verify ...'), but it's still under planning.
> In the long term, we should consider running this kind of tests in precommit
> phase, so maybe -Dmetastore.db=postgres could start metastore db
> automatically. Also we should consider running some qtests on various
> metastores. I would not pick randomly, but choose some "metastore-heavy" ones
> instead.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-21954) QTest: support for running qtests on various metastore DBs
[
https://issues.apache.org/jira/browse/HIVE-21954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16964411#comment-16964411
]
Hive QA commented on HIVE-21954:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12984481/HIVE-21954.07.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19238/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19238/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19238/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12984481/HIVE-21954.07.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12984481 - PreCommit-HIVE-Build
> QTest: support for running qtests on various metastore DBs
> --
>
> Key: HIVE-21954
> URL: https://issues.apache.org/jira/browse/HIVE-21954
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore, Testing Infrastructure
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-21954.01.patch, HIVE-21954.02.patch,
> HIVE-21954.03.patch, HIVE-21954.03.patch, HIVE-21954.03.patch,
> HIVE-21954.04.patch, HIVE-21954.05.patch, HIVE-21954.07.patch
>
>
> In HIVE-21940, a postgres metastore related issue has been fixed, and a local
> reproduction has been provided.
> {code}
> export QTEST_LEAVE_FILES=true
> docker kill metastore-test-postgres-install
> docker rm metastore-test-postgres-install
> cd standalone-metastore
> mvn verify -DskipITests=false -Dit.test=ITestPostgres#install -Dtest=nosuch
> -Dmetastore.itest.no.stop.container=true
> cd ..
> mvn test -Dtest.output.overwrite=true -Pitests,hadoop-2 -pl itests/qtest
> -Dtest=TestCliDriver -Dqfile=partition_params_postgres.q
> -Dhive.metastore.rawstore.impl=org.apache.hadoop.hive.metastore.ObjectStore
> {code}
> The problem with this solution is that data/conf/hive-site.xml has to be
> edited manually. My proposal is to introduce a property
> (-Dmetastore.db=postgres), which can take care of the parameters on the fly.
> 2 supported solutions could be:
> 1. simple parameters: -Dmetastore.db=postgres
> In this case, tests depend on settings from ITestPostgres class (password,
> db, etc.)
> 2. verbose but flexible parameters: [see hive-site.xml HIVE-21940's repro
> patch|https://issues.apache.org/jira/secure/attachment/12973534/HIVE-21940.repro.patch]
>
> In the first implementation, I would not start metastore db automatically
> (which is done be 'mvn verify ...'), but it's still under planning.
> In the long term, we should consider running this kind of tests in precommit
> phase, so maybe -Dmetastore.db=postgres could start metastore db
> automatically. Also we should consider running some qtests on various
> metastores. I would not pick randomly, but choose some "metastore-heavy" ones
> instead.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-21954) QTest: support for running qtests on various metastore DBs
[
https://issues.apache.org/jira/browse/HIVE-21954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16964171#comment-16964171
]
Hive QA commented on HIVE-21954:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12984481/HIVE-21954.07.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19230/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19230/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19230/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2019-10-31 16:05:55.335
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-19230/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2019-10-31 16:05:55.339
+ cd apache-github-source-source
+ git fetch origin
>From https://github.com/apache/hive
ca4c0a2..82d32db master -> origin/master
+ git reset --hard HEAD
HEAD is now at ca4c0a2 HIVE-22431: Hive JDBC Storage Handler:
java.lang.ClassCastException on accessing TINYINT, SMALLINT Data Type From JDBC
Data Source (Syed Shameerur Rahman, reviewed by Jesus Camacho Rodriguez)
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is behind 'origin/master' by 1 commit, and can be fast-forwarded.
(use "git pull" to update your local branch)
+ git reset --hard origin/master
HEAD is now at 82d32db HIVE-22403: Print ENV Variables In Command Line Help
Debug Mode (David Mollitor, reviewed by Peter Vary)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2019-10-31 16:05:56.489
+ rm -rf ../yetus_PreCommit-HIVE-Build-19230
+ mkdir ../yetus_PreCommit-HIVE-Build-19230
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-19230
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-19230/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
Going to apply patch with: git apply -p0
/data/hiveptest/working/scratch/build.patch:1092: trailing whitespace.
# col_name data_type comment
/data/hiveptest/working/scratch/build.patch:1093: trailing whitespace.
col1int
/data/hiveptest/working/scratch/build.patch:1094: trailing whitespace.
col3int
/data/hiveptest/working/scratch/build.patch:1095: trailing whitespace.
/data/hiveptest/working/scratch/build.patch:1096: trailing whitespace.
# Partition Information
warning: squelched 28 whitespace errors
warning: 33 lines add whitespace errors.
+ [[ maven == \m\a\v\e\n ]]
+ rm -rf /data/hiveptest/working/maven/org/apache/hive
+ mvn -B clean install -DskipTests -T 4 -q
-Dmaven.repo.local=/data/hiveptest/working/maven
protoc-jar: executing: [/tmp/protoc5631173531282180598.exe, --version]
libprotoc 2.5.0
protoc-jar: executing: [/tmp/protoc5631173531282180598.exe,
-I/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/src/main/protobuf/org/apache/hadoop/hive/metastore,
--java_out=/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/target/generated-sources,
/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/src/main/protobuf/org/apache/hadoop/hive/metastore/metastore.proto]
ANTLR Parser Generator Version 3.5.2
protoc-jar: executing: [/tmp/protoc7016061013611576857.exe, --version]
libprotoc 2.5.0
ANTLR Parser Generator Version 3.5.2
Output file
[jira] [Commented] (HIVE-21954) QTest: support for running qtests on various metastore DBs
[
https://issues.apache.org/jira/browse/HIVE-21954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16963218#comment-16963218
]
Hive QA commented on HIVE-21954:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
33s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
10s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
52s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
8s{color} | {color:blue} standalone-metastore/metastore-server in master has
171 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
0s{color} | {color:blue} ql in master has 1545 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
40s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
47s{color} | {color:blue} itests/util in master has 53 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 9m
52s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
25s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 11m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 10m
24s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
23s{color} | {color:red} standalone-metastore: The patch generated 7 new + 50
unchanged - 6 fixed = 57 total (was 56) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
17s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 7 new + 50 unchanged - 6 fixed = 57 total (was 56) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
51s{color} | {color:red} root: The patch generated 7 new + 104 unchanged - 6
fixed = 111 total (was 110) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
4s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
59s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
28s{color} | {color:red} itests_util generated 9 new + 9 unchanged - 0 fixed =
18 total (was 9) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 92m 34s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense xml javac javadoc findbugs checkstyle
compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19212/dev-support/hive-personality.sh
|
| git revision | master / 6a154ee |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19212/yetus/diff-checkstyle-standalone-metastore.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19212/yetus/diff-checkstyle-standalone-metastore_metastore-server.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19212/yetus/diff-checkstyle-root.txt
|
| javadoc |
[jira] [Commented] (HIVE-21954) QTest: support for running qtests on various metastore DBs
[
https://issues.apache.org/jira/browse/HIVE-21954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16963202#comment-16963202
]
Hive QA commented on HIVE-21954:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12984304/HIVE-21954.05.patch
{color:green}SUCCESS:{color} +1 due to 11 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17538 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestSparkNegativeCliDriver.org.apache.hadoop.hive.cli.TestSparkNegativeCliDriver
(batchId=301)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19212/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19212/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19212/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12984304 - PreCommit-HIVE-Build
> QTest: support for running qtests on various metastore DBs
> --
>
> Key: HIVE-21954
> URL: https://issues.apache.org/jira/browse/HIVE-21954
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore, Testing Infrastructure
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-21954.01.patch, HIVE-21954.02.patch,
> HIVE-21954.03.patch, HIVE-21954.03.patch, HIVE-21954.03.patch,
> HIVE-21954.04.patch, HIVE-21954.05.patch
>
>
> In HIVE-21940, a postgres metastore related issue has been fixed, and a local
> reproduction has been provided.
> {code}
> export QTEST_LEAVE_FILES=true
> docker kill metastore-test-postgres-install
> docker rm metastore-test-postgres-install
> cd standalone-metastore
> mvn verify -DskipITests=false -Dit.test=ITestPostgres#install -Dtest=nosuch
> -Dmetastore.itest.no.stop.container=true
> cd ..
> mvn test -Dtest.output.overwrite=true -Pitests,hadoop-2 -pl itests/qtest
> -Dtest=TestCliDriver -Dqfile=partition_params_postgres.q
> -Dhive.metastore.rawstore.impl=org.apache.hadoop.hive.metastore.ObjectStore
> {code}
> The problem with this solution is that data/conf/hive-site.xml has to be
> edited manually. My proposal is to introduce a property
> (-Dmetastore.db=postgres), which can take care of the parameters on the fly.
> 2 supported solutions could be:
> 1. simple parameters: -Dmetastore.db=postgres
> In this case, tests depend on settings from ITestPostgres class (password,
> db, etc.)
> 2. verbose but flexible parameters: [see hive-site.xml HIVE-21940's repro
> patch|https://issues.apache.org/jira/secure/attachment/12973534/HIVE-21940.repro.patch]
>
> In the first implementation, I would not start metastore db automatically
> (which is done be 'mvn verify ...'), but it's still under planning.
> In the long term, we should consider running this kind of tests in precommit
> phase, so maybe -Dmetastore.db=postgres could start metastore db
> automatically. Also we should consider running some qtests on various
> metastores. I would not pick randomly, but choose some "metastore-heavy" ones
> instead.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-21954) QTest: support for running qtests on various metastore DBs
[
https://issues.apache.org/jira/browse/HIVE-21954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16962292#comment-16962292
]
Hive QA commented on HIVE-21954:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12984251/HIVE-21954.04.patch
{color:green}SUCCESS:{color} +1 due to 11 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 17106 tests
executed
*Failed tests:*
{noformat}
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=299)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=299)
TestMinimrCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=299)
TestTezPerfCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=299)
TestTezPerfConstraintsCliDriver - did not produce a TEST-*.xml file (likely
timed out) (batchId=299)
org.apache.hadoop.hive.cli.TestSparkNegativeCliDriver.org.apache.hadoop.hive.cli.TestSparkNegativeCliDriver
(batchId=301)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcrossInstances.org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcrossInstances
(batchId=265)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19194/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19194/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19194/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12984251 - PreCommit-HIVE-Build
> QTest: support for running qtests on various metastore DBs
> --
>
> Key: HIVE-21954
> URL: https://issues.apache.org/jira/browse/HIVE-21954
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore, Testing Infrastructure
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-21954.01.patch, HIVE-21954.02.patch,
> HIVE-21954.03.patch, HIVE-21954.03.patch, HIVE-21954.03.patch,
> HIVE-21954.04.patch
>
>
> In HIVE-21940, a postgres metastore related issue has been fixed, and a local
> reproduction has been provided.
> {code}
> export QTEST_LEAVE_FILES=true
> docker kill metastore-test-postgres-install
> docker rm metastore-test-postgres-install
> cd standalone-metastore
> mvn verify -DskipITests=false -Dit.test=ITestPostgres#install -Dtest=nosuch
> -Dmetastore.itest.no.stop.container=true
> cd ..
> mvn test -Dtest.output.overwrite=true -Pitests,hadoop-2 -pl itests/qtest
> -Dtest=TestCliDriver -Dqfile=partition_params_postgres.q
> -Dhive.metastore.rawstore.impl=org.apache.hadoop.hive.metastore.ObjectStore
> {code}
> The problem with this solution is that data/conf/hive-site.xml has to be
> edited manually. My proposal is to introduce a property
> (-Dmetastore.db=postgres), which can take care of the parameters on the fly.
> 2 supported solutions could be:
> 1. simple parameters: -Dmetastore.db=postgres
> In this case, tests depend on settings from ITestPostgres class (password,
> db, etc.)
> 2. verbose but flexible parameters: [see hive-site.xml HIVE-21940's repro
> patch|https://issues.apache.org/jira/secure/attachment/12973534/HIVE-21940.repro.patch]
>
> In the first implementation, I would not start metastore db automatically
> (which is done be 'mvn verify ...'), but it's still under planning.
> In the long term, we should consider running this kind of tests in precommit
> phase, so maybe -Dmetastore.db=postgres could start metastore db
> automatically. Also we should consider running some qtests on various
> metastores. I would not pick randomly, but choose some "metastore-heavy" ones
> instead.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-21954) QTest: support for running qtests on various metastore DBs
[
https://issues.apache.org/jira/browse/HIVE-21954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16962290#comment-16962290
]
Hive QA commented on HIVE-21954:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
41s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m
39s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
54s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
10s{color} | {color:blue} standalone-metastore/metastore-server in master has
171 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
7s{color} | {color:blue} ql in master has 1545 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
42s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
52s{color} | {color:blue} itests/util in master has 53 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 10m
16s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
24s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 11m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 10m
22s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
21s{color} | {color:red} standalone-metastore: The patch generated 7 new + 50
unchanged - 6 fixed = 57 total (was 56) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
16s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 7 new + 50 unchanged - 6 fixed = 57 total (was 56) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
51s{color} | {color:red} root: The patch generated 7 new + 104 unchanged - 6
fixed = 111 total (was 110) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
5s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 7m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 10m
8s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 86m 3s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense xml javac javadoc findbugs checkstyle
compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19194/dev-support/hive-personality.sh
|
| git revision | master / aceb8b6 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19194/yetus/diff-checkstyle-standalone-metastore.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19194/yetus/diff-checkstyle-standalone-metastore_metastore-server.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19194/yetus/diff-checkstyle-root.txt
|
| modules | C: standalone-metastore standalone-metastore/metastore-server ql .
itests/hive-unit itests/qtest
[jira] [Commented] (HIVE-21954) QTest: support for running qtests on various metastore DBs
[
https://issues.apache.org/jira/browse/HIVE-21954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16961720#comment-16961720
]
Hive QA commented on HIVE-21954:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12984199/HIVE-21954.03.patch
{color:green}SUCCESS:{color} +1 due to 11 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 6 failed/errored test(s), 17134 tests
executed
*Failed tests:*
{noformat}
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=299)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=299)
TestMinimrCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=299)
TestTezPerfCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=299)
TestTezPerfConstraintsCliDriver - did not produce a TEST-*.xml file (likely
timed out) (batchId=299)
org.apache.hadoop.hive.cli.TestSparkNegativeCliDriver.org.apache.hadoop.hive.cli.TestSparkNegativeCliDriver
(batchId=301)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19185/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19185/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19185/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 6 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12984199 - PreCommit-HIVE-Build
> QTest: support for running qtests on various metastore DBs
> --
>
> Key: HIVE-21954
> URL: https://issues.apache.org/jira/browse/HIVE-21954
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore, Testing Infrastructure
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-21954.01.patch, HIVE-21954.02.patch,
> HIVE-21954.03.patch, HIVE-21954.03.patch, HIVE-21954.03.patch
>
>
> In HIVE-21940, a postgres metastore related issue has been fixed, and a local
> reproduction has been provided.
> {code}
> export QTEST_LEAVE_FILES=true
> docker kill metastore-test-postgres-install
> docker rm metastore-test-postgres-install
> cd standalone-metastore
> mvn verify -DskipITests=false -Dit.test=ITestPostgres#install -Dtest=nosuch
> -Dmetastore.itest.no.stop.container=true
> cd ..
> mvn test -Dtest.output.overwrite=true -Pitests,hadoop-2 -pl itests/qtest
> -Dtest=TestCliDriver -Dqfile=partition_params_postgres.q
> -Dhive.metastore.rawstore.impl=org.apache.hadoop.hive.metastore.ObjectStore
> {code}
> The problem with this solution is that data/conf/hive-site.xml has to be
> edited manually. My proposal is to introduce a property
> (-Dmetastore.db=postgres), which can take care of the parameters on the fly.
> 2 supported solutions could be:
> 1. simple parameters: -Dmetastore.db=postgres
> In this case, tests depend on settings from ITestPostgres class (password,
> db, etc.)
> 2. verbose but flexible parameters: [see hive-site.xml HIVE-21940's repro
> patch|https://issues.apache.org/jira/secure/attachment/12973534/HIVE-21940.repro.patch]
>
> In the first implementation, I would not start metastore db automatically
> (which is done be 'mvn verify ...'), but it's still under planning.
> In the long term, we should consider running this kind of tests in precommit
> phase, so maybe -Dmetastore.db=postgres could start metastore db
> automatically. Also we should consider running some qtests on various
> metastores. I would not pick randomly, but choose some "metastore-heavy" ones
> instead.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-21954) QTest: support for running qtests on various metastore DBs
[
https://issues.apache.org/jira/browse/HIVE-21954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16961713#comment-16961713
]
Hive QA commented on HIVE-21954:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
37s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
52s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
10s{color} | {color:blue} standalone-metastore/metastore-server in master has
171 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
58s{color} | {color:blue} ql in master has 1545 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
37s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
48s{color} | {color:blue} itests/util in master has 53 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 9m
46s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
23s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 11m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 10m
15s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
23s{color} | {color:red} standalone-metastore: The patch generated 7 new + 50
unchanged - 6 fixed = 57 total (was 56) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
17s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 7 new + 50 unchanged - 6 fixed = 57 total (was 56) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
50s{color} | {color:red} root: The patch generated 7 new + 104 unchanged - 6
fixed = 111 total (was 110) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
5s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 7m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 9m
47s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 83m 50s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense xml javac javadoc findbugs checkstyle
compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19185/dev-support/hive-personality.sh
|
| git revision | master / aceb8b6 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19185/yetus/diff-checkstyle-standalone-metastore.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19185/yetus/diff-checkstyle-standalone-metastore_metastore-server.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19185/yetus/diff-checkstyle-root.txt
|
| modules | C: standalone-metastore standalone-metastore/metastore-server ql .
itests/hive-unit itests/qtest
[jira] [Commented] (HIVE-21954) QTest: support for running qtests on various metastore DBs
[
https://issues.apache.org/jira/browse/HIVE-21954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16959132#comment-16959132
]
Hive QA commented on HIVE-21954:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
35s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
55s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
15s{color} | {color:blue} standalone-metastore/metastore-server in master has
171 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
0s{color} | {color:blue} ql in master has 1545 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
39s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
47s{color} | {color:blue} itests/util in master has 53 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 9m
49s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
24s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 11m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 10m
32s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
22s{color} | {color:red} standalone-metastore: The patch generated 7 new + 50
unchanged - 6 fixed = 57 total (was 56) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
18s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 7 new + 50 unchanged - 6 fixed = 57 total (was 56) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
52s{color} | {color:red} root: The patch generated 7 new + 104 unchanged - 6
fixed = 111 total (was 110) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
4s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 7m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 9m
44s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 85m 12s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense xml javac javadoc findbugs checkstyle
compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19144/dev-support/hive-personality.sh
|
| git revision | master / 0ff2354 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19144/yetus/diff-checkstyle-standalone-metastore.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19144/yetus/diff-checkstyle-standalone-metastore_metastore-server.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19144/yetus/diff-checkstyle-root.txt
|
| modules | C: standalone-metastore standalone-metastore/metastore-server ql .
itests/hive-unit itests/qtest
[jira] [Commented] (HIVE-21954) QTest: support for running qtests on various metastore DBs
[
https://issues.apache.org/jira/browse/HIVE-21954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16959134#comment-16959134
]
Hive QA commented on HIVE-21954:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12983918/HIVE-21954.03.patch
{color:green}SUCCESS:{color} +1 due to 11 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 6 failed/errored test(s), 17134 tests
executed
*Failed tests:*
{noformat}
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=299)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=299)
TestMinimrCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=299)
TestTezPerfCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=299)
TestTezPerfConstraintsCliDriver - did not produce a TEST-*.xml file (likely
timed out) (batchId=299)
org.apache.hadoop.hive.cli.TestSparkNegativeCliDriver.org.apache.hadoop.hive.cli.TestSparkNegativeCliDriver
(batchId=301)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19144/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19144/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19144/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 6 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12983918 - PreCommit-HIVE-Build
> QTest: support for running qtests on various metastore DBs
> --
>
> Key: HIVE-21954
> URL: https://issues.apache.org/jira/browse/HIVE-21954
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore, Testing Infrastructure
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-21954.01.patch, HIVE-21954.02.patch,
> HIVE-21954.03.patch, HIVE-21954.03.patch
>
>
> In HIVE-21940, a postgres metastore related issue has been fixed, and a local
> reproduction has been provided.
> {code}
> export QTEST_LEAVE_FILES=true
> docker kill metastore-test-postgres-install
> docker rm metastore-test-postgres-install
> cd standalone-metastore
> mvn verify -DskipITests=false -Dit.test=ITestPostgres#install -Dtest=nosuch
> -Dmetastore.itest.no.stop.container=true
> cd ..
> mvn test -Dtest.output.overwrite=true -Pitests,hadoop-2 -pl itests/qtest
> -Dtest=TestCliDriver -Dqfile=partition_params_postgres.q
> -Dhive.metastore.rawstore.impl=org.apache.hadoop.hive.metastore.ObjectStore
> {code}
> The problem with this solution is that data/conf/hive-site.xml has to be
> edited manually. My proposal is to introduce a property
> (-Dmetastore.db=postgres), which can take care of the parameters on the fly.
> 2 supported solutions could be:
> 1. simple parameters: -Dmetastore.db=postgres
> In this case, tests depend on settings from ITestPostgres class (password,
> db, etc.)
> 2. verbose but flexible parameters: [see hive-site.xml HIVE-21940's repro
> patch|https://issues.apache.org/jira/secure/attachment/12973534/HIVE-21940.repro.patch]
>
> In the first implementation, I would not start metastore db automatically
> (which is done be 'mvn verify ...'), but it's still under planning.
> In the long term, we should consider running this kind of tests in precommit
> phase, so maybe -Dmetastore.db=postgres could start metastore db
> automatically. Also we should consider running some qtests on various
> metastores. I would not pick randomly, but choose some "metastore-heavy" ones
> instead.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-21954) QTest: support for running qtests on various metastore DBs
[
https://issues.apache.org/jira/browse/HIVE-21954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16955645#comment-16955645
]
Hive QA commented on HIVE-21954:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12983554/HIVE-21954.03.patch
{color:green}SUCCESS:{color} +1 due to 11 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 17133 tests
executed
*Failed tests:*
{noformat}
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=299)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=299)
TestMinimrCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=299)
TestTezPerfCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=299)
TestTezPerfConstraintsCliDriver - did not produce a TEST-*.xml file (likely
timed out) (batchId=299)
org.apache.hadoop.hive.cli.TestSparkNegativeCliDriver.org.apache.hadoop.hive.cli.TestSparkNegativeCliDriver
(batchId=301)
org.apache.hadoop.hive.llap.security.TestLlapSignerImpl.testSigning
(batchId=364)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19080/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19080/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19080/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12983554 - PreCommit-HIVE-Build
> QTest: support for running qtests on various metastore DBs
> --
>
> Key: HIVE-21954
> URL: https://issues.apache.org/jira/browse/HIVE-21954
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore, Testing Infrastructure
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-21954.01.patch, HIVE-21954.02.patch,
> HIVE-21954.03.patch
>
>
> In HIVE-21940, a postgres metastore related issue has been fixed, and a local
> reproduction has been provided.
> {code}
> export QTEST_LEAVE_FILES=true
> docker kill metastore-test-postgres-install
> docker rm metastore-test-postgres-install
> cd standalone-metastore
> mvn verify -DskipITests=false -Dit.test=ITestPostgres#install -Dtest=nosuch
> -Dmetastore.itest.no.stop.container=true
> cd ..
> mvn test -Dtest.output.overwrite=true -Pitests,hadoop-2 -pl itests/qtest
> -Dtest=TestCliDriver -Dqfile=partition_params_postgres.q
> -Dhive.metastore.rawstore.impl=org.apache.hadoop.hive.metastore.ObjectStore
> {code}
> The problem with this solution is that data/conf/hive-site.xml has to be
> edited manually. My proposal is to introduce a property
> (-Dmetastore.db=postgres), which can take care of the parameters on the fly.
> 2 supported solutions could be:
> 1. simple parameters: -Dmetastore.db=postgres
> In this case, tests depend on settings from ITestPostgres class (password,
> db, etc.)
> 2. verbose but flexible parameters: [see hive-site.xml HIVE-21940's repro
> patch|https://issues.apache.org/jira/secure/attachment/12973534/HIVE-21940.repro.patch]
>
> In the first implementation, I would not start metastore db automatically
> (which is done be 'mvn verify ...'), but it's still under planning.
> In the long term, we should consider running this kind of tests in precommit
> phase, so maybe -Dmetastore.db=postgres could start metastore db
> automatically. Also we should consider running some qtests on various
> metastores. I would not pick randomly, but choose some "metastore-heavy" ones
> instead.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-21954) QTest: support for running qtests on various metastore DBs
[
https://issues.apache.org/jira/browse/HIVE-21954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16955644#comment-16955644
]
Hive QA commented on HIVE-21954:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
40s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
49s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m
2s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
13s{color} | {color:blue} standalone-metastore/metastore-server in master has
171 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
2s{color} | {color:blue} ql in master has 1547 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
39s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
46s{color} | {color:blue} itests/util in master has 53 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 10m
27s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
24s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 11m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 10m
58s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
23s{color} | {color:red} standalone-metastore: The patch generated 7 new + 50
unchanged - 6 fixed = 57 total (was 56) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
18s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 7 new + 50 unchanged - 6 fixed = 57 total (was 56) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m
0s{color} | {color:red} root: The patch generated 7 new + 104 unchanged - 6
fixed = 111 total (was 110) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
4s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 7m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 10m
25s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
15s{color} | {color:red} The patch generated 3 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 88m 31s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense xml javac javadoc findbugs checkstyle
compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19080/dev-support/hive-personality.sh
|
| git revision | master / c1e5fa3 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19080/yetus/diff-checkstyle-standalone-metastore.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19080/yetus/diff-checkstyle-standalone-metastore_metastore-server.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19080/yetus/diff-checkstyle-root.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19080/yetus/patch-asflicense-problems.txt
|
| modules | C:
[jira] [Commented] (HIVE-21954) QTest: support for running qtests on various metastore DBs
[
https://issues.apache.org/jira/browse/HIVE-21954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16954966#comment-16954966
]
Hive QA commented on HIVE-21954:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
52s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m
25s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
11s{color} | {color:blue} standalone-metastore/metastore-server in master has
170 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
18s{color} | {color:blue} ql in master has 1548 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
44s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
52s{color} | {color:blue} itests/util in master has 53 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 11m
7s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 12m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 11m
33s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
25s{color} | {color:red} standalone-metastore: The patch generated 7 new + 50
unchanged - 6 fixed = 57 total (was 56) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
18s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 7 new + 50 unchanged - 6 fixed = 57 total (was 56) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m
5s{color} | {color:red} root: The patch generated 7 new + 104 unchanged - 6
fixed = 111 total (was 110) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
3s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 7m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 13m
45s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
30s{color} | {color:red} The patch generated 3 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 96m 31s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile
xml |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19057/dev-support/hive-personality.sh
|
| git revision | master / 4d62434 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19057/yetus/diff-checkstyle-standalone-metastore.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19057/yetus/diff-checkstyle-standalone-metastore_metastore-server.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19057/yetus/diff-checkstyle-root.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19057/yetus/patch-asflicense-problems.txt
|
| modules | C:
[jira] [Commented] (HIVE-21954) QTest: support for running qtests on various metastore DBs
[
https://issues.apache.org/jira/browse/HIVE-21954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16954961#comment-16954961
]
Hive QA commented on HIVE-21954:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12983376/HIVE-21954.02.patch
{color:green}SUCCESS:{color} +1 due to 11 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 102 failed/errored test(s), 17100 tests
executed
*Failed tests:*
{noformat}
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=299)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=299)
TestMiniLlapLocalCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=161)
[external_jdbc_table_typeconversion.q,vector_udf_octet_length.q,schema_evol_orc_acidvec_table_update.q,materialized_view_rewrite_part_2.q,vector_decimal_5.q,vector_case_when_conversion.q,escape1.q,schema_evol_orc_acid_table_update_llap_io.q,cte_mat_5.q,acid_meta_columns_decode.q,vector_string_decimal.q,results_cache_lifetime.q,cross_prod_3.q,join46.q,dynpart_sort_optimization2.q,tez_bmj_schema_evolution.q,insert_into_default_keyword.q,bucketmapjoin4.q,vector_orc_null_check.q,semijoin7.q,uber_reduce.q,schema_evol_orc_nonvec_part_all_complex.q,is_distinct_from.q,schema_evol_text_vec_part_all_complex_llap_io.q,auto_sortmerge_join_3.q,vectorization_9.q,materialized_view_create_rewrite.q,merge2.q,join_nulls.q,bucketmapjoin2.q]
TestMinimrCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=299)
TestTezPerfCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=299)
TestTezPerfConstraintsCliDriver - did not produce a TEST-*.xml file (likely
timed out) (batchId=299)
org.apache.hadoop.hive.cli.TestSparkNegativeCliDriver.org.apache.hadoop.hive.cli.TestSparkNegativeCliDriver
(batchId=301)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query10]
(batchId=301)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query11]
(batchId=301)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query12]
(batchId=301)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query13]
(batchId=301)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query15]
(batchId=301)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query16]
(batchId=301)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query17]
(batchId=301)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query18]
(batchId=301)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query19]
(batchId=301)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query1]
(batchId=301)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query20]
(batchId=301)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query21]
(batchId=301)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query22]
(batchId=301)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query23]
(batchId=301)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query24]
(batchId=301)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query25]
(batchId=301)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query26]
(batchId=301)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query27]
(batchId=301)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query28]
(batchId=301)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query29]
(batchId=301)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query2]
(batchId=301)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query30]
(batchId=301)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query31]
(batchId=301)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query32]
(batchId=301)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query33]
(batchId=301)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query34]
(batchId=301)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query35]
(batchId=301)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query36]
(batchId=301)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query37]
(batchId=301)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query38]
(batchId=301)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query39]
(batchId=301)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query3]
(batchId=301)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query40]
(batchId=301)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query42]
(batchId=301)
[jira] [Commented] (HIVE-21954) QTest: support for running qtests on various metastore DBs
[
https://issues.apache.org/jira/browse/HIVE-21954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16953941#comment-16953941
]
Hive QA commented on HIVE-21954:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
52s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
49s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
14s{color} | {color:blue} standalone-metastore/metastore-server in master has
170 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
9s{color} | {color:blue} ql in master has 1548 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
48s{color} | {color:blue} itests/util in master has 53 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 10m
5s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
25s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 11m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 10m
28s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
24s{color} | {color:red} standalone-metastore: The patch generated 16 new + 50
unchanged - 6 fixed = 66 total (was 56) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
18s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 16 new + 50 unchanged - 6 fixed = 66 total (was 56) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
59s{color} | {color:red} root: The patch generated 18 new + 102 unchanged - 6
fixed = 120 total (was 108) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
15s{color} | {color:red} itests/util: The patch generated 2 new + 52 unchanged
- 0 fixed = 54 total (was 52) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 5 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 14m
29s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
33s{color} | {color:red} The patch generated 4 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 89m 3s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc xml compile findbugs
checkstyle |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19038/dev-support/hive-personality.sh
|
| git revision | master / c6626ed |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19038/yetus/diff-checkstyle-standalone-metastore.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19038/yetus/diff-checkstyle-standalone-metastore_metastore-server.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19038/yetus/diff-checkstyle-root.txt
|
| checkstyle |
[jira] [Commented] (HIVE-21954) QTest: support for running qtests on various metastore DBs
[
https://issues.apache.org/jira/browse/HIVE-21954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16953928#comment-16953928
]
Hive QA commented on HIVE-21954:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12983258/HIVE-21954.01.patch
{color:green}SUCCESS:{color} +1 due to 9 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 25 failed/errored test(s), 17058 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[mm_dp]
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_struct_type_vectorization]
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[parquet_complex_types_vectorization]
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[parquet_map_type_vectorization]
(batchId=159)
org.apache.hadoop.hive.cli.TestMinimrCliDriver.org.apache.hadoop.hive.cli.TestMinimrCliDriver
(batchId=299)
org.apache.hadoop.hive.cli.TestSparkNegativeCliDriver.org.apache.hadoop.hive.cli.TestSparkNegativeCliDriver
(batchId=301)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.org.apache.hadoop.hive.cli.TestSparkPerfCliDriver
(batchId=301)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.org.apache.hadoop.hive.cli.TestTezPerfCliDriver
(batchId=299)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver
(batchId=299)
org.apache.hadoop.hive.ql.TestLocationQueries.testAlterTablePartitionLocation_alter5
(batchId=276)
org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1 (batchId=274)
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[insert_wrong_number_columns]
(batchId=298)
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[invalid_dot]
(batchId=298)
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[invalid_function_param2]
(batchId=298)
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[invalid_index]
(batchId=298)
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[nonkey_groupby]
(batchId=298)
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[unknown_column1]
(batchId=298)
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[unknown_column2]
(batchId=298)
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[unknown_column3]
(batchId=298)
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[unknown_column4]
(batchId=298)
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[unknown_column5]
(batchId=298)
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[unknown_column6]
(batchId=298)
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[unknown_function2]
(batchId=298)
org.apache.hadoop.hive.ql.parse.TestParseNegativeDriver.testCliDriver[unknown_function3]
(batchId=298)
org.apache.hive.jdbc.TestXSRFFilter.testFilterEnabledWithInjection (batchId=282)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19038/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19038/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19038/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 25 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12983258 - PreCommit-HIVE-Build
> QTest: support for running qtests on various metastore DBs
> --
>
> Key: HIVE-21954
> URL: https://issues.apache.org/jira/browse/HIVE-21954
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore, Testing Infrastructure
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-21954.01.patch
>
>
> In HIVE-21940, a postgres metastore related issue has been fixed, and a local
> reproduction has been provided.
> {code}
> export QTEST_LEAVE_FILES=true
> docker kill metastore-test-postgres-install
> docker rm metastore-test-postgres-install
> cd standalone-metastore
> mvn verify -DskipITests=false -Dit.test=ITestPostgres#install -Dtest=nosuch
> -Dmetastore.itest.no.stop.container=true
> cd ..
> mvn test -Dtest.output.overwrite=true -Pitests,hadoop-2 -pl itests/qtest
> -Dtest=TestCliDriver -Dqfile=partition_params_postgres.q
>
