[jira] [Commented] (HIVE-15430) Change SchemaTool table validator to test based on the dbType
[ https://issues.apache.org/jira/browse/HIVE-15430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15886054#comment-15886054 ] Aihua Xu commented on HIVE-15430: - The new patch looks good to me. +1. > Change SchemaTool table validator to test based on the dbType > - > > Key: HIVE-15430 > URL: https://issues.apache.org/jira/browse/HIVE-15430 > Project: Hive > Issue Type: Sub-task > Components: Hive >Affects Versions: 2.2.0 >Reporter: Naveen Gangam >Assignee: Naveen Gangam >Priority: Minor > Attachments: HIVE-15430.1.patch, HIVE-15430.2.patch, > HIVE-15430.3.patch, HIVE-15430.4.patch > > > Currently the validator parses the "oracle" schema file to determine what > tables are expected in the database. (mostly because of ease of parsing the > schema file compared to other syntax). We have learnt from HIVE-15118, that > not all schema files have the same amount of tables. For example, derby has > an old table that is never used that other DBs do not contain). -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-15430) Change SchemaTool table validator to test based on the dbType
[ https://issues.apache.org/jira/browse/HIVE-15430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15885109#comment-15885109 ] Naveen Gangam commented on HIVE-15430: -- The test failures do not appear related to the proposed patch. > Change SchemaTool table validator to test based on the dbType > - > > Key: HIVE-15430 > URL: https://issues.apache.org/jira/browse/HIVE-15430 > Project: Hive > Issue Type: Sub-task > Components: Hive >Affects Versions: 2.2.0 >Reporter: Naveen Gangam >Assignee: Naveen Gangam >Priority: Minor > Attachments: HIVE-15430.1.patch, HIVE-15430.2.patch, > HIVE-15430.3.patch, HIVE-15430.4.patch > > > Currently the validator parses the "oracle" schema file to determine what > tables are expected in the database. (mostly because of ease of parsing the > schema file compared to other syntax). We have learnt from HIVE-15118, that > not all schema files have the same amount of tables. For example, derby has > an old table that is never used that other DBs do not contain). -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-15430) Change SchemaTool table validator to test based on the dbType
[
https://issues.apache.org/jira/browse/HIVE-15430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15884943#comment-15884943
]
Hive QA commented on HIVE-15430:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12854789/HIVE-15430.4.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 10266 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14]
(batchId=223)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query23]
(batchId=223)
org.apache.hive.hcatalog.pig.TestTextFileHCatStorer.testWriteDecimal
(batchId=173)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3798/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3798/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3798/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: NonZeroExitCodeException
Command 'cd /data/hiveptest/logs/PreCommit-HIVE-Build-3798/ && tar -zvcf
test-results.tar.gz test-results/' failed with exit status 2 and output
'test-results/
test-results/TEST-261_UTBatch_ql_10_tests-TEST-org.apache.hadoop.hive.ql.exec.vector.expressions.gen.TestColumnScalarOperationVectorExpressionEvaluation.xml
test-results/TEST-276_UTBatch_serde_20_tests-TEST-org.apache.hadoop.hive.serde2.avro.TestThatEvolvedSchemasActAsWeWant.xml
test-results/TEST-271_UTBatch_ql_10_tests-TEST-org.apache.hadoop.hive.ql.lockmgr.zookeeper.TestZookeeperLockManager.xml
test-results/TEST-195_UTBatch_itests__hive-unit_9_tests-TEST-org.apache.hadoop.hive.metastore.TestMetaStoreConnectionUrlHook.xml
test-results/TEST-180_UTBatch_hcatalog__core_9_tests-TEST-org.apache.hive.hcatalog.mapreduce.TestHCatInputFormat.xml
test-results/TEST-182_UTBatch_hcatalog__streaming_16_tests-TEST-org.apache.hive.hcatalog.streaming.mutate.worker.TestRecordInspectorImpl.xml
test-results/TEST-252_UTBatch_ql_10_tests-TEST-org.apache.hadoop.hive.ql.parse.TestSplitSample.xml
test-results/TEST-262_UTBatch_ql_10_tests-TEST-org.apache.hadoop.hive.ql.exec.vector.TestVectorSerDeRow.xml
test-results/TEST-126-TestSparkCliDriver-ptf_seqfile.q-union_remove_23.q-parallel_join0.q-and-12-more-TEST-org.apache.hadoop.hive.cli.TestSparkCliDriver.xml
test-results/TEST-173_UTBatch_hcatalog__hcatalog-pig-adapter_20_tests-TEST-org.apache.hive.hcatalog.pig.TestUtil.xml
test-results/TEST-168_UTBatch_beeline_13_tests-TEST-org.apache.hive.beeline.TestClientCommandHookFactory.xml
test-results/TEST-241_UTBatch_ql_10_tests-TEST-org.apache.hadoop.hive.ql.udf.generic.TestGenericUDFBridge.xml
test-results/TEST-173_UTBatch_hcatalog__hcatalog-pig-adapter_20_tests-TEST-org.apache.hive.hcatalog.pig.TestHCatLoaderStorer.xml
test-results/TEST-203_UTBatch_itests__hive-unit_9_tests-TEST-org.apache.hadoop.hive.metastore.TestHiveMetaStoreTxns.xml
test-results/TEST-169_UTBatch_storage-api_13_tests-TEST-org.apache.hadoop.hive.ql.exec.vector.TestBytesColumnVector.xml
test-results/TEST-182_UTBatch_hcatalog__streaming_16_tests-TEST-org.apache.hive.hcatalog.streaming.mutate.worker.TestMutatorCoordinator.xml
test-results/TEST-213_UTBatch_itests__hive-unit_9_tests-TEST-org.apache.hive.service.TestHS2ImpersonationWithRemoteMS.xml
test-results/TEST-87-TestNegativeMinimrCliDriver-udf_local_resource.q-mapreduce_stack_trace.q-minimr_broken_pipe.q-and-5-more-TEST-org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.xml
test-results/TEST-250_UTBatch_ql_10_tests-TEST-org.apache.hadoop.hive.ql.io.orc.TestOrcSplitElimination.xml
test-results/TEST-210_UTBatch_itests__hive-unit_9_tests-TEST-org.apache.hadoop.hive.ql.security.TestStorageBasedMetastoreAuthorizationReads.xml
test-results/TEST-14-TestCliDriver-authorization_create_temp_table.q-skewjoinopt16.q-drop_partitions_filter3.q-and-27-more-TEST-org.apache.hadoop.hive.cli.TestCliDriver.xml
test-results/TEST-185_UTBatch_service_8_tests-TEST-org.apache.hive.service.auth.TestLdapAtnProviderWithMiniDS.xml
test-results/TEST-275_UTBatch_serde_20_tests-TEST-org.apache.hadoop.hive.serde2.objectinspector.TestStandardObjectInspectors.xml
test-results/TEST-73-TestCliDriver-tez_union_group_by.q-llap_acid.q-truncate_table.q-and-27-more-TEST-org.apache.hadoop.hive.cli.TestCliDriver.xml
test-results/TEST-199_UTBatch_itests__hive-unit_9_tests-TEST-org.apache.hadoop.hive.metastore.hbase.TestHBaseSchemaTool2.xml
test-results/TEST-277_UTBatch_serde_6_tests-TEST-org.apache.hadoop.hive.serde2.columnar.TestLazyBinaryColumnarSerDe.xml
[jira] [Commented] (HIVE-15430) Change SchemaTool table validator to test based on the dbType
[
https://issues.apache.org/jira/browse/HIVE-15430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15866194#comment-15866194
]
Hive QA commented on HIVE-15430:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12852598/HIVE-15430.3.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 10224 tests
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out)
(batchId=235)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys]
(batchId=159)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14]
(batchId=223)
org.apache.hadoop.hive.cli.TestSparkCliDriver.org.apache.hadoop.hive.cli.TestSparkCliDriver
(batchId=101)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3540/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3540/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3540/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12852598 - PreCommit-HIVE-Build
> Change SchemaTool table validator to test based on the dbType
> -
>
> Key: HIVE-15430
> URL: https://issues.apache.org/jira/browse/HIVE-15430
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Affects Versions: 2.2.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
>Priority: Minor
> Attachments: HIVE-15430.1.patch, HIVE-15430.2.patch,
> HIVE-15430.3.patch
>
>
> Currently the validator parses the "oracle" schema file to determine what
> tables are expected in the database. (mostly because of ease of parsing the
> schema file compared to other syntax). We have learnt from HIVE-15118, that
> not all schema files have the same amount of tables. For example, derby has
> an old table that is never used that other DBs do not contain).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-15430) Change SchemaTool table validator to test based on the dbType
[ https://issues.apache.org/jira/browse/HIVE-15430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15861539#comment-15861539 ] Naveen Gangam commented on HIVE-15430: -- review posted at https://reviews.apache.org/r/56549/ [~aihuaxu] [~ctang.ma] [~ychena] Could you please review? Thanks > Change SchemaTool table validator to test based on the dbType > - > > Key: HIVE-15430 > URL: https://issues.apache.org/jira/browse/HIVE-15430 > Project: Hive > Issue Type: Sub-task > Components: Hive >Affects Versions: 2.2.0 >Reporter: Naveen Gangam >Assignee: Naveen Gangam >Priority: Minor > Attachments: HIVE-15430.1.patch, HIVE-15430.2.patch > > > Currently the validator parses the "oracle" schema file to determine what > tables are expected in the database. (mostly because of ease of parsing the > schema file compared to other syntax). We have learnt from HIVE-15118, that > not all schema files have the same amount of tables. For example, derby has > an old table that is never used that other DBs do not contain). -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-15430) Change SchemaTool table validator to test based on the dbType
[
https://issues.apache.org/jira/browse/HIVE-15430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15861478#comment-15861478
]
Hive QA commented on HIVE-15430:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12852065/HIVE-15430.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 10241 tests
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out)
(batchId=235)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys]
(batchId=159)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14]
(batchId=223)
org.apache.hadoop.hive.cli.TestSparkNegativeCliDriver.org.apache.hadoop.hive.cli.TestSparkNegativeCliDriver
(batchId=230)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3492/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3492/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3492/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12852065 - PreCommit-HIVE-Build
> Change SchemaTool table validator to test based on the dbType
> -
>
> Key: HIVE-15430
> URL: https://issues.apache.org/jira/browse/HIVE-15430
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Affects Versions: 2.2.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
>Priority: Minor
> Attachments: HIVE-15430.1.patch, HIVE-15430.2.patch
>
>
> Currently the validator parses the "oracle" schema file to determine what
> tables are expected in the database. (mostly because of ease of parsing the
> schema file compared to other syntax). We have learnt from HIVE-15118, that
> not all schema files have the same amount of tables. For example, derby has
> an old table that is never used that other DBs do not contain).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-15430) Change SchemaTool table validator to test based on the dbType
[
https://issues.apache.org/jira/browse/HIVE-15430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15860898#comment-15860898
]
Hive QA commented on HIVE-15430:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12851985/HIVE-15430.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 10244 tests
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out)
(batchId=235)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys]
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_if_expr]
(batchId=140)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14]
(batchId=223)
org.apache.hive.beeline.TestSchemaTool.testValidateSchemaTables (batchId=211)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3483/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3483/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3483/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12851985 - PreCommit-HIVE-Build
> Change SchemaTool table validator to test based on the dbType
> -
>
> Key: HIVE-15430
> URL: https://issues.apache.org/jira/browse/HIVE-15430
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Affects Versions: 2.2.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
>Priority: Minor
> Attachments: HIVE-15430.1.patch
>
>
> Currently the validator parses the "oracle" schema file to determine what
> tables are expected in the database. (mostly because of ease of parsing the
> schema file compared to other syntax). We have learnt from HIVE-15118, that
> not all schema files have the same amount of tables. For example, derby has
> an old table that is never used that other DBs do not contain).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-15430) Change SchemaTool table validator to test based on the dbType
[ https://issues.apache.org/jira/browse/HIVE-15430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15854311#comment-15854311 ] Aihua Xu commented on HIVE-15430: - [~ngangam] I'm wondering if you can add "validate the location in SKEWED_COL_VALUE_LOC_MAP". Other locations are validated but seems this one is missing. > Change SchemaTool table validator to test based on the dbType > - > > Key: HIVE-15430 > URL: https://issues.apache.org/jira/browse/HIVE-15430 > Project: Hive > Issue Type: Sub-task > Components: Hive >Affects Versions: 2.2.0 >Reporter: Naveen Gangam >Assignee: Naveen Gangam >Priority: Minor > > Currently the validator parses the "oracle" schema file to determine what > tables are expected in the database. (mostly because of ease of parsing the > schema file compared to other syntax). We have learnt from HIVE-15118, that > not all schema files have the same amount of tables. For example, derby has > an old table that is never used that other DBs do not contain). -- This message was sent by Atlassian JIRA (v6.3.15#6346)
