[jira] [Commented] (HIVE-15910) Improvements in Hive Unit Test by using In-memory Derby DB

[Sankar Hariappan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-15910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15875466#comment-15875466
 ] 

Sankar Hariappan commented on HIVE-15910:
-

Thanks a lot [~thejas] and [~wzheng]!

> Improvements in Hive Unit Test by using In-memory Derby DB
> --
>
> Key: HIVE-15910
> URL: https://issues.apache.org/jira/browse/HIVE-15910
> Project: Hive
>  Issue Type: Test
>  Components: Tests
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
> Fix For: 2.2.0
>
> Attachments: HIVE-15910.01.patch, HIVE-15910.05.patch, 
> HIVE-15910.06.patch, HIVE-15910.2.patch, HIVE-15910.3.patch, 
> HIVE-15910.4.patch
>
>
> Hive UT currently uses Derby DB with storage on disk which have some 
> practical problems.
> 1. The run-time of Hive unit tests are high as need to operate on the disk 
> quite often.
> 2. It can cause conflict if multiple test cases operates on the same table 
> name (such as table being created already exist).
> To solve these problems, we shall use an in-memory storage option of Derby DB 
> which can be even persisted if the test case demands that.
> https://db.apache.org/derby/docs/10.8/devguide/cdevdvlpinmemdb.html



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-15910) Improvements in Hive Unit Test by using In-memory Derby DB

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-15910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15875467#comment-15875467
 ] 

ASF GitHub Bot commented on HIVE-15910:
---

Github user sankarh closed the pull request at:

https://github.com/apache/hive/pull/147


> Improvements in Hive Unit Test by using In-memory Derby DB
> --
>
> Key: HIVE-15910
> URL: https://issues.apache.org/jira/browse/HIVE-15910
> Project: Hive
>  Issue Type: Test
>  Components: Tests
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
> Fix For: 2.2.0
>
> Attachments: HIVE-15910.01.patch, HIVE-15910.05.patch, 
> HIVE-15910.06.patch, HIVE-15910.2.patch, HIVE-15910.3.patch, 
> HIVE-15910.4.patch
>
>
> Hive UT currently uses Derby DB with storage on disk which have some 
> practical problems.
> 1. The run-time of Hive unit tests are high as need to operate on the disk 
> quite often.
> 2. It can cause conflict if multiple test cases operates on the same table 
> name (such as table being created already exist).
> To solve these problems, we shall use an in-memory storage option of Derby DB 
> which can be even persisted if the test case demands that.
> https://db.apache.org/derby/docs/10.8/devguide/cdevdvlpinmemdb.html



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-15910) Improvements in Hive Unit Test by using In-memory Derby DB

[Thejas M Nair (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-15910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15875455#comment-15875455
 ] 

Thejas M Nair commented on HIVE-15910:
--

+1


> Improvements in Hive Unit Test by using In-memory Derby DB
> --
>
> Key: HIVE-15910
> URL: https://issues.apache.org/jira/browse/HIVE-15910
> Project: Hive
>  Issue Type: Test
>  Components: Tests
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
> Attachments: HIVE-15910.01.patch, HIVE-15910.05.patch, 
> HIVE-15910.06.patch, HIVE-15910.2.patch, HIVE-15910.3.patch, 
> HIVE-15910.4.patch
>
>
> Hive UT currently uses Derby DB with storage on disk which have some 
> practical problems.
> 1. The run-time of Hive unit tests are high as need to operate on the disk 
> quite often.
> 2. It can cause conflict if multiple test cases operates on the same table 
> name (such as table being created already exist).
> To solve these problems, we shall use an in-memory storage option of Derby DB 
> which can be even persisted if the test case demands that.
> https://db.apache.org/derby/docs/10.8/devguide/cdevdvlpinmemdb.html



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-15910) Improvements in Hive Unit Test by using In-memory Derby DB

[Sankar Hariappan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-15910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15873736#comment-15873736
 ] 

Sankar Hariappan commented on HIVE-15910:
-

[~wzheng]
Currently, I didn't check the performance impacts due to creation and deletion 
of temp dir for each test case. 
However, this can be tracked using another JIRA ticket to make the temp dir 
creation only once for whole of TestWorker.

> Improvements in Hive Unit Test by using In-memory Derby DB
> --
>
> Key: HIVE-15910
> URL: https://issues.apache.org/jira/browse/HIVE-15910
> Project: Hive
>  Issue Type: Test
>  Components: Tests
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
> Attachments: HIVE-15910.01.patch, HIVE-15910.05.patch, 
> HIVE-15910.06.patch, HIVE-15910.2.patch, HIVE-15910.3.patch, 
> HIVE-15910.4.patch
>
>
> Hive UT currently uses Derby DB with storage on disk which have some 
> practical problems.
> 1. The run-time of Hive unit tests are high as need to operate on the disk 
> quite often.
> 2. It can cause conflict if multiple test cases operates on the same table 
> name (such as table being created already exist).
> To solve these problems, we shall use an in-memory storage option of Derby DB 
> which can be even persisted if the test case demands that.
> https://db.apache.org/derby/docs/10.8/devguide/cdevdvlpinmemdb.html



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-15910) Improvements in Hive Unit Test by using In-memory Derby DB

[Wei Zheng (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-15910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872741#comment-15872741
 ] 

Wei Zheng commented on HIVE-15910:
--

The code change looks good. Btw, did we do any performance analysis for this 
change?

> Improvements in Hive Unit Test by using In-memory Derby DB
> --
>
> Key: HIVE-15910
> URL: https://issues.apache.org/jira/browse/HIVE-15910
> Project: Hive
>  Issue Type: Test
>  Components: Tests
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
> Attachments: HIVE-15910.01.patch, HIVE-15910.05.patch, 
> HIVE-15910.06.patch, HIVE-15910.2.patch, HIVE-15910.3.patch, 
> HIVE-15910.4.patch
>
>
> Hive UT currently uses Derby DB with storage on disk which have some 
> practical problems.
> 1. The run-time of Hive unit tests are high as need to operate on the disk 
> quite often.
> 2. It can cause conflict if multiple test cases operates on the same table 
> name (such as table being created already exist).
> To solve these problems, we shall use an in-memory storage option of Derby DB 
> which can be even persisted if the test case demands that.
> https://db.apache.org/derby/docs/10.8/devguide/cdevdvlpinmemdb.html



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-15910) Improvements in Hive Unit Test by using In-memory Derby DB

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-15910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15871999#comment-15871999
 ] 

Hive QA commented on HIVE-15910:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12853254/HIVE-15910.06.patch

{color:green}SUCCESS:{color} +1 due to 4 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 30 failed/errored test(s), 10244 tests 
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out) 
(batchId=235)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_auto_join1] 
(batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join31] (batchId=81)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys]
 (batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[multiMapJoin2]
 (batchId=152)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[create_external_acid]
 (batchId=86)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[create_not_acid] 
(batchId=86)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[create_view_failure1]
 (batchId=85)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[create_view_failure2]
 (batchId=85)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[external1] 
(batchId=85)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[join31] 
(batchId=133)
org.apache.hadoop.hive.metastore.TestEmbeddedHiveMetaStore.testAlterTable 
(batchId=194)
org.apache.hadoop.hive.metastore.TestEmbeddedHiveMetaStore.testTransactionalValidation
 (batchId=194)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStore.testAlterTable 
(batchId=197)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStore.testTransactionalValidation
 (batchId=197)
org.apache.hadoop.hive.metastore.TestSetUGIOnBothClientServer.testAlterTable 
(batchId=193)
org.apache.hadoop.hive.metastore.TestSetUGIOnBothClientServer.testTransactionalValidation
 (batchId=193)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testAlterTable 
(batchId=191)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testTransactionalValidation
 (batchId=191)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testAlterTable 
(batchId=202)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testTransactionalValidation
 (batchId=202)
org.apache.hadoop.hive.ql.security.TestMetastoreAuthorizationProvider.testSimplePrivileges
 (batchId=210)
org.apache.hadoop.hive.ql.security.TestStorageBasedMetastoreAuthorizationProvider.testSimplePrivileges
 (batchId=208)
org.apache.hadoop.hive.ql.security.TestStorageBasedMetastoreAuthorizationProviderWithACL.testSimplePrivileges
 (batchId=220)
org.apache.hive.hcatalog.api.TestHCatClient.testBasicDDLCommands (batchId=170)
org.apache.hive.hcatalog.api.TestHCatClient.testTransportFailure (batchId=170)
org.apache.hive.hcatalog.listener.TestDbNotificationListener.createTable 
(batchId=221)
org.apache.hive.service.cli.TestEmbeddedThriftBinaryCLIService.testExecuteStatementAsync
 (batchId=213)
org.apache.hive.service.cli.thrift.TestThriftCLIServiceWithBinary.testExecuteStatementAsync
 (batchId=213)
org.apache.hive.service.cli.thrift.TestThriftCLIServiceWithHttp.testExecuteStatementAsync
 (batchId=213)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3627/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3627/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3627/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 30 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12853254 - PreCommit-HIVE-Build

> Improvements in Hive Unit Test by using In-memory Derby DB
> --
>
> Key: HIVE-15910
> URL: https://issues.apache.org/jira/browse/HIVE-15910
> Project: Hive
>  Issue Type: Test
>  Components: Tests
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
> Attachments: HIVE-15910.01.patch, HIVE-15910.05.patch, 
> HIVE-15910.06.patch, HIVE-15910.2.patch, HIVE-15910.3.patch, 
> HIVE-15910.4.patch
>
>
> Hive UT currently uses Derby DB with storage on disk which have some 
> practical problems.
> 1. The run-time of Hive unit tests are high as need to operate on the disk 
> quite often.
> 2. It can cause conflict if multiple test cases operates on the same table 
> name (such as table being created already exist).
> To solve these problems, we shall use an 

[jira] [Commented] (HIVE-15910) Improvements in Hive Unit Test by using In-memory Derby DB

[Alan Gates (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-15910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15870462#comment-15870462
 ] 

Alan Gates commented on HIVE-15910:
---

Getting a new TxnHandler each time is good, as that will be sure to clean up 
any state.  I agree the rest of the lines above could probably be moved to a 
@BeforeClass method.




> Improvements in Hive Unit Test by using In-memory Derby DB
> --
>
> Key: HIVE-15910
> URL: https://issues.apache.org/jira/browse/HIVE-15910
> Project: Hive
>  Issue Type: Test
>  Components: Tests
>Reporter: Sankar Hariappan
>Assignee: Wei Zheng
> Attachments: HIVE-15910.01.patch, HIVE-15910.05.patch, 
> HIVE-15910.2.patch, HIVE-15910.3.patch, HIVE-15910.4.patch
>
>
> Hive UT currently uses Derby DB with storage on disk which have some 
> practical problems.
> 1. The run-time of Hive unit tests are high as need to operate on the disk 
> quite often.
> 2. It can cause conflict if multiple test cases operates on the same table 
> name (such as table being created already exist).
> To solve these problems, we shall use an in-memory storage option of Derby DB 
> which can be even persisted if the test case demands that.
> https://db.apache.org/derby/docs/10.8/devguide/cdevdvlpinmemdb.html



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-15910) Improvements in Hive Unit Test by using In-memory Derby DB

[Thejas M Nair (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-15910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15870399#comment-15870399
 ] 

Thejas M Nair commented on HIVE-15910:
--

In addition, I think we should look at the methods called in CompactorTest 
constructor.
CompactorTest constructor would be called for every test function. Ie it is 
equivalent of '@Before' and not '@BeforeClass' .
It seems like some what is done there makes more sense in a '@BeforeClass' , so 
that its just done once. But I haven't taken a very close look at the tests, so 
I am not sure. Maybe this should be addressed in a follow up jira, and not this 
one. 

cc [~ekoifman] [~wzheng] [~alangates]
{code}
ms = new HiveMetaStoreClient(conf);
txnHandler = TxnUtils.getTxnStore(conf);
tmpdir = new File(System.getProperty("java.io.tmpdir") +
System.getProperty("file.separator") + "compactor_test_tables");
tmpdir.mkdir();
tmpdir.deleteOnExit();
{code}

> Improvements in Hive Unit Test by using In-memory Derby DB
> --
>
> Key: HIVE-15910
> URL: https://issues.apache.org/jira/browse/HIVE-15910
> Project: Hive
>  Issue Type: Test
>  Components: Tests
>Reporter: Sankar Hariappan
>Assignee: Wei Zheng
> Attachments: HIVE-15910.01.patch, HIVE-15910.05.patch, 
> HIVE-15910.2.patch, HIVE-15910.3.patch, HIVE-15910.4.patch
>
>
> Hive UT currently uses Derby DB with storage on disk which have some 
> practical problems.
> 1. The run-time of Hive unit tests are high as need to operate on the disk 
> quite often.
> 2. It can cause conflict if multiple test cases operates on the same table 
> name (such as table being created already exist).
> To solve these problems, we shall use an in-memory storage option of Derby DB 
> which can be even persisted if the test case demands that.
> https://db.apache.org/derby/docs/10.8/devguide/cdevdvlpinmemdb.html



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-15910) Improvements in Hive Unit Test by using In-memory Derby DB

[Thejas M Nair (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-15910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15870372#comment-15870372
 ] 

Thejas M Nair commented on HIVE-15910:
--

Added a comment in github pull request. Its simpler to just delete the temp dir 
where all tables are being created.
Also realized, that  tmpdir.deleteOnExit() doesn't work unless the dir is 
empty. That call would be useless. 


> Improvements in Hive Unit Test by using In-memory Derby DB
> --
>
> Key: HIVE-15910
> URL: https://issues.apache.org/jira/browse/HIVE-15910
> Project: Hive
>  Issue Type: Test
>  Components: Tests
>Reporter: Sankar Hariappan
>Assignee: Wei Zheng
> Attachments: HIVE-15910.01.patch, HIVE-15910.05.patch, 
> HIVE-15910.2.patch, HIVE-15910.3.patch, HIVE-15910.4.patch
>
>
> Hive UT currently uses Derby DB with storage on disk which have some 
> practical problems.
> 1. The run-time of Hive unit tests are high as need to operate on the disk 
> quite often.
> 2. It can cause conflict if multiple test cases operates on the same table 
> name (such as table being created already exist).
> To solve these problems, we shall use an in-memory storage option of Derby DB 
> which can be even persisted if the test case demands that.
> https://db.apache.org/derby/docs/10.8/devguide/cdevdvlpinmemdb.html



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-15910) Improvements in Hive Unit Test by using In-memory Derby DB

[Wei Zheng (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-15910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15870293#comment-15870293
 ] 

Wei Zheng commented on HIVE-15910:
--

Thanks [~sankarh] for finding this out! It makes sense.

Seems you missed adding that deleteTableDir call for minorPartitionWithBase :) 
Maybe it's better to add tearDown to TestWorker to perform delete. [~ekoifman]

> Improvements in Hive Unit Test by using In-memory Derby DB
> --
>
> Key: HIVE-15910
> URL: https://issues.apache.org/jira/browse/HIVE-15910
> Project: Hive
>  Issue Type: Test
>  Components: Tests
>Reporter: Sankar Hariappan
>Assignee: Wei Zheng
> Attachments: HIVE-15910.01.patch, HIVE-15910.05.patch, 
> HIVE-15910.2.patch, HIVE-15910.3.patch, HIVE-15910.4.patch
>
>
> Hive UT currently uses Derby DB with storage on disk which have some 
> practical problems.
> 1. The run-time of Hive unit tests are high as need to operate on the disk 
> quite often.
> 2. It can cause conflict if multiple test cases operates on the same table 
> name (such as table being created already exist).
> To solve these problems, we shall use an in-memory storage option of Derby DB 
> which can be even persisted if the test case demands that.
> https://db.apache.org/derby/docs/10.8/devguide/cdevdvlpinmemdb.html



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-15910) Improvements in Hive Unit Test by using In-memory Derby DB

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-15910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15870004#comment-15870004
 ] 

Hive QA commented on HIVE-15910:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12853001/HIVE-15910.05.patch

{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 10238 tests 
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out) 
(batchId=235)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_auto_join1] 
(batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join31] (batchId=81)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys]
 (batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[multiMapJoin2]
 (batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_if_expr]
 (batchId=140)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14] 
(batchId=223)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query23] 
(batchId=223)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[join31] 
(batchId=133)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker.minorPartitionWithBase 
(batchId=246)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3597/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3597/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3597/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 10 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12853001 - PreCommit-HIVE-Build

> Improvements in Hive Unit Test by using In-memory Derby DB
> --
>
> Key: HIVE-15910
> URL: https://issues.apache.org/jira/browse/HIVE-15910
> Project: Hive
>  Issue Type: Test
>  Components: Tests
>Reporter: Sankar Hariappan
>Assignee: Wei Zheng
> Attachments: HIVE-15910.01.patch, HIVE-15910.05.patch, 
> HIVE-15910.2.patch, HIVE-15910.3.patch, HIVE-15910.4.patch
>
>
> Hive UT currently uses Derby DB with storage on disk which have some 
> practical problems.
> 1. The run-time of Hive unit tests are high as need to operate on the disk 
> quite often.
> 2. It can cause conflict if multiple test cases operates on the same table 
> name (such as table being created already exist).
> To solve these problems, we shall use an in-memory storage option of Derby DB 
> which can be even persisted if the test case demands that.
> https://db.apache.org/derby/docs/10.8/devguide/cdevdvlpinmemdb.html



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-15910) Improvements in Hive Unit Test by using In-memory Derby DB

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-15910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15869058#comment-15869058
 ] 

Hive QA commented on HIVE-15910:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12852925/HIVE-15910.4.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 14 failed/errored test(s), 10238 tests 
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out) 
(batchId=235)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_auto_join1] 
(batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join31] (batchId=81)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys]
 (batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[multiMapJoin2]
 (batchId=152)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14] 
(batchId=223)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[join31] 
(batchId=133)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker.majorPartitionWithBase 
(batchId=246)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker.majorPartitionWithBaseMissingBuckets
 (batchId=246)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker.majorTableNoBase 
(batchId=246)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker.majorTableWithBase 
(batchId=246)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker.minorPartitionWithBase 
(batchId=246)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker.minorTableLegacy 
(batchId=246)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker.minorTableNoBase 
(batchId=246)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3582/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3582/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3582/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 14 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12852925 - PreCommit-HIVE-Build

> Improvements in Hive Unit Test by using In-memory Derby DB
> --
>
> Key: HIVE-15910
> URL: https://issues.apache.org/jira/browse/HIVE-15910
> Project: Hive
>  Issue Type: Test
>  Components: Tests
>Reporter: Sankar Hariappan
>Assignee: Wei Zheng
> Attachments: HIVE-15910.01.patch, HIVE-15910.2.patch, 
> HIVE-15910.3.patch, HIVE-15910.4.patch
>
>
> Hive UT currently uses Derby DB with storage on disk which have some 
> practical problems.
> 1. The run-time of Hive unit tests are high as need to operate on the disk 
> quite often.
> 2. It can cause conflict if multiple test cases operates on the same table 
> name (such as table being created already exist).
> To solve these problems, we shall use an in-memory storage option of Derby DB 
> which can be even persisted if the test case demands that.
> https://db.apache.org/derby/docs/10.8/devguide/cdevdvlpinmemdb.html



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-15910) Improvements in Hive Unit Test by using In-memory Derby DB

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-15910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15869009#comment-15869009
 ] 

Hive QA commented on HIVE-15910:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12852925/HIVE-15910.4.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 14 failed/errored test(s), 10238 tests 
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out) 
(batchId=235)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_auto_join1] 
(batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join31] (batchId=81)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys]
 (batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[multiMapJoin2]
 (batchId=152)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14] 
(batchId=223)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[join31] 
(batchId=133)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker.majorPartitionWithBase 
(batchId=246)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker.majorPartitionWithBaseMissingBuckets
 (batchId=246)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker.majorTableNoBase 
(batchId=246)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker.majorTableWithBase 
(batchId=246)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker.minorPartitionWithBase 
(batchId=246)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker.minorTableLegacy 
(batchId=246)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker.minorTableNoBase 
(batchId=246)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3581/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3581/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3581/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 14 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12852925 - PreCommit-HIVE-Build

> Improvements in Hive Unit Test by using In-memory Derby DB
> --
>
> Key: HIVE-15910
> URL: https://issues.apache.org/jira/browse/HIVE-15910
> Project: Hive
>  Issue Type: Test
>  Components: Tests
>Reporter: Sankar Hariappan
>Assignee: Wei Zheng
> Attachments: HIVE-15910.01.patch, HIVE-15910.2.patch, 
> HIVE-15910.3.patch, HIVE-15910.4.patch
>
>
> Hive UT currently uses Derby DB with storage on disk which have some 
> practical problems.
> 1. The run-time of Hive unit tests are high as need to operate on the disk 
> quite often.
> 2. It can cause conflict if multiple test cases operates on the same table 
> name (such as table being created already exist).
> To solve these problems, we shall use an in-memory storage option of Derby DB 
> which can be even persisted if the test case demands that.
> https://db.apache.org/derby/docs/10.8/devguide/cdevdvlpinmemdb.html



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-15910) Improvements in Hive Unit Test by using In-memory Derby DB

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-15910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15868548#comment-15868548
 ] 

Hive QA commented on HIVE-15910:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12852873/HIVE-15910.3.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 15 failed/errored test(s), 10238 tests 
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out) 
(batchId=235)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_auto_join1] 
(batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join31] (batchId=81)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys]
 (batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[multiMapJoin2]
 (batchId=152)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14] 
(batchId=223)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query23] 
(batchId=223)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[join31] 
(batchId=133)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker.majorPartitionWithBase 
(batchId=246)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker.majorPartitionWithBaseMissingBuckets
 (batchId=246)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker.majorTableNoBase 
(batchId=246)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker.majorTableWithBase 
(batchId=246)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker.minorPartitionWithBase 
(batchId=246)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker.minorTableLegacy 
(batchId=246)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker.minorTableNoBase 
(batchId=246)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3573/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3573/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3573/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 15 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12852873 - PreCommit-HIVE-Build

> Improvements in Hive Unit Test by using In-memory Derby DB
> --
>
> Key: HIVE-15910
> URL: https://issues.apache.org/jira/browse/HIVE-15910
> Project: Hive
>  Issue Type: Test
>  Components: Tests
>Reporter: Sankar Hariappan
>Assignee: Wei Zheng
> Attachments: HIVE-15910.01.patch, HIVE-15910.2.patch, 
> HIVE-15910.3.patch
>
>
> Hive UT currently uses Derby DB with storage on disk which have some 
> practical problems.
> 1. The run-time of Hive unit tests are high as need to operate on the disk 
> quite often.
> 2. It can cause conflict if multiple test cases operates on the same table 
> name (such as table being created already exist).
> To solve these problems, we shall use an in-memory storage option of Derby DB 
> which can be even persisted if the test case demands that.
> https://db.apache.org/derby/docs/10.8/devguide/cdevdvlpinmemdb.html



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-15910) Improvements in Hive Unit Test by using In-memory Derby DB

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-15910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15866715#comment-15866715
 ] 

Hive QA commented on HIVE-15910:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12852638/HIVE-15910.2.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 11 failed/errored test(s), 10238 tests 
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out) 
(batchId=235)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys]
 (batchId=159)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14] 
(batchId=223)
org.apache.hadoop.hive.ql.security.TestStorageBasedMetastoreAuthorizationDrops.testDropTable
 (batchId=210)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker.majorPartitionWithBase 
(batchId=246)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker.majorPartitionWithBaseMissingBuckets
 (batchId=246)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker.majorTableNoBase 
(batchId=246)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker.majorTableWithBase 
(batchId=246)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker.minorPartitionWithBase 
(batchId=246)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker.minorTableLegacy 
(batchId=246)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker.minorTableNoBase 
(batchId=246)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3545/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3545/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3545/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 11 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12852638 - PreCommit-HIVE-Build

> Improvements in Hive Unit Test by using In-memory Derby DB
> --
>
> Key: HIVE-15910
> URL: https://issues.apache.org/jira/browse/HIVE-15910
> Project: Hive
>  Issue Type: Test
>  Components: Tests
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
> Attachments: HIVE-15910.01.patch, HIVE-15910.2.patch
>
>
> Hive UT currently uses Derby DB with storage on disk which have some 
> practical problems.
> 1. The run-time of Hive unit tests are high as need to operate on the disk 
> quite often.
> 2. It can cause conflict if multiple test cases operates on the same table 
> name (such as table being created already exist).
> To solve these problems, we shall use an in-memory storage option of Derby DB 
> which can be even persisted if the test case demands that.
> https://db.apache.org/derby/docs/10.8/devguide/cdevdvlpinmemdb.html



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-15910) Improvements in Hive Unit Test by using In-memory Derby DB

[Wei Zheng (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-15910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15866405#comment-15866405
 ] 

Wei Zheng commented on HIVE-15910:
--

Btw TestWorker tests all passed locally. Keep looking..

> Improvements in Hive Unit Test by using In-memory Derby DB
> --
>
> Key: HIVE-15910
> URL: https://issues.apache.org/jira/browse/HIVE-15910
> Project: Hive
>  Issue Type: Test
>  Components: Tests
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
> Attachments: HIVE-15910.01.patch
>
>
> Hive UT currently uses Derby DB with storage on disk which have some 
> practical problems.
> 1. The run-time of Hive unit tests are high as need to operate on the disk 
> quite often.
> 2. It can cause conflict if multiple test cases operates on the same table 
> name (such as table being created already exist).
> To solve these problems, we shall use an in-memory storage option of Derby DB 
> which can be even persisted if the test case demands that.
> https://db.apache.org/derby/docs/10.8/devguide/cdevdvlpinmemdb.html



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-15910) Improvements in Hive Unit Test by using In-memory Derby DB

[Wei Zheng (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-15910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15866361#comment-15866361
 ] 

Wei Zheng commented on HIVE-15910:
--

We do start new thread which is common for these tests. Let me debug the error 
and update you.

> Improvements in Hive Unit Test by using In-memory Derby DB
> --
>
> Key: HIVE-15910
> URL: https://issues.apache.org/jira/browse/HIVE-15910
> Project: Hive
>  Issue Type: Test
>  Components: Tests
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
> Attachments: HIVE-15910.01.patch
>
>
> Hive UT currently uses Derby DB with storage on disk which have some 
> practical problems.
> 1. The run-time of Hive unit tests are high as need to operate on the disk 
> quite often.
> 2. It can cause conflict if multiple test cases operates on the same table 
> name (such as table being created already exist).
> To solve these problems, we shall use an in-memory storage option of Derby DB 
> which can be even persisted if the test case demands that.
> https://db.apache.org/derby/docs/10.8/devguide/cdevdvlpinmemdb.html



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-15910) Improvements in Hive Unit Test by using In-memory Derby DB

[Thejas M Nair (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-15910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15866271#comment-15866271
 ] 

Thejas M Nair commented on HIVE-15910:
--

[~wzheng] [~sankarh]

TestWorker failures seem to be new and likely related to this change. 
Wei, any thoughts on why it cause those to fail ? Are we spawning another 
process from those ?
We could change the config for those tests if necessary.

> Improvements in Hive Unit Test by using In-memory Derby DB
> --
>
> Key: HIVE-15910
> URL: https://issues.apache.org/jira/browse/HIVE-15910
> Project: Hive
>  Issue Type: Test
>  Components: Tests
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
> Attachments: HIVE-15910.01.patch
>
>
> Hive UT currently uses Derby DB with storage on disk which have some 
> practical problems.
> 1. The run-time of Hive unit tests are high as need to operate on the disk 
> quite often.
> 2. It can cause conflict if multiple test cases operates on the same table 
> name (such as table being created already exist).
> To solve these problems, we shall use an in-memory storage option of Derby DB 
> which can be even persisted if the test case demands that.
> https://db.apache.org/derby/docs/10.8/devguide/cdevdvlpinmemdb.html



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-15910) Improvements in Hive Unit Test by using In-memory Derby DB

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-15910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15865748#comment-15865748
 ] 

Hive QA commented on HIVE-15910:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12852552/HIVE-15910.01.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 11 failed/errored test(s), 10238 tests 
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out) 
(batchId=235)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys]
 (batchId=159)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14] 
(batchId=223)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query23] 
(batchId=223)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker.majorPartitionWithBase 
(batchId=246)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker.majorPartitionWithBaseMissingBuckets
 (batchId=246)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker.majorTableNoBase 
(batchId=246)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker.majorTableWithBase 
(batchId=246)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker.minorPartitionWithBase 
(batchId=246)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker.minorTableLegacy 
(batchId=246)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker.minorTableNoBase 
(batchId=246)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3538/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3538/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3538/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 11 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12852552 - PreCommit-HIVE-Build

> Improvements in Hive Unit Test by using In-memory Derby DB
> --
>
> Key: HIVE-15910
> URL: https://issues.apache.org/jira/browse/HIVE-15910
> Project: Hive
>  Issue Type: Test
>  Components: Tests
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
> Attachments: HIVE-15910.01.patch
>
>
> Hive UT currently uses Derby DB with storage on disk which have some 
> practical problems.
> 1. The run-time of Hive unit tests are high as need to operate on the disk 
> quite often.
> 2. It can cause conflict if multiple test cases operates on the same table 
> name (such as table being created already exist).
> To solve these problems, we shall use an in-memory storage option of Derby DB 
> which can be even persisted if the test case demands that.
> https://db.apache.org/derby/docs/10.8/devguide/cdevdvlpinmemdb.html



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-15910) Improvements in Hive Unit Test by using In-memory Derby DB

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-15910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15865657#comment-15865657
 ] 

ASF GitHub Bot commented on HIVE-15910:
---

GitHub user sankarh opened a pull request:

https://github.com/apache/hive/pull/147

HIVE-15910: Improvements in Hive Unit Test by using In-memory Derby DB



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sankarh/hive HIVE-15910

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/hive/pull/147.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #147


commit 4c6596e94524eb8ac0066e1371ed6ceb2c650036
Author: Sankar Hariappan 
Date:   2017-02-14T11:39:02Z

HIVE-15910: Improvements in Hive Unit Test by using In-memory Derby DB




> Improvements in Hive Unit Test by using In-memory Derby DB
> --
>
> Key: HIVE-15910
> URL: https://issues.apache.org/jira/browse/HIVE-15910
> Project: Hive
>  Issue Type: Test
>  Components: Tests
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
> Attachments: HIVE-15910.01.patch
>
>
> Hive UT currently uses Derby DB with storage on disk which have some 
> practical problems.
> 1. The run-time of Hive unit tests are high as need to operate on the disk 
> quite often.
> 2. It can cause conflict if multiple test cases operates on the same table 
> name (such as table being created already exist).
> To solve these problems, we shall use an in-memory storage option of Derby DB 
> which can be even persisted if the test case demands that.
> https://db.apache.org/derby/docs/10.8/devguide/cdevdvlpinmemdb.html



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-15910) Improvements in Hive Unit Test by using In-memory Derby DB

[Sankar Hariappan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-15910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15865655#comment-15865655
 ] 

Sankar Hariappan commented on HIVE-15910:
-

[~thejas] [~wzheng] 
Can you please review the patch and help to commit?

> Improvements in Hive Unit Test by using In-memory Derby DB
> --
>
> Key: HIVE-15910
> URL: https://issues.apache.org/jira/browse/HIVE-15910
> Project: Hive
>  Issue Type: Test
>  Components: Tests
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
> Attachments: HIVE-15910.01.patch
>
>
> Hive UT currently uses Derby DB with storage on disk which have some 
> practical problems.
> 1. The run-time of Hive unit tests are high as need to operate on the disk 
> quite often.
> 2. It can cause conflict if multiple test cases operates on the same table 
> name (such as table being created already exist).
> To solve these problems, we shall use an in-memory storage option of Derby DB 
> which can be even persisted if the test case demands that.
> https://db.apache.org/derby/docs/10.8/devguide/cdevdvlpinmemdb.html



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)