[jira] [Commented] (HIVE-17234) Remove HBase metastore from master

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-17234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16261689#comment-16261689
 ] 

ASF GitHub Bot commented on HIVE-17234:
---

Github user alanfgates closed the pull request at:

https://github.com/apache/hive/pull/218


> Remove HBase metastore from master
> --
>
> Key: HIVE-17234
> URL: https://issues.apache.org/jira/browse/HIVE-17234
> Project: Hive
>  Issue Type: Task
>  Components: HBase Metastore
>Affects Versions: 3.0.0
>Reporter: Alan Gates
>Assignee: Alan Gates
>  Labels: pull-request-available
> Fix For: 3.0.0
>
> Attachments: HIVE-17234.patch
>
>
> No new development has been done on the HBase metastore in at least a year, 
> and to my knowledge no one is using it (nor is it even in a state to be fully 
> usable).  Given the lack of interest in continuing to develop it, we should 
> remove it rather than leave dead code hanging around and extra tests taking 
> up time in test runs.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17234) Remove HBase metastore from master

[Lefty Leverenz (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-17234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16119476#comment-16119476
 ] 

Lefty Leverenz commented on HIVE-17234:
---

Doc done for the configs:

* [Configuration Properties -- Hive Metastore HBase | 
https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-HiveMetastoreHBase]

(I'm still wondering why *hive.metastore.hbase.file.metadata.threads* wasn't 
removed.)

> Remove HBase metastore from master
> --
>
> Key: HIVE-17234
> URL: https://issues.apache.org/jira/browse/HIVE-17234
> Project: Hive
>  Issue Type: Task
>  Components: HBase Metastore
>Affects Versions: 3.0.0
>Reporter: Alan Gates
>Assignee: Alan Gates
> Fix For: 3.0.0
>
> Attachments: HIVE-17234.patch
>
>
> No new development has been done on the HBase metastore in at least a year, 
> and to my knowledge no one is using it (nor is it even in a state to be fully 
> usable).  Given the lack of interest in continuing to develop it, we should 
> remove it rather than leave dead code hanging around and extra tests taking 
> up time in test runs.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17234) Remove HBase metastore from master

[Lefty Leverenz (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-17234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117879#comment-16117879
 ] 

Lefty Leverenz commented on HIVE-17234:
---

Doc note:  This removes 15 hive.metastore.hbase.* configs from HiveConf.java.  
(Why wasn't *hive.metastore.hbase.file.metadata.threads* removed?)

Most of them haven't been documented in the wiki yet.  The section "Hive 
Metastore HBase" in Configuration Properties only has 3 configs:

* [hive.metastore.hbase.cache.size | 
https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-hive.metastore.hbase.cache.size]
 (_still in the code_)
* [hive.metastore.hbase.cache.ttl | 
https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-hive.metastore.hbase.cache.ttl]
 (_removed by this patch, so need to update wiki_)
* [hive.metastore.hbase.cache.size | 
https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-hive.metastore.hbase.cache.size]
 (_removed by HIVE-9693 before the 2.0.0 release, so didn't belong in the wiki 
in the first place_)

Here's the complete list, just for the record:

* hive.metastore.hbase.catalog.cache.size
* hive.metastore.hbase.aggregate.stats.cache.size
* hive.metastore.hbase.aggregate.stats.max.partitions
* hive.metastore.hbase.aggregate.stats.false.positive.probability
* hive.metastore.hbase.aggregate.stats.max.variance
* hive.metastore.hbase.cache.ttl (_documented in wiki_)
* hive.metastore.hbase.cache.max.writer.wait
* hive.metastore.hbase.cache.max.reader.wait
* hive.metastore.hbase.cache.max.full
* hive.metastore.hbase.cache.clean.until
* hive.metastore.hbase.connection.class
* hive.metastore.hbase.aggr.stats.cache.entries
* hive.metastore.hbase.aggr.stats.memory.ttl
* hive.metastore.hbase.aggr.stats.invalidator.frequency
* hive.metastore.hbase.aggr.stats.hbase.ttl
* hive.metastore.hbase.file.metadata.threads (_documented in wiki; not removed 
here_)

> Remove HBase metastore from master
> --
>
> Key: HIVE-17234
> URL: https://issues.apache.org/jira/browse/HIVE-17234
> Project: Hive
>  Issue Type: Task
>  Components: HBase Metastore
>Affects Versions: 3.0.0
>Reporter: Alan Gates
>Assignee: Alan Gates
> Fix For: 3.0.0
>
> Attachments: HIVE-17234.patch
>
>
> No new development has been done on the HBase metastore in at least a year, 
> and to my knowledge no one is using it (nor is it even in a state to be fully 
> usable).  Given the lack of interest in continuing to develop it, we should 
> remove it rather than leave dead code hanging around and extra tests taking 
> up time in test runs.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17234) Remove HBase metastore from master

[Sergey Shelukhin (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-17234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113260#comment-16113260
 ] 

Sergey Shelukhin commented on HIVE-17234:
-

+1, I'm assuming any green in the pull request is just the code that was 
relocated

> Remove HBase metastore from master
> --
>
> Key: HIVE-17234
> URL: https://issues.apache.org/jira/browse/HIVE-17234
> Project: Hive
>  Issue Type: Task
>  Components: HBase Metastore
>Affects Versions: 3.0.0
>Reporter: Alan Gates
>Assignee: Alan Gates
> Attachments: HIVE-17234.patch
>
>
> No new development has been done on the HBase metastore in at least a year, 
> and to my knowledge no one is using it (nor is it even in a state to be fully 
> usable).  Given the lack of interest in continuing to develop it, we should 
> remove it rather than leave dead code hanging around and extra tests taking 
> up time in test runs.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17234) Remove HBase metastore from master

[Daniel Dai (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-17234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113242#comment-16113242
 ] 

Daniel Dai commented on HIVE-17234:
---

+1

> Remove HBase metastore from master
> --
>
> Key: HIVE-17234
> URL: https://issues.apache.org/jira/browse/HIVE-17234
> Project: Hive
>  Issue Type: Task
>  Components: HBase Metastore
>Affects Versions: 3.0.0
>Reporter: Alan Gates
>Assignee: Alan Gates
> Attachments: HIVE-17234.patch
>
>
> No new development has been done on the HBase metastore in at least a year, 
> and to my knowledge no one is using it (nor is it even in a state to be fully 
> usable).  Given the lack of interest in continuing to develop it, we should 
> remove it rather than leave dead code hanging around and extra tests taking 
> up time in test runs.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17234) Remove HBase metastore from master

[Alan Gates (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-17234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16112892#comment-16112892
 ] 

Alan Gates commented on HIVE-17234:
---

[~sershe], [~daijy] can you take a look and make sure I didn't break anything 
you are using or planning to use?

> Remove HBase metastore from master
> --
>
> Key: HIVE-17234
> URL: https://issues.apache.org/jira/browse/HIVE-17234
> Project: Hive
>  Issue Type: Task
>  Components: HBase Metastore
>Affects Versions: 3.0.0
>Reporter: Alan Gates
>Assignee: Alan Gates
> Attachments: HIVE-17234.patch
>
>
> No new development has been done on the HBase metastore in at least a year, 
> and to my knowledge no one is using it (nor is it even in a state to be fully 
> usable).  Given the lack of interest in continuing to develop it, we should 
> remove it rather than leave dead code hanging around and extra tests taking 
> up time in test runs.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17234) Remove HBase metastore from master

[Alan Gates (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-17234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16112891#comment-16112891
 ] 

Alan Gates commented on HIVE-17234:
---

Me too :(

Thanks, I forgot to check the docs.

> Remove HBase metastore from master
> --
>
> Key: HIVE-17234
> URL: https://issues.apache.org/jira/browse/HIVE-17234
> Project: Hive
>  Issue Type: Task
>  Components: HBase Metastore
>Affects Versions: 3.0.0
>Reporter: Alan Gates
>Assignee: Alan Gates
> Attachments: HIVE-17234.patch
>
>
> No new development has been done on the HBase metastore in at least a year, 
> and to my knowledge no one is using it (nor is it even in a state to be fully 
> usable).  Given the lack of interest in continuing to develop it, we should 
> remove it rather than leave dead code hanging around and extra tests taking 
> up time in test runs.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17234) Remove HBase metastore from master

[Lefty Leverenz (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-17234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16112251#comment-16112251
 ] 

Lefty Leverenz commented on HIVE-17234:
---

Sorry to see it go.

Doc note:  The design doc should be moved to the "Abandoned" list.

* [Design Docs | https://cwiki.apache.org/confluence/display/Hive/DesignDocs]
** [HBase Metastore Development Guide | 
https://cwiki.apache.org/confluence/display/Hive/HBaseMetastoreDevelopmentGuide]

> Remove HBase metastore from master
> --
>
> Key: HIVE-17234
> URL: https://issues.apache.org/jira/browse/HIVE-17234
> Project: Hive
>  Issue Type: Task
>  Components: HBase Metastore
>Affects Versions: 3.0.0
>Reporter: Alan Gates
>Assignee: Alan Gates
> Attachments: HIVE-17234.patch
>
>
> No new development has been done on the HBase metastore in at least a year, 
> and to my knowledge no one is using it (nor is it even in a state to be fully 
> usable).  Given the lack of interest in continuing to develop it, we should 
> remove it rather than leave dead code hanging around and extra tests taking 
> up time in test runs.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17234) Remove HBase metastore from master

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-17234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16112185#comment-16112185
 ] 

Hive QA commented on HIVE-17234:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12880133/HIVE-17234.patch

{color:green}SUCCESS:{color} +1 due to 4 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 8 failed/errored test(s), 10961 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[llap_uncompressed] 
(batchId=56)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_vectorized_dynamic_partition_pruning]
 (batchId=168)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14] 
(batchId=234)
org.apache.hive.hcatalog.api.TestHCatClient.testPartitionRegistrationWithCustomSchema
 (batchId=179)
org.apache.hive.hcatalog.api.TestHCatClient.testPartitionSpecRegistrationWithCustomSchema
 (batchId=179)
org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation 
(batchId=179)
org.apache.hive.hcatalog.pig.TestRCFileHCatStorer.testWriteTimestamp 
(batchId=182)
org.apache.hive.hcatalog.pig.TestTextFileHCatStorer.testWriteSmallint 
(batchId=182)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/6240/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/6240/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-6240/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 8 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12880133 - PreCommit-HIVE-Build

> Remove HBase metastore from master
> --
>
> Key: HIVE-17234
> URL: https://issues.apache.org/jira/browse/HIVE-17234
> Project: Hive
>  Issue Type: Task
>  Components: HBase Metastore
>Affects Versions: 3.0.0
>Reporter: Alan Gates
>Assignee: Alan Gates
> Attachments: HIVE-17234.patch
>
>
> No new development has been done on the HBase metastore in at least a year, 
> and to my knowledge no one is using it (nor is it even in a state to be fully 
> usable).  Given the lack of interest in continuing to develop it, we should 
> remove it rather than leave dead code hanging around and extra tests taking 
> up time in test runs.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17234) Remove HBase metastore from master

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-17234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16111981#comment-16111981
 ] 

ASF GitHub Bot commented on HIVE-17234:
---

GitHub user alanfgates opened a pull request:

https://github.com/apache/hive/pull/218

HIVE-17234

This patch removes all of the unused parts of HBase metastore.  The 
aggregate stats work is kept, and moves into directories that match the already 
changed packages.  Two methods that were still used from HBaseUtils move into 
MetaStoreUtils.  FileMetadata remains in the hbase package, since I believe 
Sergey wants to use it sometime in the future.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/alanfgates/hive hive17234

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/hive/pull/218.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #218


commit 6afb3e1335e3541b6775a58d00b383cc36024d18
Author: Alan Gates 
Date:   2017-08-03T00:12:09Z

HIVE-17234 Remove HBase metastore from master

commit fbce5296c95f38bb56ff2ba3a670f02533ed6661
Author: Alan Gates 
Date:   2017-08-03T00:32:59Z

Removed one more file I missed previously.




> Remove HBase metastore from master
> --
>
> Key: HIVE-17234
> URL: https://issues.apache.org/jira/browse/HIVE-17234
> Project: Hive
>  Issue Type: Task
>  Components: HBase Metastore
>Affects Versions: 3.0.0
>Reporter: Alan Gates
>Assignee: Alan Gates
>
> No new development has been done on the HBase metastore in at least a year, 
> and to my knowledge no one is using it (nor is it even in a state to be fully 
> usable).  Given the lack of interest in continuing to develop it, we should 
> remove it rather than leave dead code hanging around and extra tests taking 
> up time in test runs.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)