[jira] [Commented] (HIVE-17234) Remove HBase metastore from master
[ https://issues.apache.org/jira/browse/HIVE-17234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16261689#comment-16261689 ] ASF GitHub Bot commented on HIVE-17234: --- Github user alanfgates closed the pull request at: https://github.com/apache/hive/pull/218 > Remove HBase metastore from master > -- > > Key: HIVE-17234 > URL: https://issues.apache.org/jira/browse/HIVE-17234 > Project: Hive > Issue Type: Task > Components: HBase Metastore >Affects Versions: 3.0.0 >Reporter: Alan Gates >Assignee: Alan Gates > Labels: pull-request-available > Fix For: 3.0.0 > > Attachments: HIVE-17234.patch > > > No new development has been done on the HBase metastore in at least a year, > and to my knowledge no one is using it (nor is it even in a state to be fully > usable). Given the lack of interest in continuing to develop it, we should > remove it rather than leave dead code hanging around and extra tests taking > up time in test runs. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17234) Remove HBase metastore from master
[ https://issues.apache.org/jira/browse/HIVE-17234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16119476#comment-16119476 ] Lefty Leverenz commented on HIVE-17234: --- Doc done for the configs: * [Configuration Properties -- Hive Metastore HBase | https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-HiveMetastoreHBase] (I'm still wondering why *hive.metastore.hbase.file.metadata.threads* wasn't removed.) > Remove HBase metastore from master > -- > > Key: HIVE-17234 > URL: https://issues.apache.org/jira/browse/HIVE-17234 > Project: Hive > Issue Type: Task > Components: HBase Metastore >Affects Versions: 3.0.0 >Reporter: Alan Gates >Assignee: Alan Gates > Fix For: 3.0.0 > > Attachments: HIVE-17234.patch > > > No new development has been done on the HBase metastore in at least a year, > and to my knowledge no one is using it (nor is it even in a state to be fully > usable). Given the lack of interest in continuing to develop it, we should > remove it rather than leave dead code hanging around and extra tests taking > up time in test runs. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17234) Remove HBase metastore from master
[ https://issues.apache.org/jira/browse/HIVE-17234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16117879#comment-16117879 ] Lefty Leverenz commented on HIVE-17234: --- Doc note: This removes 15 hive.metastore.hbase.* configs from HiveConf.java. (Why wasn't *hive.metastore.hbase.file.metadata.threads* removed?) Most of them haven't been documented in the wiki yet. The section "Hive Metastore HBase" in Configuration Properties only has 3 configs: * [hive.metastore.hbase.cache.size | https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-hive.metastore.hbase.cache.size] (_still in the code_) * [hive.metastore.hbase.cache.ttl | https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-hive.metastore.hbase.cache.ttl] (_removed by this patch, so need to update wiki_) * [hive.metastore.hbase.cache.size | https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-hive.metastore.hbase.cache.size] (_removed by HIVE-9693 before the 2.0.0 release, so didn't belong in the wiki in the first place_) Here's the complete list, just for the record: * hive.metastore.hbase.catalog.cache.size * hive.metastore.hbase.aggregate.stats.cache.size * hive.metastore.hbase.aggregate.stats.max.partitions * hive.metastore.hbase.aggregate.stats.false.positive.probability * hive.metastore.hbase.aggregate.stats.max.variance * hive.metastore.hbase.cache.ttl (_documented in wiki_) * hive.metastore.hbase.cache.max.writer.wait * hive.metastore.hbase.cache.max.reader.wait * hive.metastore.hbase.cache.max.full * hive.metastore.hbase.cache.clean.until * hive.metastore.hbase.connection.class * hive.metastore.hbase.aggr.stats.cache.entries * hive.metastore.hbase.aggr.stats.memory.ttl * hive.metastore.hbase.aggr.stats.invalidator.frequency * hive.metastore.hbase.aggr.stats.hbase.ttl * hive.metastore.hbase.file.metadata.threads (_documented in wiki; not removed here_) > Remove HBase metastore from master > -- > > Key: HIVE-17234 > URL: https://issues.apache.org/jira/browse/HIVE-17234 > Project: Hive > Issue Type: Task > Components: HBase Metastore >Affects Versions: 3.0.0 >Reporter: Alan Gates >Assignee: Alan Gates > Fix For: 3.0.0 > > Attachments: HIVE-17234.patch > > > No new development has been done on the HBase metastore in at least a year, > and to my knowledge no one is using it (nor is it even in a state to be fully > usable). Given the lack of interest in continuing to develop it, we should > remove it rather than leave dead code hanging around and extra tests taking > up time in test runs. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17234) Remove HBase metastore from master
[ https://issues.apache.org/jira/browse/HIVE-17234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16113260#comment-16113260 ] Sergey Shelukhin commented on HIVE-17234: - +1, I'm assuming any green in the pull request is just the code that was relocated > Remove HBase metastore from master > -- > > Key: HIVE-17234 > URL: https://issues.apache.org/jira/browse/HIVE-17234 > Project: Hive > Issue Type: Task > Components: HBase Metastore >Affects Versions: 3.0.0 >Reporter: Alan Gates >Assignee: Alan Gates > Attachments: HIVE-17234.patch > > > No new development has been done on the HBase metastore in at least a year, > and to my knowledge no one is using it (nor is it even in a state to be fully > usable). Given the lack of interest in continuing to develop it, we should > remove it rather than leave dead code hanging around and extra tests taking > up time in test runs. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17234) Remove HBase metastore from master
[ https://issues.apache.org/jira/browse/HIVE-17234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16113242#comment-16113242 ] Daniel Dai commented on HIVE-17234: --- +1 > Remove HBase metastore from master > -- > > Key: HIVE-17234 > URL: https://issues.apache.org/jira/browse/HIVE-17234 > Project: Hive > Issue Type: Task > Components: HBase Metastore >Affects Versions: 3.0.0 >Reporter: Alan Gates >Assignee: Alan Gates > Attachments: HIVE-17234.patch > > > No new development has been done on the HBase metastore in at least a year, > and to my knowledge no one is using it (nor is it even in a state to be fully > usable). Given the lack of interest in continuing to develop it, we should > remove it rather than leave dead code hanging around and extra tests taking > up time in test runs. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17234) Remove HBase metastore from master
[ https://issues.apache.org/jira/browse/HIVE-17234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16112892#comment-16112892 ] Alan Gates commented on HIVE-17234: --- [~sershe], [~daijy] can you take a look and make sure I didn't break anything you are using or planning to use? > Remove HBase metastore from master > -- > > Key: HIVE-17234 > URL: https://issues.apache.org/jira/browse/HIVE-17234 > Project: Hive > Issue Type: Task > Components: HBase Metastore >Affects Versions: 3.0.0 >Reporter: Alan Gates >Assignee: Alan Gates > Attachments: HIVE-17234.patch > > > No new development has been done on the HBase metastore in at least a year, > and to my knowledge no one is using it (nor is it even in a state to be fully > usable). Given the lack of interest in continuing to develop it, we should > remove it rather than leave dead code hanging around and extra tests taking > up time in test runs. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17234) Remove HBase metastore from master
[ https://issues.apache.org/jira/browse/HIVE-17234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16112891#comment-16112891 ] Alan Gates commented on HIVE-17234: --- Me too :( Thanks, I forgot to check the docs. > Remove HBase metastore from master > -- > > Key: HIVE-17234 > URL: https://issues.apache.org/jira/browse/HIVE-17234 > Project: Hive > Issue Type: Task > Components: HBase Metastore >Affects Versions: 3.0.0 >Reporter: Alan Gates >Assignee: Alan Gates > Attachments: HIVE-17234.patch > > > No new development has been done on the HBase metastore in at least a year, > and to my knowledge no one is using it (nor is it even in a state to be fully > usable). Given the lack of interest in continuing to develop it, we should > remove it rather than leave dead code hanging around and extra tests taking > up time in test runs. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17234) Remove HBase metastore from master
[ https://issues.apache.org/jira/browse/HIVE-17234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16112251#comment-16112251 ] Lefty Leverenz commented on HIVE-17234: --- Sorry to see it go. Doc note: The design doc should be moved to the "Abandoned" list. * [Design Docs | https://cwiki.apache.org/confluence/display/Hive/DesignDocs] ** [HBase Metastore Development Guide | https://cwiki.apache.org/confluence/display/Hive/HBaseMetastoreDevelopmentGuide] > Remove HBase metastore from master > -- > > Key: HIVE-17234 > URL: https://issues.apache.org/jira/browse/HIVE-17234 > Project: Hive > Issue Type: Task > Components: HBase Metastore >Affects Versions: 3.0.0 >Reporter: Alan Gates >Assignee: Alan Gates > Attachments: HIVE-17234.patch > > > No new development has been done on the HBase metastore in at least a year, > and to my knowledge no one is using it (nor is it even in a state to be fully > usable). Given the lack of interest in continuing to develop it, we should > remove it rather than leave dead code hanging around and extra tests taking > up time in test runs. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17234) Remove HBase metastore from master
[ https://issues.apache.org/jira/browse/HIVE-17234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16112185#comment-16112185 ] Hive QA commented on HIVE-17234: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12880133/HIVE-17234.patch {color:green}SUCCESS:{color} +1 due to 4 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 8 failed/errored test(s), 10961 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[llap_uncompressed] (batchId=56) org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_vectorized_dynamic_partition_pruning] (batchId=168) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14] (batchId=234) org.apache.hive.hcatalog.api.TestHCatClient.testPartitionRegistrationWithCustomSchema (batchId=179) org.apache.hive.hcatalog.api.TestHCatClient.testPartitionSpecRegistrationWithCustomSchema (batchId=179) org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation (batchId=179) org.apache.hive.hcatalog.pig.TestRCFileHCatStorer.testWriteTimestamp (batchId=182) org.apache.hive.hcatalog.pig.TestTextFileHCatStorer.testWriteSmallint (batchId=182) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/6240/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/6240/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-6240/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 8 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12880133 - PreCommit-HIVE-Build > Remove HBase metastore from master > -- > > Key: HIVE-17234 > URL: https://issues.apache.org/jira/browse/HIVE-17234 > Project: Hive > Issue Type: Task > Components: HBase Metastore >Affects Versions: 3.0.0 >Reporter: Alan Gates >Assignee: Alan Gates > Attachments: HIVE-17234.patch > > > No new development has been done on the HBase metastore in at least a year, > and to my knowledge no one is using it (nor is it even in a state to be fully > usable). Given the lack of interest in continuing to develop it, we should > remove it rather than leave dead code hanging around and extra tests taking > up time in test runs. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17234) Remove HBase metastore from master
[ https://issues.apache.org/jira/browse/HIVE-17234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111981#comment-16111981 ] ASF GitHub Bot commented on HIVE-17234: --- GitHub user alanfgates opened a pull request: https://github.com/apache/hive/pull/218 HIVE-17234 This patch removes all of the unused parts of HBase metastore. The aggregate stats work is kept, and moves into directories that match the already changed packages. Two methods that were still used from HBaseUtils move into MetaStoreUtils. FileMetadata remains in the hbase package, since I believe Sergey wants to use it sometime in the future. You can merge this pull request into a Git repository by running: $ git pull https://github.com/alanfgates/hive hive17234 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/hive/pull/218.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #218 commit 6afb3e1335e3541b6775a58d00b383cc36024d18 Author: Alan Gates Date: 2017-08-03T00:12:09Z HIVE-17234 Remove HBase metastore from master commit fbce5296c95f38bb56ff2ba3a670f02533ed6661 Author: Alan Gates Date: 2017-08-03T00:32:59Z Removed one more file I missed previously. > Remove HBase metastore from master > -- > > Key: HIVE-17234 > URL: https://issues.apache.org/jira/browse/HIVE-17234 > Project: Hive > Issue Type: Task > Components: HBase Metastore >Affects Versions: 3.0.0 >Reporter: Alan Gates >Assignee: Alan Gates > > No new development has been done on the HBase metastore in at least a year, > and to my knowledge no one is using it (nor is it even in a state to be fully > usable). Given the lack of interest in continuing to develop it, we should > remove it rather than leave dead code hanging around and extra tests taking > up time in test runs. -- This message was sent by Atlassian JIRA (v6.4.14#64029)