[jira] [Created] (HIVE-23086) Two tests fail on branch-2.3
Alan Gates created HIVE-23086:
-
Summary: Two tests fail on branch-2.3
Key: HIVE-23086
URL: https://issues.apache.org/jira/browse/HIVE-23086
Project: Hive
Issue Type: Bug
Components: Test
Affects Versions: 2.3.6
Reporter: Alan Gates
Assignee: Alan Gates
TestPerfCli query88.q and TestMiniLlapLocalCliDriver union_fast_stats.q fail on
the 2.3 branch.
The TestMiniLlapLocalCliDriver failure is fixed in HIVE-14977 where
union_fast_stats.q is removed from the list of tests run with
TestMiniLlapLocalCliDriver.
The TestPerfCli failure is fixed HIVE-16602 where one line is added to
query88.q to allow cartesian products:
{{set hive.strict.checks.cartesian.product=false;}}
I propose to backport these two changes to branch-2.3
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HIVE-21850) branch-3 metastore installation installs wrong version
Alan Gates created HIVE-21850: - Summary: branch-3 metastore installation installs wrong version Key: HIVE-21850 URL: https://issues.apache.org/jira/browse/HIVE-21850 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 3.2.0 Reporter: Alan Gates Assignee: Alan Gates Fix For: 3.2.0 hive.version.shortname in standalone-metastore/pom.xml was not properly updated in branch-3. It is still set to 3.1.0, which causes the MetastoreSchemaTool to install the wrong version. Part of this Jira should include updating the HowToRelease doc to include updating this value. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-21826) Backport HIVE-21786 to branch-3
Alan Gates created HIVE-21826: - Summary: Backport HIVE-21786 to branch-3 Key: HIVE-21826 URL: https://issues.apache.org/jira/browse/HIVE-21826 Project: Hive Issue Type: Bug Reporter: Alan Gates Assignee: Alan Gates Missed branch-3 in the original fix. Need to apply the patch here as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-21809) Backport HIVE-21786 to branch-2.3
Alan Gates created HIVE-21809: - Summary: Backport HIVE-21786 to branch-2.3 Key: HIVE-21809 URL: https://issues.apache.org/jira/browse/HIVE-21809 Project: Hive Issue Type: Bug Affects Versions: 2.3.5 Reporter: Alan Gates Assignee: Alan Gates Fix For: 2.3.6 Need to update the URLs in the pom on branch-2.3 as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-21808) Backport HIVE-21786 to branch-3.1
Alan Gates created HIVE-21808: - Summary: Backport HIVE-21786 to branch-3.1 Key: HIVE-21808 URL: https://issues.apache.org/jira/browse/HIVE-21808 Project: Hive Issue Type: Bug Reporter: Alan Gates Assignee: Alan Gates Fix For: 3.1.2 Need to update the URLs in the poms for branch 3 as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-21786) Update repo URLs in poms
Alan Gates created HIVE-21786: - Summary: Update repo URLs in poms Key: HIVE-21786 URL: https://issues.apache.org/jira/browse/HIVE-21786 Project: Hive Issue Type: Bug Components: Build Infrastructure Affects Versions: 2.3.5, 3.1.1, 4.0.0 Reporter: Alan Gates Assignee: Alan Gates Need to update repo URLs in the poms. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-21758) DBInstall tests broken on master and branch-3.1
Alan Gates created HIVE-21758: - Summary: DBInstall tests broken on master and branch-3.1 Key: HIVE-21758 URL: https://issues.apache.org/jira/browse/HIVE-21758 Project: Hive Issue Type: Bug Components: Metastore, Tests Affects Versions: 3.1.1 Reporter: Alan Gates Assignee: Alan Gates The Oracle and SqlServer install and upgrade tests in standalone-metastore fail in master and branch-3.1. In the Oracle case it appears the docker container that was used no longer exists. For SqlServer the cause of the failures is not immediately clear. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-21616) Implement JSON_VALUE, JSON_QUERY, and IS [NOT] JSON
Alan Gates created HIVE-21616: - Summary: Implement JSON_VALUE, JSON_QUERY, and IS [NOT] JSON Key: HIVE-21616 URL: https://issues.apache.org/jira/browse/HIVE-21616 Project: Hive Issue Type: Sub-task Components: UDF Affects Versions: 4.0.0 Reporter: Alan Gates Assignee: Alan Gates -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-21615) Implement SQL:2016 JSON functions
Alan Gates created HIVE-21615: - Summary: Implement SQL:2016 JSON functions Key: HIVE-21615 URL: https://issues.apache.org/jira/browse/HIVE-21615 Project: Hive Issue Type: New Feature Components: UDF Affects Versions: 4.0.0 Reporter: Alan Gates Assignee: Alan Gates SQL:2016 specifies several new functions for processing JSON in SQL. Hive should implement these functions. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-20451) Metastore client and server tarball issues
Alan Gates created HIVE-20451: - Summary: Metastore client and server tarball issues Key: HIVE-20451 URL: https://issues.apache.org/jira/browse/HIVE-20451 Project: Hive Issue Type: Bug Components: Standalone Metastore Affects Versions: 4.0.0 Reporter: Alan Gates With the split of the metastore into common and server there are now two sets of tarballs. There are a couple of issues here. # It doesn't make sense to have separate source tarballs for each. The source release should still be done from the standalone-metastore directory and include all code for the metastore. # The binary tarballs should have separate names. At the moment both are named apache-hive-metastore. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-20106) Backport HIVE-20060 (HiveSchemaTool and MetastoreSchemaTool refactor) to branch-3
Alan Gates created HIVE-20106: - Summary: Backport HIVE-20060 (HiveSchemaTool and MetastoreSchemaTool refactor) to branch-3 Key: HIVE-20106 URL: https://issues.apache.org/jira/browse/HIVE-20106 Project: Hive Issue Type: Task Components: Beeline, Metastore Reporter: Alan Gates Assignee: Alan Gates -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-20060) Refactor HiveSchemaTool and MetastoreSchemaTool
Alan Gates created HIVE-20060: - Summary: Refactor HiveSchemaTool and MetastoreSchemaTool Key: HIVE-20060 URL: https://issues.apache.org/jira/browse/HIVE-20060 Project: Hive Issue Type: Task Components: Beeline, Metastore Reporter: Alan Gates Assignee: Alan Gates These two classes are 95% the same. Now that HIVE-19711 has split HiveSchemaTool into multiple components it will be much easier to refactor these so that there is only one version of the code that each shares. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-20058) Backport HIVE-19711 (HiveSchemaTool refactor) to branch-3
Alan Gates created HIVE-20058: - Summary: Backport HIVE-19711 (HiveSchemaTool refactor) to branch-3 Key: HIVE-20058 URL: https://issues.apache.org/jira/browse/HIVE-20058 Project: Hive Issue Type: Task Components: Beeline Reporter: Alan Gates Assignee: Alan Gates -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-19984) Backport HIVE-15976 to branch-3
Alan Gates created HIVE-19984: - Summary: Backport HIVE-15976 to branch-3 Key: HIVE-19984 URL: https://issues.apache.org/jira/browse/HIVE-19984 Project: Hive Issue Type: Bug Components: UDF Affects Versions: 3.1.0 Reporter: Alan Gates Assignee: Alan Gates -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-19983) Backport HIVE-19769 to branch-3
Alan Gates created HIVE-19983: - Summary: Backport HIVE-19769 to branch-3 Key: HIVE-19983 URL: https://issues.apache.org/jira/browse/HIVE-19983 Project: Hive Issue Type: Bug Components: storage-api Affects Versions: 3.1.0 Reporter: Alan Gates Assignee: Alan Gates This patch will be needed for other catalog related work to be backported to branch-3. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-19982) Make JDBC work with Catalogs
Alan Gates created HIVE-19982: - Summary: Make JDBC work with Catalogs Key: HIVE-19982 URL: https://issues.apache.org/jira/browse/HIVE-19982 Project: Hive Issue Type: Sub-task Components: JDBC Affects Versions: 3.0.0 Reporter: Alan Gates Many JDBC methods include a catalog specifier in the call. We need to update these to work with multiple catalogs. Also we will need to also support the JDBC calls for catalogs. This at least includes Connection: getCatalog() and setCatalog() DatabaseMetaData: getCatalogs(), getCatalogSeparator(), and getCatalogTerm() and maybe others. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-19871) add_partitions does not properly handle client being configured with a non-Hive catalog.
Alan Gates created HIVE-19871:
-
Summary: add_partitions does not properly handle client being
configured with a non-Hive catalog.
Key: HIVE-19871
URL: https://issues.apache.org/jira/browse/HIVE-19871
Project: Hive
Issue Type: Bug
Components: Metastore
Affects Versions: 3.0.0
Reporter: Alan Gates
Assignee: Alan Gates
If a client calls
{{add_partitions(List parts, boolean ifNotExists, boolean
needResults)}}
and the catalog name is set to a non-default value in the config file but unset
in the partitions, the request to add the partition will fail with an error
message "table not found" even when the table is valid.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HIVE-19822) Make new streaming interface work with catalogs
Alan Gates created HIVE-19822: - Summary: Make new streaming interface work with catalogs Key: HIVE-19822 URL: https://issues.apache.org/jira/browse/HIVE-19822 Project: Hive Issue Type: Sub-task Components: Streaming Affects Versions: 3.0.0 Reporter: Alan Gates Assignee: Alan Gates The new hive-streaming module should respect catalogs. This will not include hive-hcatalog-streaming, since that module is deprecated it doesn't make sense to update it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-19806) Several tests do not properly sort their output
Alan Gates created HIVE-19806: - Summary: Several tests do not properly sort their output Key: HIVE-19806 URL: https://issues.apache.org/jira/browse/HIVE-19806 Project: Hive Issue Type: Bug Components: Test Affects Versions: 3.0.0 Reporter: Alan Gates Assignee: Alan Gates A number of the tests produce unsorted output that happens to come out the same on people's laptops and the ptest infrastructure. But when run on a separate linux box the sort differences show up. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-19803) Make import/export work with catalogs
Alan Gates created HIVE-19803: - Summary: Make import/export work with catalogs Key: HIVE-19803 URL: https://issues.apache.org/jira/browse/HIVE-19803 Project: Hive Issue Type: Sub-task Components: Import/Export Affects Versions: 3.0.0 Reporter: Alan Gates Assignee: Alan Gates Import and export will need changes to handle the catalogs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-19802) Modify parser to accept catalog in object names
Alan Gates created HIVE-19802: - Summary: Modify parser to accept catalog in object names Key: HIVE-19802 URL: https://issues.apache.org/jira/browse/HIVE-19802 Project: Hive Issue Type: Sub-task Components: Parser Affects Versions: 3.0.0 Reporter: Alan Gates Assignee: Alan Gates Objects (databases, tables, functions, etc.) should be addressable via catalog. That is, a database should be addressable as catalog.database or database, tables should be addressable as catalog.database.table or database.table or table, etc. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-19791) Modify TableDesc to contain the catalog
Alan Gates created HIVE-19791: - Summary: Modify TableDesc to contain the catalog Key: HIVE-19791 URL: https://issues.apache.org/jira/browse/HIVE-19791 Project: Hive Issue Type: Sub-task Components: Query Planning Affects Versions: 3.0.0 Reporter: Alan Gates Assignee: Alan Gates TableDesc currently only contains a table's database and tablename. It needs to also have the catalog name. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-19769) Create dedicated objects for DB and Table names
Alan Gates created HIVE-19769: - Summary: Create dedicated objects for DB and Table names Key: HIVE-19769 URL: https://issues.apache.org/jira/browse/HIVE-19769 Project: Hive Issue Type: Sub-task Components: storage-api Affects Versions: 3.0.0 Reporter: Alan Gates Assignee: Alan Gates Currently table names are always strings. Sometimes that string is just tablename, sometimes it is dbname.tablename. Sometimes the code expects one or the other, sometimes it handles either. This is burdensome for developers and error prone. With the addition of catalog to the hierarchy, this becomes even worse. I propose to add two objects, DatabaseName and TableName. These will track full names of each object. They will handle inserting default catalog and database names when those are not provided. They will handle the conversions to and from strings. These will need to be added to storage-api because ValidTxnList will use it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-19688) Make catalogs updatable
Alan Gates created HIVE-19688: - Summary: Make catalogs updatable Key: HIVE-19688 URL: https://issues.apache.org/jira/browse/HIVE-19688 Project: Hive Issue Type: Sub-task Components: Metastore Affects Versions: 3.0.0 Reporter: Alan Gates Assignee: Alan Gates The initial changes for catalogs did not include an ability to alter catalogs. We need to add that. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-19686) schematool --createCatalog option fails when using Oracle as the RDBMS
Alan Gates created HIVE-19686:
-
Summary: schematool --createCatalog option fails when using
Oracle as the RDBMS
Key: HIVE-19686
URL: https://issues.apache.org/jira/browse/HIVE-19686
Project: Hive
Issue Type: Bug
Components: Metastore
Affects Versions: 3.0.0
Reporter: Alan Gates
Assignee: Alan Gates
Fix For: 3.0.1
Attempts to use the schematool --createCatalog option when the metastore is
using Oracle result in
{code:java}
SQL Error code: 1786
org.apache.hadoop.hive.metastore.HiveMetaException: Failed to add catalog
at org.apache.hive.beeline.HiveSchemaTool.createCatalog(HiveSchemaTool.java:941)
at org.apache.hive.beeline.HiveSchemaTool.main(HiveSchemaTool.java:1459)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:308)
at org.apache.hadoop.util.RunJar.main(RunJar.java:222)
Caused by: java.sql.SQLSyntaxErrorException: ORA-01786: FOR UPDATE of this
query expression is not allowed
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:450)
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:399)
at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:1059)
at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:522)
at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:257)
at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:587)
at oracle.jdbc.driver.T4CStatement.doOall8(T4CStatement.java:210)
at oracle.jdbc.driver.T4CStatement.doOall8(T4CStatement.java:30)
at oracle.jdbc.driver.T4CStatement.executeForDescribe(T4CStatement.java:762)
at
oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:925)
at
oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:)
at oracle.jdbc.driver.OracleStatement.executeQuery(OracleStatement.java:1309)
at
oracle.jdbc.driver.OracleStatementWrapper.executeQuery(OracleStatementWrapper.java:422)
at org.apache.hive.beeline.HiveSchemaTool.createCatalog(HiveSchemaTool.java:926)
... 7 more
*** schemaTool failed ***{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HIVE-19576) IHMSHandler.getTable not always fetching the right catalog
Alan Gates created HIVE-19576:
-
Summary: IHMSHandler.getTable not always fetching the right catalog
Key: HIVE-19576
URL: https://issues.apache.org/jira/browse/HIVE-19576
Project: Hive
Issue Type: Bug
Components: Metastore
Affects Versions: 3.0.0
Reporter: Alan Gates
Assignee: Alan Gates
Fix For: 3.0.1
{{IHMSHandler.get_table_core(String dbName, String tableName)}} fetches the
catalog name from the conf. This causes issues when doing an operation where
the catalog is known and does not match the default provided in the
configuration file (e.g. adding a partition). This method should be removed
and callers forced to use {{IHMSHandler.get_table_core(String catName, String
dbName, String tableName)}} instead since callers will know whether they have
the catalog name or not.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HIVE-19558) HiveAuthorizationProviderBase gets catalog name from config rather than db object
Alan Gates created HIVE-19558: - Summary: HiveAuthorizationProviderBase gets catalog name from config rather than db object Key: HIVE-19558 URL: https://issues.apache.org/jira/browse/HIVE-19558 Project: Hive Issue Type: Bug Components: Authorization Affects Versions: 3.0.0 Reporter: Alan Gates Assignee: Alan Gates Fix For: 3.0.1 HiveAuthorizationProviderBase.getDatabase uses just the database name to fetch the database, relying on getDefaultCatalog() to fetch the catalog name from the conf file. This does not work when the client has passed in an object for a different catalog. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-19531) TransactionalValidationListener is getting catalog name from conf instead of table object.
Alan Gates created HIVE-19531: - Summary: TransactionalValidationListener is getting catalog name from conf instead of table object. Key: HIVE-19531 URL: https://issues.apache.org/jira/browse/HIVE-19531 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 3.0.0 Reporter: Alan Gates Assignee: Alan Gates Fix For: 3.0.0 TransactionalValidationListener.validateTableStructure get the catalog from the conf file rather than taking it from the passed in table structure. This causes createTable operations to fail. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-19365) Index on COMPLETED_TXN_COMPONENTS in Metastore RDBMS has different names in different scripts
Alan Gates created HIVE-19365: - Summary: Index on COMPLETED_TXN_COMPONENTS in Metastore RDBMS has different names in different scripts Key: HIVE-19365 URL: https://issues.apache.org/jira/browse/HIVE-19365 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 3.0.0 Reporter: Alan Gates Assignee: Alan Gates Fix For: 3.0.0 In mysql and mssql install scripts the index is called COMPLETED_TXN_COMPONENTS_IDX2 Everywhere else it is called COMPLETED_TXN_COMPONENTS_IDX, which is breaking upgrade scripts for 3.0 to 3.1 since they don't know which index to update. One name should be chosen and used everywhere. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-19323) Create metastore SQL install and upgrade scripts for 3.1
Alan Gates created HIVE-19323: - Summary: Create metastore SQL install and upgrade scripts for 3.1 Key: HIVE-19323 URL: https://issues.apache.org/jira/browse/HIVE-19323 Project: Hive Issue Type: Task Components: Metastore Affects Versions: 3.1.0 Reporter: Alan Gates Assignee: Alan Gates Now that we've branched for 3.0 we need to create SQL install and upgrade scripts for 3.1 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-19151) Update expected result for some TestNegativeCliDriver tests
Alan Gates created HIVE-19151: - Summary: Update expected result for some TestNegativeCliDriver tests Key: HIVE-19151 URL: https://issues.apache.org/jira/browse/HIVE-19151 Project: Hive Issue Type: Bug Components: Tests Affects Versions: 3.0.0 Reporter: Alan Gates Assignee: Alan Gates Fix For: 3.0.0 Several TestNegativeCliDriver tests are failing because error messages have changed. They need their expected results updated. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-19135) Need tool to allow admins to create catalogs and move existing dbs to catalog during upgrade
Alan Gates created HIVE-19135: - Summary: Need tool to allow admins to create catalogs and move existing dbs to catalog during upgrade Key: HIVE-19135 URL: https://issues.apache.org/jira/browse/HIVE-19135 Project: Hive Issue Type: Sub-task Components: Metastore Affects Versions: 3.0.0 Reporter: Alan Gates Assignee: Alan Gates As part of upgrading to Hive 3 admins may wish to create new catalogs and move some existing databases into those catalogs. We can do this by adding options to schematool. This guarantees that only admins can do these operations. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-19120) catalog not properly set for some tables in SQL upgrade scripts
Alan Gates created HIVE-19120: - Summary: catalog not properly set for some tables in SQL upgrade scripts Key: HIVE-19120 URL: https://issues.apache.org/jira/browse/HIVE-19120 Project: Hive Issue Type: Bug Components: Standalone Metastore Affects Versions: 3.0.0 Reporter: Alan Gates Assignee: Alan Gates Fix For: 3.0.0 A catalog column is added to the PARTITION_EVENTS and NOTIFICATION_LOG but the upgrade scripts do not include an UPDATE statement to set this to the default value. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-19105) HIVE-18781 broke WarehouseInstance
Alan Gates created HIVE-19105:
-
Summary: HIVE-18781 broke WarehouseInstance
Key: HIVE-19105
URL: https://issues.apache.org/jira/browse/HIVE-19105
Project: Hive
Issue Type: Bug
Components: repl, Tests
Affects Versions: 3.0.0
Reporter: Alan Gates
Fix For: 3.0.0
HIVE-18781 removed the line
{{hiveConf.set(HiveConf.ConfVars.HIVE_SUPPORT_CONCURRENCY.varname, "false");}}
from WarehouseInstance. This causes tests such as
TestReplicationOnHDFSEncryptedZones and TestExportImport to hang as they are
trying to connect to a non-existent ZooKeeper instance.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HIVE-19099) HIVE-18755 forgot to update derby install script in metastore
Alan Gates created HIVE-19099: - Summary: HIVE-18755 forgot to update derby install script in metastore Key: HIVE-19099 URL: https://issues.apache.org/jira/browse/HIVE-19099 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 3.0.0 Reporter: Alan Gates Assignee: Alan Gates Fix For: 3.0.0 metastore/srcripts/upgrade/derby/hive-schema-3.0 was not properly updated with the new and changed tables for catalogs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-19031) Mark duplicate configs in HiveConf as deprecated
Alan Gates created HIVE-19031: - Summary: Mark duplicate configs in HiveConf as deprecated Key: HIVE-19031 URL: https://issues.apache.org/jira/browse/HIVE-19031 Project: Hive Issue Type: Sub-task Components: Configuration, Standalone Metastore Affects Versions: 3.0.0 Reporter: Alan Gates Assignee: Alan Gates There are a number of configuration values that were copied from HiveConf to MetastoreConf. They have been left in HiveConf for backwards compatibility. But they need to be marked as deprecated so that users know to use the new values in MetastoreConf. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-18991) Drop database cascade doesn't work with materialized views
Alan Gates created HIVE-18991: - Summary: Drop database cascade doesn't work with materialized views Key: HIVE-18991 URL: https://issues.apache.org/jira/browse/HIVE-18991 Project: Hive Issue Type: Bug Components: Materialized views, Metastore Affects Versions: 3.0.0 Reporter: Alan Gates Create a database, add a table and then a materialized view that depends on the table. Then drop the database with cascade set. Sometimes this will fail because when HiveMetaStore.drop_database_core goes to drop all of the tables it may drop the base table before the materialized view, which will cause an integrity constraint violation in the RDBMS. To resolve this that method should change to fetch and drop materialized views before tables. cc [~jcamachorodriguez] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-18977) Listing partitions returns different results with JDO and direct SQL
Alan Gates created HIVE-18977: - Summary: Listing partitions returns different results with JDO and direct SQL Key: HIVE-18977 URL: https://issues.apache.org/jira/browse/HIVE-18977 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 3.0.0 Reporter: Alan Gates Some of the tests in TestListPartitions fail when using JDO instead of direct SQL. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-18973) Make transaction system work with catalogs
Alan Gates created HIVE-18973: - Summary: Make transaction system work with catalogs Key: HIVE-18973 URL: https://issues.apache.org/jira/browse/HIVE-18973 Project: Hive Issue Type: Sub-task Components: Metastore Affects Versions: 3.0.0 Reporter: Alan Gates Assignee: Alan Gates The transaction tables need to understand catalogs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-18967) Standalone metastore SQL upgrade scripts do not properly set schema version
Alan Gates created HIVE-18967: - Summary: Standalone metastore SQL upgrade scripts do not properly set schema version Key: HIVE-18967 URL: https://issues.apache.org/jira/browse/HIVE-18967 Project: Hive Issue Type: Bug Components: Standalone Metastore Affects Versions: 3.0.0 Reporter: Alan Gates Assignee: Alan Gates Fix For: 3.0.0 The new combined upgrade scripts for Hive 2.3 to 3.0 transition do not properly set the schema version after they have completed the upgrade. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-18965) HIVE-17990 didn't update derby SQL scripts
Alan Gates created HIVE-18965: - Summary: HIVE-17990 didn't update derby SQL scripts Key: HIVE-18965 URL: https://issues.apache.org/jira/browse/HIVE-18965 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 3.0.0 Reporter: Alan Gates Assignee: Alan Gates Fix For: 3.0.0 Owing to HIVE-17983, there are currently two copies of the derby installation and upgrade script. HIVE-17990 only updated the one in standalone-metastore. The one in metastore needs to be updated as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-18960) Make Materialized view invalidation cache work with catalogs
Alan Gates created HIVE-18960: - Summary: Make Materialized view invalidation cache work with catalogs Key: HIVE-18960 URL: https://issues.apache.org/jira/browse/HIVE-18960 Project: Hive Issue Type: Sub-task Components: Materialized views, Metastore Affects Versions: 3.0.0 Reporter: Alan Gates Assignee: Alan Gates MaterializationsInvalidationCache needs to be made catalog aware. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-18776) MaterializationsInvalidationCache loading causes race condition in the metastore
Alan Gates created HIVE-18776: - Summary: MaterializationsInvalidationCache loading causes race condition in the metastore Key: HIVE-18776 URL: https://issues.apache.org/jira/browse/HIVE-18776 Project: Hive Issue Type: Bug Components: Materialized views, Metastore Affects Versions: 3.0.0 Reporter: Alan Gates Assignee: Jesus Camacho Rodriguez I am seeing occasional failures running metastore tests where operations are failing saying that there is no open transaction. I have traced this to a race condition in loading the materialized view invalidation cache. When it is initialized (either in HiveMetaStoreClient in embedded mode or in HiveMetaStore in remote mode) it grabs a copy of the current RawStore instance and then loads the cache in a separate thread. But ObjectStore keeps state regarding JDO transactions with the underlying RDBMS. So with the loader thread and the initial thread both doing operations against the RawStore they sometimes mess up each others transaction stack. In a quick test I used HMSHandler.newRawStoreForConf() to fix this, which seemed to work. A reference to the TxnHandler is also called. I suspect this will run into a similar issue. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-18755) Modifications to the metastore for catalogs
Alan Gates created HIVE-18755: - Summary: Modifications to the metastore for catalogs Key: HIVE-18755 URL: https://issues.apache.org/jira/browse/HIVE-18755 Project: Hive Issue Type: Sub-task Components: Metastore Reporter: Alan Gates Assignee: Alan Gates Step 1 of adding catalogs is to add support in the metastore. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-18686) Installation on Postgres and Oracle broken
Alan Gates created HIVE-18686: - Summary: Installation on Postgres and Oracle broken Key: HIVE-18686 URL: https://issues.apache.org/jira/browse/HIVE-18686 Project: Hive Issue Type: Bug Reporter: Alan Gates Assignee: Alan Gates HIVE-18614 broke the installation and upgrade on Postgres and Oracle. It calls Connection.setSchema in the JDBC driver. But the JDBC drivers for these databases don't support that call. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-18685) Add catalogs to metastore
Alan Gates created HIVE-18685: - Summary: Add catalogs to metastore Key: HIVE-18685 URL: https://issues.apache.org/jira/browse/HIVE-18685 Project: Hive Issue Type: New Feature Components: Metastore Affects Versions: 3.0.0 Reporter: Alan Gates Assignee: Alan Gates Attachments: HMS Catalog Design Doc.pdf SQL supports two levels of namespaces, called in the spec catalogs and schemas (with schema being equivalent to Hive's database). I propose to add the upper level of catalog. The attached design doc covers the use cases, requirements, and brief discussion of how it will be implemented in a backwards compatible way. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-18588) Add 'checkin' profile that runs slower tests in standalone-metastore
Alan Gates created HIVE-18588: - Summary: Add 'checkin' profile that runs slower tests in standalone-metastore Key: HIVE-18588 URL: https://issues.apache.org/jira/browse/HIVE-18588 Project: Hive Issue Type: Test Components: Standalone Metastore Affects Versions: 3.0.0 Reporter: Alan Gates Runtime for unit tests in standalone-metastore are now exceeding 25 minutes. Ideally unit tests should finish within 2-3 minutes so users will run them frequently. To solve this I propose to carve off many of the slower tests to run in a new 'checkin' profile. This profile should be run before checkin and by the ptest infrastructure. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-18297) Add builder for metastore Thrift classes missed in the first pass
Alan Gates created HIVE-18297: - Summary: Add builder for metastore Thrift classes missed in the first pass Key: HIVE-18297 URL: https://issues.apache.org/jira/browse/HIVE-18297 Project: Hive Issue Type: Task Components: Standalone Metastore Affects Versions: 3.0.0 Reporter: Alan Gates Assignee: Alan Gates The first pass of adding builders for the metastore Thrift classes missed Function, statistics, and WM* objects. Builders for these should be added. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-17990) Add Thrift and DB storage for Schema Registry objects
Alan Gates created HIVE-17990: - Summary: Add Thrift and DB storage for Schema Registry objects Key: HIVE-17990 URL: https://issues.apache.org/jira/browse/HIVE-17990 Project: Hive Issue Type: Sub-task Components: Standalone Metastore Reporter: Alan Gates Assignee: Alan Gates This JIRA tracks changes to Thrift, RawStore, and DB scripts to support objects in the Schema Registry. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-17989) Adding support for Schema Registry objects to standalone
Alan Gates created HIVE-17989: - Summary: Adding support for Schema Registry objects to standalone Key: HIVE-17989 URL: https://issues.apache.org/jira/browse/HIVE-17989 Project: Hive Issue Type: New Feature Components: Standalone Metastore Reporter: Alan Gates Assignee: Alan Gates The team building the [Hortonworks Schema Registry|https://github.com/hortonworks/registry/] has expressed interest in integrating it into the standalone metastore. This will allow users to have one data catalog for table based and stream based data. This JIRA will serve as an umbrella JIRA for tracking related issues. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-17983) Make the standalone metastore generate tarballs etc.
Alan Gates created HIVE-17983: - Summary: Make the standalone metastore generate tarballs etc. Key: HIVE-17983 URL: https://issues.apache.org/jira/browse/HIVE-17983 Project: Hive Issue Type: Sub-task Components: Standalone Metastore Reporter: Alan Gates Assignee: Alan Gates Priority: Major In order to be separately installable the standalone metastore needs its own tarballs, startup scripts, etc. All of the SQL installation and upgrade scripts also need to move from metastore to standalone-metastore. I also plan to create Dockerfiles for different database types so that developers can test the SQL installation and upgrade scripts. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-17982) Move metastore specific itests
Alan Gates created HIVE-17982: - Summary: Move metastore specific itests Key: HIVE-17982 URL: https://issues.apache.org/jira/browse/HIVE-17982 Project: Hive Issue Type: Sub-task Components: Standalone Metastore Reporter: Alan Gates Assignee: Alan Gates Priority: Major There are a number of tests in itests/hive-unit/.../metastore that are metastore specific. I suspect they were initially placed in itests only because the metastore pulling in a few plugins from ql. Given that we need to be able to release the metastore separately, we need to be able to test it completely as a standalone entity. So I propose to move a number of the itests over into standalone-metastore. I will only move tests that are isolated to the metastore. Anything that tests wider functionality I plan to leave in itests. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-17981) Create a set of builders for Thrift classes
Alan Gates created HIVE-17981:
-
Summary: Create a set of builders for Thrift classes
Key: HIVE-17981
URL: https://issues.apache.org/jira/browse/HIVE-17981
Project: Hive
Issue Type: Sub-task
Components: Standalone Metastore
Reporter: Alan Gates
Assignee: Alan Gates
Priority: Major
Instantiating some of the Thrift classes is painful. Consider building a
{{Table}} object, which requires a {{StorageDescriptor}}, which requires a
{{SerDeInfo}} and a list of {{FieldInfo}}. All that is really necessary for a
Table in the most simple case is a name, a database, and some columns. But
currently creating even a simple Table requires 20+ lines of code. This is
particularly painful in tests.
I propose to add a set of builders. These will come with reasonable defaults
to minimize the boilerplate code. They will also include simple methods for
common operations (like adding columns, or a parameter) without requiring the
user to create all the sub-objects (like {{StorageDescriptor}}).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HIVE-17980) Move HiveMetaStoreClient plus a few remaining classes.
Alan Gates created HIVE-17980: - Summary: Move HiveMetaStoreClient plus a few remaining classes. Key: HIVE-17980 URL: https://issues.apache.org/jira/browse/HIVE-17980 Project: Hive Issue Type: Sub-task Components: Standalone Metastore Reporter: Alan Gates Assignee: Alan Gates Priority: Major This moves over HiveMetaStoreClient a few more remaining things that haven't been moved yet. It also cleans up the mess I made of MetaStoreUtils. I renamed the class in metastore to HiveMetaStoreUtils and left a few methods in there that could be moved. Everything else has been moved to standalone-metastore/.../MetaStoreUtils. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-17967) Move HiveMetaStore class
Alan Gates created HIVE-17967: - Summary: Move HiveMetaStore class Key: HIVE-17967 URL: https://issues.apache.org/jira/browse/HIVE-17967 Project: Hive Issue Type: Sub-task Components: Standalone Metastore Reporter: Alan Gates Assignee: Alan Gates Priority: Major We need to move HiveMetaStore and a few tightly integrated classes into the standalone metastore. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-17812) Move remaining classes that HiveMetaStore depends on
Alan Gates created HIVE-17812: - Summary: Move remaining classes that HiveMetaStore depends on Key: HIVE-17812 URL: https://issues.apache.org/jira/browse/HIVE-17812 Project: Hive Issue Type: Sub-task Components: Metastore Reporter: Alan Gates Assignee: Alan Gates There are several remaining pieces that need moved before we can move HiveMetaStore itself. These include NotificationListener and implementations, Events, AlterHandler, and a few other miscellaneous pieces. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-17788) MetastoreConf not properly handling variable interpolation in config values
Alan Gates created HIVE-17788:
-
Summary: MetastoreConf not properly handling variable
interpolation in config values
Key: HIVE-17788
URL: https://issues.apache.org/jira/browse/HIVE-17788
Project: Hive
Issue Type: Bug
Components: Metastore
Affects Versions: 3.0.0
Reporter: Alan Gates
Assignee: Alan Gates
HiveConf allows the user to put System properties into values in the conf file,
which will later be interpolated at read time. For example:
{code}
hadoop.tmp.dir
${test.tmp.dir}/hadoop-tmp
A base for other temporary directories.
{code}
The value for test.tmp.dir is read from the System properties at runtime.
MetastoreConf is instead returning the uninterpolated value.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HIVE-17733) Move RawStore to standalone metastore
Alan Gates created HIVE-17733: - Summary: Move RawStore to standalone metastore Key: HIVE-17733 URL: https://issues.apache.org/jira/browse/HIVE-17733 Project: Hive Issue Type: Sub-task Components: Metastore Reporter: Alan Gates Assignee: Alan Gates This includes moving implementations of RawStore (like ObjectStore), MetastoreDirectSql, and stats related classes like ColumnStatsAggregator and the NDV classes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-17561) Move TxnStore and implementations to standalone metastore
Alan Gates created HIVE-17561: - Summary: Move TxnStore and implementations to standalone metastore Key: HIVE-17561 URL: https://issues.apache.org/jira/browse/HIVE-17561 Project: Hive Issue Type: Sub-task Components: Metastore, Transactions Reporter: Alan Gates Assignee: Alan Gates We need to move the metastore handling of transactions into the standalone metastore. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-17537) Move Warehouse class to standalone metastore
Alan Gates created HIVE-17537: - Summary: Move Warehouse class to standalone metastore Key: HIVE-17537 URL: https://issues.apache.org/jira/browse/HIVE-17537 Project: Hive Issue Type: Sub-task Reporter: Alan Gates Assignee: Alan Gates Move the Warehouse class. This is done in its own JIRA as it was somewhat more involved than some of the other classes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-17506) Fix standalone-metastore pom.xml to not depend on hive's main pom
Alan Gates created HIVE-17506: - Summary: Fix standalone-metastore pom.xml to not depend on hive's main pom Key: HIVE-17506 URL: https://issues.apache.org/jira/browse/HIVE-17506 Project: Hive Issue Type: Sub-task Components: Metastore Reporter: Alan Gates Assignee: Alan Gates In order to be separately releasable the standalone metastore needs to have its own pom rather than inherit from Hive's. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-17488) Move first set of classes to standalone metastore
Alan Gates created HIVE-17488: - Summary: Move first set of classes to standalone metastore Key: HIVE-17488 URL: https://issues.apache.org/jira/browse/HIVE-17488 Project: Hive Issue Type: Sub-task Components: Metastore Reporter: Alan Gates Assignee: Alan Gates There are a whole set of classes that can be moved with few changes other than the config file, shims, etc. This task will move those classes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-17425) Change MetastoreConf.ConfVars internal members to be private
Alan Gates created HIVE-17425: - Summary: Change MetastoreConf.ConfVars internal members to be private Key: HIVE-17425 URL: https://issues.apache.org/jira/browse/HIVE-17425 Project: Hive Issue Type: Task Components: Metastore Affects Versions: 3.0.0 Reporter: Alan Gates Assignee: Alan Gates MetastoreConf's dual use of metastore keys and Hive keys is causing confusion for developers. We should make the relevant members private and provide getter methods with comments on when it is appropriate to use them. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-17359) Deal with TypeInfo dependencies in the metastore
Alan Gates created HIVE-17359: - Summary: Deal with TypeInfo dependencies in the metastore Key: HIVE-17359 URL: https://issues.apache.org/jira/browse/HIVE-17359 Project: Hive Issue Type: Sub-task Components: Metastore Affects Versions: 3.0.0 Reporter: Alan Gates Assignee: Alan Gates The metastore uses TypeInfo, which resides in the serdes package. In order to move the metastore to be separately releasable we need to deal with this. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-17325) Clean up intermittently failing uni tests
Alan Gates created HIVE-17325: - Summary: Clean up intermittently failing uni tests Key: HIVE-17325 URL: https://issues.apache.org/jira/browse/HIVE-17325 Project: Hive Issue Type: Test Components: Tests Reporter: Alan Gates Assignee: Alan Gates We have a number of intermittently failing tests. I propose to disable these so that we can get clean (or at least cleaner) CI runs. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-17307) Change the metastore to not use the metrics code in hive/common
Alan Gates created HIVE-17307: - Summary: Change the metastore to not use the metrics code in hive/common Key: HIVE-17307 URL: https://issues.apache.org/jira/browse/HIVE-17307 Project: Hive Issue Type: Sub-task Components: Metastore Reporter: Alan Gates Assignee: Alan Gates As we move code into the standalone metastore module, it cannot use the metrics in hive-common. We could copy the current Metrics interface or we could change the metastore code to directly use codahale metrics. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-17241) Change metastore classes to not use the shims
Alan Gates created HIVE-17241: - Summary: Change metastore classes to not use the shims Key: HIVE-17241 URL: https://issues.apache.org/jira/browse/HIVE-17241 Project: Hive Issue Type: Sub-task Components: Metastore Reporter: Alan Gates Assignee: Alan Gates As part of moving the metastore into a standalone package, it will no longer have access to the shims. This means we need to either copy them or access the underlying Hadoop operations directly. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-17234) Remove HBase metastore from master
Alan Gates created HIVE-17234: - Summary: Remove HBase metastore from master Key: HIVE-17234 URL: https://issues.apache.org/jira/browse/HIVE-17234 Project: Hive Issue Type: Task Components: HBase Metastore Affects Versions: 3.0.0 Reporter: Alan Gates Assignee: Alan Gates No new development has been done on the HBase metastore in at least a year, and to my knowledge no one is using it (nor is it even in a state to be fully usable). Given the lack of interest in continuing to develop it, we should remove it rather than leave dead code hanging around and extra tests taking up time in test runs. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-17224) Move JDO classes to standalone metastore
Alan Gates created HIVE-17224: - Summary: Move JDO classes to standalone metastore Key: HIVE-17224 URL: https://issues.apache.org/jira/browse/HIVE-17224 Project: Hive Issue Type: Sub-task Components: Metastore Reporter: Alan Gates Assignee: Alan Gates The JDO model classes (MDatabase, MTable, etc.) and the package.jdo file that defines the DB mapping need to be moved to the standalone metastore. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-17170) Move thrift generated code to stand alone metastore
Alan Gates created HIVE-17170: - Summary: Move thrift generated code to stand alone metastore Key: HIVE-17170 URL: https://issues.apache.org/jira/browse/HIVE-17170 Project: Hive Issue Type: Sub-task Components: Metastore Reporter: Alan Gates Assignee: Alan Gates hive_metastore.thrift and the code it generates needs to be moved into the standalone metastore module. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-17168) Create separate module for stand alone metastore
Alan Gates created HIVE-17168: - Summary: Create separate module for stand alone metastore Key: HIVE-17168 URL: https://issues.apache.org/jira/browse/HIVE-17168 Project: Hive Issue Type: Sub-task Components: Metastore Reporter: Alan Gates Assignee: Alan Gates We need to create a separate maven module for the stand alone metastore. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-17167) Create metastore specific configuration tool
Alan Gates created HIVE-17167: - Summary: Create metastore specific configuration tool Key: HIVE-17167 URL: https://issues.apache.org/jira/browse/HIVE-17167 Project: Hive Issue Type: Sub-task Components: Metastore Reporter: Alan Gates Assignee: Alan Gates As part of making the metastore a separately releasable module we need configuration tools that are specific to that module. It cannot use or extend HiveConf as that is in hive common. But it must take a HiveConf object and be able to operate on it. The best way to achieve this is using Hadoop's Configuration object (which HiveConf extends) together with enums and static methods. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-17159) Make metastore a separately releasable module
Alan Gates created HIVE-17159: - Summary: Make metastore a separately releasable module Key: HIVE-17159 URL: https://issues.apache.org/jira/browse/HIVE-17159 Project: Hive Issue Type: New Feature Components: Metastore Reporter: Alan Gates Assignee: Alan Gates As proposed in this [thread|https://lists.apache.org/thread.html/5e75f45d60f0b819510814a126cfd3809dd24b1c7035a1c8c41b0c5c@%3Cdev.hive.apache.org%3E] on the dev list, we should move the metastore into a separately releasable module. This is a POC of and potential first step towards separating out the metastore as a separate Apache TLP. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HIVE-16535) Hive fails to build from source code tarball
Alan Gates created HIVE-16535:
-
Summary: Hive fails to build from source code tarball
Key: HIVE-16535
URL: https://issues.apache.org/jira/browse/HIVE-16535
Project: Hive
Issue Type: Bug
Affects Versions: 2.3.0
Reporter: Alan Gates
Priority: Blocker
To reproduce:
# mvn package -Pdist -DskipTests
# cp packaging/target/apache-hive-2.3.0-SNAPSHOT-src.tar.gz /tmp
# cd /tmp
# tar zxf apache-hive-2.3.0-SNAPSHOT-src.tar.gz
# cd apache-hive-2.3.0-SNAPSHOT-src
# mvn install
You will get
{code}
[INFO] Scanning for projects...
[ERROR] [ERROR] Some problems were encountered while processing the POMs:
[ERROR] Child module
/Users/gates/tmp/hive-check/apache-hive-2.3.0-SNAPSHOT-src/vector-code-gen of
/Users/gates/tmp/hive-check/apache-hive-2.3.0-SNAPSHOT-src/pom.xml does not
exist @
@
[ERROR] The build could not read 1 project -> [Help 1]
[ERROR]
[ERROR] The project org.apache.hive:hive:2.3.0-SNAPSHOT
(/Users/gates/tmp/hive-check/apache-hive-2.3.0-SNAPSHOT-src/pom.xml) has 1 error
[ERROR] Child module
/Users/gates/tmp/hive-check/apache-hive-2.3.0-SNAPSHOT-src/vector-code-gen of
/Users/gates/tmp/hive-check/apache-hive-2.3.0-SNAPSHOT-src/pom.xml does not
exist
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please
read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException
{code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (HIVE-16504) Addition of binary licenses broke rat check
Alan Gates created HIVE-16504: - Summary: Addition of binary licenses broke rat check Key: HIVE-16504 URL: https://issues.apache.org/jira/browse/HIVE-16504 Project: Hive Issue Type: Bug Affects Versions: 2.2.0, 2.3.0 Reporter: Alan Gates Assignee: Alan Gates Priority: Blocker The clean up of Hive's licenses (HIVE-15035) broke the rat check, as all the license files get reported as invalid licenses. The rat check needs to be modified to ignore those files. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HIVE-16463) Change license for java transaction jta jar to CDDL 1.0
Alan Gates created HIVE-16463: - Summary: Change license for java transaction jta jar to CDDL 1.0 Key: HIVE-16463 URL: https://issues.apache.org/jira/browse/HIVE-16463 Project: Hive Issue Type: Bug Reporter: Alan Gates Assignee: Alan Gates Previously I erroneously said that this jar was under the SCSL 3.0 license. But further research has shown I was wrong and it is released under CDDL 1.0. So we need to change the license file for this jar in the binaries directory. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HIVE-15413) Primary key constraints forced to be unique across database and table names
Alan Gates created HIVE-15413: - Summary: Primary key constraints forced to be unique across database and table names Key: HIVE-15413 URL: https://issues.apache.org/jira/browse/HIVE-15413 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 2.2.0 Reporter: Alan Gates Priority: Critical In the RDBMS underlying the metastore the table that stores primary and foreign keys has it's own primary key (at the RDBMS level) of (constraint_name, position). This means that a constraint name must be unique across all tables and databases in a system. This is not reasonable. Database and table name should be included in the RDBMS primary key. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-15412) Partition columns should be allowed in primary keys
Alan Gates created HIVE-15412: - Summary: Partition columns should be allowed in primary keys Key: HIVE-15412 URL: https://issues.apache.org/jira/browse/HIVE-15412 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 2.1.1 Reporter: Alan Gates If a primary key includes a partition column the attempt to create the primary key returns an error saying that the partition column cannot be found. It appears that the logic in the ObjectStore is checking against the list of valid columns in the storage descriptor, which doesn't include the partition columns. Yet it is perfectly reasonable for a user to want to include a partition column in a primary key. And we want to encourage this behavior because it will make query optimization easier. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-15401) Import constraints into HBase metastore
Alan Gates created HIVE-15401: - Summary: Import constraints into HBase metastore Key: HIVE-15401 URL: https://issues.apache.org/jira/browse/HIVE-15401 Project: Hive Issue Type: Sub-task Components: HBase Metastore Affects Versions: 2.1.1 Reporter: Alan Gates Assignee: Alan Gates Since HIVE-15342 added support for primary and foreign keys in the HBase metastore we should support them in HBaseImport as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-15342) Add support for primary/foreign keys in HBase metastore
Alan Gates created HIVE-15342: - Summary: Add support for primary/foreign keys in HBase metastore Key: HIVE-15342 URL: https://issues.apache.org/jira/browse/HIVE-15342 Project: Hive Issue Type: Improvement Components: HBase Metastore Reporter: Alan Gates Assignee: Alan Gates Fix For: 2.2.0 When HIVE-13076 was committed the calls into the HBase metastore were stubbed out. We need to implement support for constraints in the HBase metastore. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-15037) Figure out whether the SCSL 3.0 license is acceptable
Alan Gates created HIVE-15037: - Summary: Figure out whether the SCSL 3.0 license is acceptable Key: HIVE-15037 URL: https://issues.apache.org/jira/browse/HIVE-15037 Project: Hive Issue Type: Bug Components: distribution Affects Versions: 2.2.0 Reporter: Alan Gates Assignee: Alan Gates Priority: Blocker The jar jta included in Hive's binary distribution is licensed under SCSL 3.0 (http://jcp.org/aboutJava/communityprocess/SCSL3.0.rtf). This isn't listed on the Apache page regarding licenses (https://www.apache.org/legal/resolved.html). We need to figure out if this is acceptable or not. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-15036) Druid code recently included in Hive pulls in GPL jar
Alan Gates created HIVE-15036: - Summary: Druid code recently included in Hive pulls in GPL jar Key: HIVE-15036 URL: https://issues.apache.org/jira/browse/HIVE-15036 Project: Hive Issue Type: Bug Components: Druid integration Affects Versions: 2.2.0 Reporter: Alan Gates Assignee: Gunther Hagleitner Priority: Blocker Druid pulls in a jar annotation-2.3.jar. According to its pom file it is licensed under GPL. We cannot ship a binary distribution that includes this jar. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-15035) Clean up Hive licenses for binary distribution
Alan Gates created HIVE-15035: - Summary: Clean up Hive licenses for binary distribution Key: HIVE-15035 URL: https://issues.apache.org/jira/browse/HIVE-15035 Project: Hive Issue Type: Bug Components: distribution Affects Versions: 2.1.0 Reporter: Alan Gates Assignee: Alan Gates Hive's current LICENSE file contains information not needed for the source distribution. For the binary distribution we are missing many license files as a number of jars included in Hive come with various licenses. This all needs cleaned up. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-13683) Remove erroneously included patch file
Alan Gates created HIVE-13683: - Summary: Remove erroneously included patch file Key: HIVE-13683 URL: https://issues.apache.org/jira/browse/HIVE-13683 Project: Hive Issue Type: Bug Affects Versions: 2.1.0 Reporter: Alan Gates Assignee: Alan Gates Fix For: 2.1.0 The commit for HIVE-13509 accidentally included a patch file in the source tree. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-13681) Update README with latest Hive functionality
Alan Gates created HIVE-13681: - Summary: Update README with latest Hive functionality Key: HIVE-13681 URL: https://issues.apache.org/jira/browse/HIVE-13681 Project: Hive Issue Type: Improvement Components: Documentation Affects Versions: 2.0.0 Reporter: Alan Gates Assignee: Alan Gates In line with the latest changes to the website (see HIVE-13607) and the wiki to remove references to Hive QL/HQL and update use cases Hive is appropriate for we need to update the README file as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-13607) Change website references to HQL/HiveQL to SQL
Alan Gates created HIVE-13607: - Summary: Change website references to HQL/HiveQL to SQL Key: HIVE-13607 URL: https://issues.apache.org/jira/browse/HIVE-13607 Project: Hive Issue Type: Improvement Components: Website Reporter: Alan Gates Assignee: Alan Gates When it started Hive's SQL dialect was far enough from standard SQL that the developers called it HQL or HiveQL. Over the years Hive's SQL dialect has matured. It still has some oddities but it is explicitly pushing towards SQL 2011 conformance. Calling the language anything but SQL now is confusing for users. In addition to changing the website I propose to make changes in the wiki. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-13406) HBase metastore needs to support ACID
Alan Gates created HIVE-13406: - Summary: HBase metastore needs to support ACID Key: HIVE-13406 URL: https://issues.apache.org/jira/browse/HIVE-13406 Project: Hive Issue Type: Improvement Components: HBase Metastore, Transactions Affects Versions: 2.0.0 Reporter: Alan Gates Assignee: Alan Gates The current HBase metastore code does not support any of the acid operations. It needs to. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-12902) Refactor TxnHandler to be an interface
Alan Gates created HIVE-12902: - Summary: Refactor TxnHandler to be an interface Key: HIVE-12902 URL: https://issues.apache.org/jira/browse/HIVE-12902 Project: Hive Issue Type: Improvement Components: Metastore, Transactions Affects Versions: 2.0.0 Reporter: Alan Gates Assignee: Alan Gates TxnHandler (and it's extender CompactionTxnHandler) implement the transaction functionality in the metastore. These need to be re-factored as interfaces so that we can have a separate implementation for the HBase metastore. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-12903) Add ability to run HBase metastore in test mode along with RDBMS
Alan Gates created HIVE-12903: - Summary: Add ability to run HBase metastore in test mode along with RDBMS Key: HIVE-12903 URL: https://issues.apache.org/jira/browse/HIVE-12903 Project: Hive Issue Type: Improvement Components: HBase Metastore, Metastore Affects Versions: 2.0.0 Reporter: Alan Gates Assignee: Daniel Dai -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-12829) Thrift changes for HIVE-12686
Alan Gates created HIVE-12829: - Summary: Thrift changes for HIVE-12686 Key: HIVE-12829 URL: https://issues.apache.org/jira/browse/HIVE-12829 Project: Hive Issue Type: Sub-task Components: Metastore Affects Versions: 1.3.0 Reporter: Alan Gates Assignee: Alan Gates -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-12832) RDBMS schema changes for HIVE-11388
Alan Gates created HIVE-12832: - Summary: RDBMS schema changes for HIVE-11388 Key: HIVE-12832 URL: https://issues.apache.org/jira/browse/HIVE-12832 Project: Hive Issue Type: Sub-task Components: Metastore Affects Versions: 1.0.0 Reporter: Alan Gates Assignee: Alan Gates -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-12830) Thrift changes for HIVE-11793
Alan Gates created HIVE-12830: - Summary: Thrift changes for HIVE-11793 Key: HIVE-12830 URL: https://issues.apache.org/jira/browse/HIVE-12830 Project: Hive Issue Type: Sub-task Components: Metastore Affects Versions: 1.0.0 Reporter: Alan Gates Assignee: Alan Gates Priority: Minor -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-12831) Thrift and RDBMS changes for HIVE-10249
Alan Gates created HIVE-12831: - Summary: Thrift and RDBMS changes for HIVE-10249 Key: HIVE-12831 URL: https://issues.apache.org/jira/browse/HIVE-12831 Project: Hive Issue Type: Sub-task Components: Metastore, Transactions Affects Versions: 1.0.0 Reporter: Alan Gates Assignee: Alan Gates -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-12814) Make thrift and DB changes for HIVE-11444
Alan Gates created HIVE-12814: - Summary: Make thrift and DB changes for HIVE-11444 Key: HIVE-12814 URL: https://issues.apache.org/jira/browse/HIVE-12814 Project: Hive Issue Type: Sub-task Components: Metastore Affects Versions: 1.0.0 Reporter: Alan Gates Assignee: Alan Gates This JIRA tracks the Thrift and DB schema changes for HIVE-11444. It depends on HIVE-12807. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-12816) Thrift and schema changes for HIVE-11685
Alan Gates created HIVE-12816: - Summary: Thrift and schema changes for HIVE-11685 Key: HIVE-12816 URL: https://issues.apache.org/jira/browse/HIVE-12816 Project: Hive Issue Type: Sub-task Components: Metastore Affects Versions: 1.0.1 Reporter: Alan Gates Assignee: Alan Gates This JIRA depends on HIVE-12814. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-12821) Thrift and RDBMS schema changes for HIVE-11965
Alan Gates created HIVE-12821: - Summary: Thrift and RDBMS schema changes for HIVE-11965 Key: HIVE-12821 URL: https://issues.apache.org/jira/browse/HIVE-12821 Project: Hive Issue Type: Sub-task Components: Metastore Affects Versions: 0.14.0 Reporter: Alan Gates Assignee: Alan Gates -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-12822) Thrift and RDBMS schema changes for HIVE-11495
Alan Gates created HIVE-12822: - Summary: Thrift and RDBMS schema changes for HIVE-11495 Key: HIVE-12822 URL: https://issues.apache.org/jira/browse/HIVE-12822 Project: Hive Issue Type: Sub-task Components: Metastore Affects Versions: 1.0.0 Reporter: Alan Gates Assignee: Alan Gates -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-12823) Thrift and RDBMS schema changes for HIVE-11956
Alan Gates created HIVE-12823: - Summary: Thrift and RDBMS schema changes for HIVE-11956 Key: HIVE-12823 URL: https://issues.apache.org/jira/browse/HIVE-12823 Project: Hive Issue Type: Sub-task Components: Metastore Affects Versions: 0.14.0 Reporter: Alan Gates Assignee: Alan Gates -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-12818) Schema changes for HIVE-12353
Alan Gates created HIVE-12818: - Summary: Schema changes for HIVE-12353 Key: HIVE-12818 URL: https://issues.apache.org/jira/browse/HIVE-12818 Project: Hive Issue Type: Sub-task Components: Metastore Affects Versions: 1.0.0 Reporter: Alan Gates Assignee: Alan Gates This JIRA just covers RDBMS schema changes for HIVE-12353. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-12819) Thrift and RDBMS schema changes for HIVE-11957
Alan Gates created HIVE-12819: - Summary: Thrift and RDBMS schema changes for HIVE-11957 Key: HIVE-12819 URL: https://issues.apache.org/jira/browse/HIVE-12819 Project: Hive Issue Type: Sub-task Components: Metastore Affects Versions: 1.0.0 Reporter: Alan Gates Assignee: Alan Gates -- This message was sent by Atlassian JIRA (v6.3.4#6332)
