[jira] [Created] (HIVE-22442) Datanucleus cannot map MTableColumnStatistics.bitVector when writing to HiveMetastore

2019-10-31 Thread Rentao Wu (Jira) 
Rentao Wu created HIVE-22442:


 Summary: Datanucleus cannot map MTableColumnStatistics.bitVector 
when writing to HiveMetastore 
 Key: HIVE-22442
 URL: https://issues.apache.org/jira/browse/HIVE-22442
 Project: Hive
  Issue Type: Bug
  Components: Database/Schema, Metastore, SQL, Standalone Metastore
Affects Versions: 3.1.2
Reporter: Rentao Wu


I'm seeing on insert statements, in StatsTasks as part of persisting 
MTableColumnStatistics into the metastore, the new column bitVector (defined to 
map to BLOB sql type) is failing to be mapped, causing my insert statements to 
fail. Any ideas on how to fix this issue?

 

I'm using:

Hive 3.1.2

Hadoop 3.2.1

Hive Metastore: Mariadb 5.5.64

Datanucleus: (default versions defined in pom.xml)

/usr/lib/hive/lib/datanucleus-api-jdo-4.2.4.jar
/usr/lib/hive/lib/datanucleus-core-4.1.17.jar
/usr/lib/hive/lib/datanucleus-rdbms-4.1.19.jar

 

2019-10-29T19:01:11,799 ERROR [aba24ff8-5560-411d-a1ee-141871cd4b4b main([])]: 
exec.StatsTask (:()) - Failed to run stats task
org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:Failed 
to generate new Mapping of type 
org.datanucleus.store.rdbms.mapping.java.ArrayMapping, exception : JDBC type 
BLOB declared for field 
"org.apache.hadoop.hive.metastore.model.MTableColumnStatistics.bitVector" of 
java type java.io.Serializable cant be mapped for this datastore.
JDBC type BLOB declared for field 
"org.apache.hadoop.hive.metastore.model.MTableColumnStatistics.bitVector" of 
java type java.io.Serializable cant be mapped for this datastore.
org.datanucleus.exceptions.NucleusException: JDBC type BLOB declared for field 
"org.apache.hadoop.hive.metastore.model.MTableColumnStatistics.bitVector" of 
java type java.io.Serializable cant be mapped for this datastore.
 at 
org.datanucleus.store.rdbms.mapping.RDBMSMappingManager.getDatastoreMappingClass(RDBMSMappingManager.java:1386)
 at 
org.datanucleus.store.rdbms.mapping.RDBMSMappingManager.createDatastoreMapping(RDBMSMappingManager.java:1616)
 at 
org.datanucleus.store.rdbms.mapping.java.SingleFieldMapping.prepareDatastoreMapping(SingleFieldMapping.java:59)
 at 
org.datanucleus.store.rdbms.mapping.java.AbstractContainerMapping.prepareDatastoreMapping(AbstractContainerMapping.java:99)
 at 
org.datanucleus.store.rdbms.mapping.java.SingleFieldMapping.initialize(SingleFieldMapping.java:48)
 at 
org.datanucleus.store.rdbms.mapping.java.AbstractContainerMapping.initialize(AbstractContainerMapping.java:67)
 at 
org.datanucleus.store.rdbms.mapping.RDBMSMappingManager.getMapping(RDBMSMappingManager.java:482)
 at 
org.datanucleus.store.rdbms.table.ClassTable.manageMembers(ClassTable.java:536)
 at 
org.datanucleus.store.rdbms.table.ClassTable.manageClass(ClassTable.java:442)
 at 
org.datanucleus.store.rdbms.table.ClassTable.initializeForClass(ClassTable.java:1270)
 at org.datanucleus.store.rdbms.table.ClassTable.initialize(ClassTable.java:276)
 at 
org.datanucleus.store.rdbms.RDBMSStoreManager$ClassAdder.initializeClassTables(RDBMSStoreManager.java:3279)
 at 
org.datanucleus.store.rdbms.RDBMSStoreManager$ClassAdder.run(RDBMSStoreManager.java:2889)
 at 
org.datanucleus.store.rdbms.AbstractSchemaTransaction.execute(AbstractSchemaTransaction.java:119)
 at 
org.datanucleus.store.rdbms.RDBMSStoreManager.manageClasses(RDBMSStoreManager.java:1627)
 at 
org.datanucleus.store.rdbms.RDBMSStoreManager.getDatastoreClass(RDBMSStoreManager.java:672)
 at 
org.datanucleus.store.rdbms.RDBMSStoreManager.getPropertiesForGenerator(RDBMSStoreManager.java:2088)
 at 
org.datanucleus.store.AbstractStoreManager.getStrategyValue(AbstractStoreManager.java:1271)
 at 
org.datanucleus.ExecutionContextImpl.newObjectId(ExecutionContextImpl.java:3760)
 at 
org.datanucleus.state.StateManagerImpl.setIdentity(StateManagerImpl.java:2267)
 at 
org.datanucleus.state.StateManagerImpl.initialiseForPersistentNew(StateManagerImpl.java:484)
 at 
org.datanucleus.state.StateManagerImpl.initialiseForPersistentNew(StateManagerImpl.java:120)
 at 
org.datanucleus.state.ObjectProviderFactoryImpl.newForPersistentNew(ObjectProviderFactoryImpl.java:218)
 at 
org.datanucleus.ExecutionContextImpl.persistObjectInternal(ExecutionContextImpl.java:2079)
 at 
org.datanucleus.ExecutionContextImpl.persistObjectWork(ExecutionContextImpl.java:1923)
 at 
org.datanucleus.ExecutionContextImpl.persistObject(ExecutionContextImpl.java:1778)
 at 
org.datanucleus.ExecutionContextThreadedImpl.persistObject(ExecutionContextThreadedImpl.java:217)
 at 
org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:724)
 at 
org.datanucleus.api.jdo.JDOPersistenceManager.makePersistent(JDOPersistenceManager.java:749)
 at 
org.apache.hadoop.hive.metastore.ObjectStore.writeMTableColumnStatistics(ObjectStore.java:8152)
 at

[jira] [Created] (HIVE-19638) Configuration not passed to ORC Reader.Options

2018-05-21 Thread Rentao Wu (JIRA) 
Rentao Wu created HIVE-19638:


 Summary: Configuration not passed to ORC Reader.Options
 Key: HIVE-19638
 URL: https://issues.apache.org/jira/browse/HIVE-19638
 Project: Hive
  Issue Type: Bug
  Components: Configuration, File Formats, ORC
Affects Versions: 2.3.2, 2.3.1, 2.3.0
Reporter: Rentao Wu
Assignee: Rentao Wu


Configuration is not passed to ORC's Reader.Option in OrcFileInputFormat which 
causes some [ORC 
configurations|https://github.com/apache/orc/blob/master/java/core/src/java/org/apache/orc/Reader.java#L170-L176]
 to not be able to be picked up.

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (HIVE-17729) Add Database & Explain related blobstore tests

2017-10-06 Thread Rentao Wu (JIRA) 
Rentao Wu created HIVE-17729:


 Summary: Add Database & Explain related blobstore tests
 Key: HIVE-17729
 URL: https://issues.apache.org/jira/browse/HIVE-17729
 Project: Hive
  Issue Type: Test
  Components: Tests
Reporter: Rentao Wu
Assignee: Rentao Wu


This patch introduces the following regression tests into the hive-blobstore 
qtest module:
* create_database.q  -> tests tables with location inherited from database
* multiple_db.q  -> tests query spanning multiple databases
* explain.q -> tests EXPLAIN INSERT OVERWRITE command
 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HIVE-16664) Add join related Hive blobstore tests

2017-05-12 Thread Rentao Wu (JIRA) 
Rentao Wu created HIVE-16664:


 Summary: Add join related Hive blobstore tests
 Key: HIVE-16664
 URL: https://issues.apache.org/jira/browse/HIVE-16664
 Project: Hive
  Issue Type: Test
  Components: Tests
Affects Versions: 2.1.1
Reporter: Rentao Wu
 Fix For: 2.2.0, 2.3.0, 3.0.0, 2.4.0


This patch introduces the following regression tests into the hive-blobstore 
qtest module:
· join.q -> Test inner join query
· join2.q   -> Test left and right outer 
join
· map_join.q-> Test mapjoin hint
· map_join_on_filter.q -> Test mapjoin hint with filter
· nested_outer_join.q -> Test nested outer join queries



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HIVE-16454) Add blobstore tests for inserting empty into dynamic partition/list bucket tables & inserting cross blobstore tables

2017-04-14 Thread Rentao Wu (JIRA) 
Rentao Wu created HIVE-16454:


 Summary: Add blobstore tests for inserting empty into dynamic 
partition/list bucket tables & inserting cross blobstore tables
 Key: HIVE-16454
 URL: https://issues.apache.org/jira/browse/HIVE-16454
 Project: Hive
  Issue Type: Test
  Components: Tests
Affects Versions: 2.1.1
Reporter: Rentao Wu
Assignee: Rentao Wu
 Fix For: 2.3.0, 3.0.0


This patch introduces two regression tests into the hive-blobstore qtest 
module: insert_empty_into_blobstore.q and insert_blobstore_to_blobstore.q. 
These tests the following cases:
1.   Insert empty data into dynamic partitioned and list bucketed tables.
2.   Insert data from a blobstore table to another blobstore table.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)