[jira] [Commented] (HIVE-3521) Concurrent metastore calls provoke Datanucleus IllegalStateException: Table object has not been been initialised
[
https://issues.apache.org/jira/browse/HIVE-3521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13688853#comment-13688853
]
Brock Noland commented on HIVE-3521:
Nice work Jakub! I am not able to reproduce once placing:
{noformat}
property
namedatanucleus.autoStartMechanism/name
valueSchemaTable/value
/property
{noformat}
in my hive-site.xml
Concurrent metastore calls provoke Datanucleus IllegalStateException: Table
object has not been been initialised
--
Key: HIVE-3521
URL: https://issues.apache.org/jira/browse/HIVE-3521
Project: Hive
Issue Type: Bug
Components: Metastore
Affects Versions: 0.9.0
Reporter: Carl Steinbach
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3521) Concurrent metastore calls provoke Datanucleus IllegalStateException: Table object has not been been initialised
[ https://issues.apache.org/jira/browse/HIVE-3521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13666256#comment-13666256 ] Jakub Holy commented on HIVE-3521: -- Thank you for the tips, Andy. I have seen similar problem in our Hive 0.9.0 logs regularly in the past few months and had no idea what to do about it. I did not bother configuring DataNucelus because there is nowhere any information about 1) Hive requiring that and 2) how it should be configured (nowhere = Google, Programming Hive). Anyway, from your comment and the sparse documentation I have figured out that setting datanucleus.autoStartMechanism to SchemaTable in hive-site.xml might help. After I restarted Hive, it has created NUCLEUS_TABLES in the database so it seems to have some effect. I will see in a few weeks/months if it will indeed fix the problem, i.e. if DataNucleus will load the list of tables from it on start, avoiding not knowing them later. For other readers trying to interpret your comment: * PMF/EMF are JDO's PersistenceManagerFactory / JPA's EntityManagerFactory, i.e. the classes used to get hold of something actually usable for persisting / reading stuff * Regarding the advice of setting datanucleus.PersistenceUnitLoadClassesto true in hive-site.xml, I don't know enough to be able to apply it. If I read the docs correctly, it requires datanucleus.PersistenceUnitName to be set to a name of a persistence-unit defined in a persistence.xml file - but I haven't found any such file in Hive 0.9.0 (the only JDO-related thing I have found aside of the metastore connection properties is package.jdo in hive-metastore-0.9.0.jar. I would appreciate tips for how to use this setting with Hive. * To me, locking of your PMF/EMF seems to be out of question since all that code is inside Hive and I only use Hive via hiveserver Concurrent metastore calls provoke Datanucleus IllegalStateException: Table object has not been been initialised -- Key: HIVE-3521 URL: https://issues.apache.org/jira/browse/HIVE-3521 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.9.0 Reporter: Carl Steinbach -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3521) Concurrent metastore calls provoke Datanucleus IllegalStateException: Table object has not been been initialised
[ https://issues.apache.org/jira/browse/HIVE-3521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13575197#comment-13575197 ] Andy Jefferson commented on HIVE-3521: -- I've never seen it, and that private company running that proprietary benchmark (for their proprietary product) didn't bother configuring DataNucleus (surprising?). Use of the DN persistence property datanucleus.PersistenceUnitLoadClasses at startup together with a persistence-unit, or alternatively use of an auto-start mechanism, as well as locking of your PMF/EMF until it was initialised would clearly mean that the internal schema was initialised. You seem to be executing a query and it is still discovering classes that map to your schema; not a good way of running Concurrent metastore calls provoke Datanucleus IllegalStateException: Table object has not been been initialised -- Key: HIVE-3521 URL: https://issues.apache.org/jira/browse/HIVE-3521 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.9.0 Reporter: Carl Steinbach -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3521) Concurrent metastore calls provoke Datanucleus IllegalStateException: Table object has not been been initialised
[
https://issues.apache.org/jira/browse/HIVE-3521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13467293#comment-13467293
]
Carl Steinbach commented on HIVE-3521:
--
I encountered this error while running a test fires metastore calls from
approximately ten threads at the same time.
Here is the exception stack trace:
{noformat}
12/10/01 14:46:53 ERROR hive.log: java.lang.IllegalStateException: Table object
has not been been initialised : `TBLS`
at
org.datanucleus.store.rdbms.table.AbstractTable.assertIsInitialized(AbstractTable.java:625)
at
org.datanucleus.store.rdbms.table.ClassTable.getMemberMapping(ClassTable.java:2996)
at
org.datanucleus.store.mapped.expression.LogicSetExpression.newFieldExpression(LogicSetExpression.java:195)
at
org.datanucleus.store.rdbms.query.legacy.JDOQLQueryCompiler.compileIdentifier(JDOQLQueryCompiler.java:953)
at
org.datanucleus.store.rdbms.query.legacy.JDOQLQueryCompiler.compilePrimary(JDOQLQueryCompiler.java:774)
at
org.datanucleus.store.rdbms.query.legacy.QueryCompiler.compileUnaryExpressionNotPlusMinus(QueryCompiler.java:1333)
at
org.datanucleus.store.rdbms.query.legacy.QueryCompiler.compileUnaryExpression(QueryCompiler.java:1314)
at
org.datanucleus.store.rdbms.query.legacy.QueryCompiler.compileMultiplicativeExpression(QueryCompiler.java:1267)
at
org.datanucleus.store.rdbms.query.legacy.QueryCompiler.compileAdditiveExpression(QueryCompiler.java:1244)
at
org.datanucleus.store.rdbms.query.legacy.JDOQLQueryCompiler.compileRelationalExpression(JDOQLQueryCompiler.java:687)
at
org.datanucleus.store.rdbms.query.legacy.JDOQLQueryCompiler.compileEqualityExpression(JDOQLQueryCompiler.java:659)
at
org.datanucleus.store.rdbms.query.legacy.JDOQLQueryCompiler.compileAndExpression(JDOQLQueryCompiler.java:647)
at
org.datanucleus.store.rdbms.query.legacy.JDOQLQueryCompiler.compileExclusiveOrExpression(JDOQLQueryCompiler.java:635)
at
org.datanucleus.store.rdbms.query.legacy.JDOQLQueryCompiler.compileInclusiveOrExpression(JDOQLQueryCompiler.java:623)
at
org.datanucleus.store.rdbms.query.legacy.JDOQLQueryCompiler.compileConditionalAndExpression(JDOQLQueryCompiler.java:611)
at
org.datanucleus.store.rdbms.query.legacy.JDOQLQueryCompiler.compileConditionalOrExpression(JDOQLQueryCompiler.java:593)
at
org.datanucleus.store.rdbms.query.legacy.JDOQLQueryCompiler.compileExpression(JDOQLQueryCompiler.java:570)
at
org.datanucleus.store.rdbms.query.legacy.JDOQLQueryCompiler.compileExpressionFromString(JDOQLQueryCompiler.java:550)
at
org.datanucleus.store.rdbms.query.legacy.QueryCompiler.compileExpressionsFromString(QueryCompiler.java:1221)
at
org.datanucleus.store.rdbms.query.legacy.QueryCompiler.compileResult(QueryCompiler.java:876)
at
org.datanucleus.store.rdbms.query.legacy.JDOQLQueryCompiler.performCompile(JDOQLQueryCompiler.java:246)
at
org.datanucleus.store.rdbms.query.legacy.QueryCompiler.executionCompile(QueryCompiler.java:374)
at
org.datanucleus.store.rdbms.query.legacy.JDOQLQueryCompiler.compile(JDOQLQueryCompiler.java:225)
at
org.datanucleus.store.rdbms.query.legacy.JDOQLQuery.compileInternal(JDOQLQuery.java:175)
at org.datanucleus.store.query.Query.executeQuery(Query.java:1628)
at
org.datanucleus.store.rdbms.query.legacy.JDOQLQuery.executeQuery(JDOQLQuery.java:245)
at org.datanucleus.store.query.Query.executeWithArray(Query.java:1499)
at org.datanucleus.jdo.JDOQuery.execute(JDOQuery.java:243)
at
org.apache.hadoop.hive.metastore.ObjectStore.getTables(ObjectStore.java:759)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.hive.metastore.RetryingRawStore.invoke(RetryingRawStore.java:111)
at $Proxy0.getTables(Unknown Source)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_tables(HiveMetaStore.java:1928)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTables(HiveMetaStoreClient.java:786)
at
org.apache.hadoop.hive.ql.metadata.Hive.getTablesByPattern(Hive.java:976)
at org.apache.hadoop.hive.ql.metadata.Hive.getAllTables(Hive.java:950)
at org.apache.hadoop.hive.ql.exec.DDLTask.showTables(DDLTask.java:2007)
at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:324)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153)
at
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
at
