Hi Jae, Sounds like your problem is related to HIVE-1435 ( https://issues.apache.org/jira/browse/HIVE-1435). You need to make sure that the Datanucleus ORM layer is getting initialized with the configuration property datanucleus.identifierFactory=datanucleus. Probably the easiest way to fix this problem is to make sure that the 0.7.0 version of hive-default.xml is available on the CLASSPATH and is getting loaded into HiveConf. Try dumping the contents of your HiveConf object and make sure that the values match those that appear in the 0.7.0 version of hive-default.xml
Hope this helps. Carl On Mon, Mar 14, 2011 at 10:41 AM, Jae Lee <jlee...@gmail.com> wrote: > just a bit more information from my debugging so far > > my mysql hive metastore have columns like > "integer_idx" at "columns" table > "integer_idx" at "sort_cols" table > > those columns looks pretty suspicious in that it is similar to "idx" > columns > that HiveMetaSotreClient complains missing. > > It looks like expectation of having "idx" column is auto-generated (not > from > package.jdo document) > Can anybody tell me whether "integer_idx" column should have been "idx" > column at "columns" table? > or am I suppose to have custom package.jdo file that specify the index > column name to "integer_idx" instead of "idx" column? > > J > > On Mon, Mar 14, 2011 at 2:27 PM, Jae Lee <jlee...@gmail.com> wrote: > > > Hi, > > > > I've had this code below working with Hive 0.5 > > > > String databaseName = "default"; > > String tableName = "foobar"; > > List<org.apache.hadoop.hive.metastore.api.FieldSchema> hiveTable = new > > HiveMetaStoreClient(new HiveConf(new Configuration(), > > SessionState.class)).getSchema(databaseName, tableName); > > > > > > to produce list of FieldSchema for a table foobar in default database > > > > I've recently upgraded hive to 0.7, and the same code now generates an > > error messages such as > > > > 11/03/14 14:22:02 ERROR DataNucleus.Datastore: An exception was thrown > > while adding/validating class(es) : Required columns missing from table > > "`COLUMNS`" : `IDX`. Perhaps your MetaData is incorrect, or you havent > > enabled "datanucleus.autoCreateColumns". > > Required columns missing from table "`COLUMNS`" : `IDX`. Perhaps your > > MetaData is incorrect, or you havent enabled > > "datanucleus.autoCreateColumns". > > org.datanucleus.store.rdbms.exceptions.MissingColumnException: Required > > columns missing from table "`COLUMNS`" : `IDX`. Perhaps your MetaData is > > incorrect, or you havent enabled "datanucleus.autoCreateColumns". > > at > > > org.datanucleus.store.rdbms.table.TableImpl.validateColumns(TableImpl.java:282) > > at > org.datanucleus.store.rdbms.table.TableImpl.validate(TableImpl.java:175) > > at > > > org.datanucleus.store.rdbms.RDBMSStoreManager$ClassAdder.performTablesValidation(RDBMSStoreManager.java:2711) > > at > > > org.datanucleus.store.rdbms.RDBMSStoreManager$ClassAdder.addClassTablesAndValidate(RDBMSStoreManager.java:2503) > > at > > > org.datanucleus.store.rdbms.RDBMSStoreManager$ClassAdder.run(RDBMSStoreManager.java:2148) > > at > > > org.datanucleus.store.rdbms.AbstractSchemaTransaction.execute(AbstractSchemaTransaction.java:113) > > at > > > org.datanucleus.store.rdbms.RDBMSStoreManager.addClasses(RDBMSStoreManager.java:986) > > at > > > org.datanucleus.store.rdbms.RDBMSStoreManager.addClasses(RDBMSStoreManager.java:952) > > at > > > org.datanucleus.store.AbstractStoreManager.addClass(AbstractStoreManager.java:919) > > at > > > org.datanucleus.store.mapped.MappedStoreManager.getDatastoreClass(MappedStoreManager.java:356) > > at > > > org.datanucleus.store.rdbms.query.legacy.ExtentHelper.getExtent(ExtentHelper.java:48) > > at > > > org.datanucleus.store.rdbms.RDBMSStoreManager.getExtent(RDBMSStoreManager.java:1332) > > at > > org.datanucleus.ObjectManagerImpl.getExtent(ObjectManagerImpl.java:4149) > > at > > > org.datanucleus.store.rdbms.query.legacy.JDOQLQueryCompiler.compileCandidates(JDOQLQueryCompiler.java:411) > > at > > > org.datanucleus.store.rdbms.query.legacy.QueryCompiler.executionCompile(QueryCompiler.java:312) > > at > > > org.datanucleus.store.rdbms.query.legacy.JDOQLQueryCompiler.compile(JDOQLQueryCompiler.java:225) > > at > > > org.datanucleus.store.rdbms.query.legacy.JDOQLQuery.compileInternal(JDOQLQuery.java:175) > > at org.datanucleus.store.query.Query.executeQuery(Query.java:1628) > > at > > > org.datanucleus.store.rdbms.query.legacy.JDOQLQuery.executeQuery(JDOQLQuery.java:245) > > at org.datanucleus.store.query.Query.executeWithArray(Query.java:1499) > > at org.datanucleus.jdo.JDOQuery.execute(JDOQuery.java:266) > > at > > > org.apache.hadoop.hive.metastore.ObjectStore.getMTable(ObjectStore.java:775) > > at > > > org.apache.hadoop.hive.metastore.ObjectStore.getTable(ObjectStore.java:709) > > at > > > org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler$17.run(HiveMetaStore.java:1076) > > at > > > org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler$17.run(HiveMetaStore.java:1073) > > at > > > org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.executeWithRetry(HiveMetaStore.java:307) > > at > > > org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table(HiveMetaStore.java:1073) > > at > > > org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_schema(HiveMetaStore.java:1785) > > at > > > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getSchema(HiveMetaStoreClient.java:857) > > at > > > HiveMetaStoreClientTest.shouldGetSchemaFromMetaStore(HiveMetaStoreClientTest.java:10) > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > at > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > > at > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > > at java.lang.reflect.Method.invoke(Method.java:597) > > at > > > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44) > > at > > > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) > > at > > > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41) > > at > > > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) > > at > > > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) > > at > > > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31) > > at > > > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:73) > > at > > > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:46) > > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:180) > > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:41) > > at org.junit.runners.ParentRunner$1.evaluate(ParentRunner.java:173) > > at > > > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) > > at > > > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31) > > at org.junit.runners.ParentRunner.run(ParentRunner.java:220) > > at org.junit.runner.JUnitCore.run(JUnitCore.java:137) > > at > > > com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:94) > > at > > > com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:196) > > at > com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:65) > > > > 11/03/14 14:22:02 ERROR DataNucleus.Datastore: An exception was thrown > > while adding/validating class(es) : Required columns missing from table > > "`SORT_COLS`" : `IDX`. Perhaps your MetaData is incorrect, or you havent > > enabled "datanucleus.autoCreateColumns". > > Required columns missing from table "`SORT_COLS`" : `IDX`. Perhaps your > > MetaData is incorrect, or you havent enabled > > "datanucleus.autoCreateColumns". > > org.datanucleus.store.rdbms.exceptions.MissingColumnException: Required > > columns missing from table "`SORT_COLS`" : `IDX`. Perhaps your MetaData > is > > incorrect, or you havent enabled "datanucleus.autoCreateColumns". > > at > > > org.datanucleus.store.rdbms.table.TableImpl.validateColumns(TableImpl.java:282) > > at > org.datanucleus.store.rdbms.table.TableImpl.validate(TableImpl.java:175) > > at > > > org.datanucleus.store.rdbms.RDBMSStoreManager$ClassAdder.performTablesValidation(RDBMSStoreManager.java:2711) > > at > > > org.datanucleus.store.rdbms.RDBMSStoreManager$ClassAdder.addClassTablesAndValidate(RDBMSStoreManager.java:2503) > > at > > > org.datanucleus.store.rdbms.RDBMSStoreManager$ClassAdder.run(RDBMSStoreManager.java:2148) > > at > > > org.datanucleus.store.rdbms.AbstractSchemaTransaction.execute(AbstractSchemaTransaction.java:113) > > at > > > org.datanucleus.store.rdbms.RDBMSStoreManager.addClasses(RDBMSStoreManager.java:986) > > at > > > org.datanucleus.store.rdbms.RDBMSStoreManager.addClasses(RDBMSStoreManager.java:952) > > at > > > org.datanucleus.store.AbstractStoreManager.addClass(AbstractStoreManager.java:919) > > at > > > org.datanucleus.store.mapped.MappedStoreManager.getDatastoreClass(MappedStoreManager.java:356) > > at > > > org.datanucleus.store.rdbms.query.legacy.ExtentHelper.getExtent(ExtentHelper.java:48) > > at > > > org.datanucleus.store.rdbms.RDBMSStoreManager.getExtent(RDBMSStoreManager.java:1332) > > at > > org.datanucleus.ObjectManagerImpl.getExtent(ObjectManagerImpl.java:4149) > > at > > > org.datanucleus.store.rdbms.query.legacy.JDOQLQueryCompiler.compileCandidates(JDOQLQueryCompiler.java:411) > > at > > > org.datanucleus.store.rdbms.query.legacy.QueryCompiler.executionCompile(QueryCompiler.java:312) > > at > > > org.datanucleus.store.rdbms.query.legacy.JDOQLQueryCompiler.compile(JDOQLQueryCompiler.java:225) > > at > > > org.datanucleus.store.rdbms.query.legacy.JDOQLQuery.compileInternal(JDOQLQuery.java:175) > > at org.datanucleus.store.query.Query.executeQuery(Query.java:1628) > > at > > > org.datanucleus.store.rdbms.query.legacy.JDOQLQuery.executeQuery(JDOQLQuery.java:245) > > at org.datanucleus.store.query.Query.executeWithArray(Query.java:1499) > > at org.datanucleus.jdo.JDOQuery.execute(JDOQuery.java:266) > > at > > > org.apache.hadoop.hive.metastore.ObjectStore.getMTable(ObjectStore.java:775) > > at > > > org.apache.hadoop.hive.metastore.ObjectStore.getTable(ObjectStore.java:709) > > at > > > org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler$17.run(HiveMetaStore.java:1076) > > at > > > org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler$17.run(HiveMetaStore.java:1073) > > at > > > org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.executeWithRetry(HiveMetaStore.java:307) > > at > > > org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table(HiveMetaStore.java:1073) > > at > > > org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_schema(HiveMetaStore.java:1785) > > at > > > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getSchema(HiveMetaStoreClient.java:857) > > at > > > HiveMetaStoreClientTest.shouldGetSchemaFromMetaStore(HiveMetaStoreClientTest.java:10) > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > at > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > > at > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > > at java.lang.reflect.Method.invoke(Method.java:597) > > at > > > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44) > > at > > > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) > > at > > > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41) > > at > > > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) > > at > > > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) > > at > > > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31) > > at > > > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:73) > > at > > > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:46) > > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:180) > > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:41) > > at org.junit.runners.ParentRunner$1.evaluate(ParentRunner.java:173) > > at > > > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) > > at > > > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31) > > at org.junit.runners.ParentRunner.run(ParentRunner.java:220) > > at org.junit.runner.JUnitCore.run(JUnitCore.java:137) > > at > > > com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:94) > > at > > > com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:196) > > at > com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:65) > > > > 11/03/14 14:22:02 ERROR DataNucleus.Datastore: An exception was thrown > > while adding/validating class(es) : Expected primary key for table > > `SORT_COLS` PRIMARY KEY (`SD_ID`,`IDX`) not found in existing keys > PRIMARY > > KEY (`SD_ID`) > > Expected primary key for table `SORT_COLS` PRIMARY KEY (`SD_ID`,`IDX`) > not > > found in existing keys PRIMARY KEY (`SD_ID`) > > org.datanucleus.store.rdbms.exceptions.WrongPrimaryKeyException: Expected > > primary key for table `SORT_COLS` PRIMARY KEY (`SD_ID`,`IDX`) not found > in > > existing keys PRIMARY KEY (`SD_ID`) > > at > > > org.datanucleus.store.rdbms.table.TableImpl.validatePrimaryKey(TableImpl.java:368) > > at > org.datanucleus.store.rdbms.table.TableImpl.validate(TableImpl.java:180) > > at > > > org.datanucleus.store.rdbms.RDBMSStoreManager$ClassAdder.performTablesValidation(RDBMSStoreManager.java:2711) > > at > > > org.datanucleus.store.rdbms.RDBMSStoreManager$ClassAdder.addClassTablesAndValidate(RDBMSStoreManager.java:2503) > > at > > > org.datanucleus.store.rdbms.RDBMSStoreManager$ClassAdder.run(RDBMSStoreManager.java:2148) > > at > > > org.datanucleus.store.rdbms.AbstractSchemaTransaction.execute(AbstractSchemaTransaction.java:113) > > at > > > org.datanucleus.store.rdbms.RDBMSStoreManager.addClasses(RDBMSStoreManager.java:986) > > at > > > org.datanucleus.store.rdbms.RDBMSStoreManager.addClasses(RDBMSStoreManager.java:952) > > at > > > org.datanucleus.store.AbstractStoreManager.addClass(AbstractStoreManager.java:919) > > at > > > org.datanucleus.store.mapped.MappedStoreManager.getDatastoreClass(MappedStoreManager.java:356) > > at > > > org.datanucleus.store.rdbms.query.legacy.ExtentHelper.getExtent(ExtentHelper.java:48) > > at > > > org.datanucleus.store.rdbms.RDBMSStoreManager.getExtent(RDBMSStoreManager.java:1332) > > at > > org.datanucleus.ObjectManagerImpl.getExtent(ObjectManagerImpl.java:4149) > > at > > > org.datanucleus.store.rdbms.query.legacy.JDOQLQueryCompiler.compileCandidates(JDOQLQueryCompiler.java:411) > > at > > > org.datanucleus.store.rdbms.query.legacy.QueryCompiler.executionCompile(QueryCompiler.java:312) > > at > > > org.datanucleus.store.rdbms.query.legacy.JDOQLQueryCompiler.compile(JDOQLQueryCompiler.java:225) > > at > > > org.datanucleus.store.rdbms.query.legacy.JDOQLQuery.compileInternal(JDOQLQuery.java:175) > > at org.datanucleus.store.query.Query.executeQuery(Query.java:1628) > > at > > > org.datanucleus.store.rdbms.query.legacy.JDOQLQuery.executeQuery(JDOQLQuery.java:245) > > at org.datanucleus.store.query.Query.executeWithArray(Query.java:1499) > > at org.datanucleus.jdo.JDOQuery.execute(JDOQuery.java:266) > > at > > > org.apache.hadoop.hive.metastore.ObjectStore.getMTable(ObjectStore.java:775) > > at > > > org.apache.hadoop.hive.metastore.ObjectStore.getTable(ObjectStore.java:709) > > at > > > org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler$17.run(HiveMetaStore.java:1076) > > at > > > org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler$17.run(HiveMetaStore.java:1073) > > at > > > org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.executeWithRetry(HiveMetaStore.java:307) > > at > > > org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table(HiveMetaStore.java:1073) > > at > > > org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_schema(HiveMetaStore.java:1785) > > at > > > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getSchema(HiveMetaStoreClient.java:857) > > at > > > HiveMetaStoreClientTest.shouldGetSchemaFromMetaStore(HiveMetaStoreClientTest.java:10) > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > at > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > > at > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > > at java.lang.reflect.Method.invoke(Method.java:597) > > at > > > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44) > > at > > > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) > > at > > > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41) > > at > > > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) > > at > > > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) > > at > > > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31) > > at > > > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:73) > > at > > > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:46) > > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:180) > > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:41) > > at org.junit.runners.ParentRunner$1.evaluate(ParentRunner.java:173) > > at > > > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) > > at > > > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31) > > at org.junit.runners.ParentRunner.run(ParentRunner.java:220) > > at org.junit.runner.JUnitCore.run(JUnitCore.java:137) > > at > > > com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:94) > > at > > > com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:196) > > at > com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:65) > > > > 11/03/14 14:22:02 ERROR DataNucleus.Datastore: An exception was thrown > > while adding/validating class(es) : Required columns missing from table > > "`BUCKETING_COLS`" : `IDX`. Perhaps your MetaData is incorrect, or you > > havent enabled "datanucleus.autoCreateColumns". > > Required columns missing from table "`BUCKETING_COLS`" : `IDX`. Perhaps > > your MetaData is incorrect, or you havent enabled > > "datanucleus.autoCreateColumns". > > org.datanucleus.store.rdbms.exceptions.MissingColumnException: Required > > columns missing from table "`BUCKETING_COLS`" : `IDX`. Perhaps your > MetaData > > is incorrect, or you havent enabled "datanucleus.autoCreateColumns". > > at > > > org.datanucleus.store.rdbms.table.TableImpl.validateColumns(TableImpl.java:282) > > at > org.datanucleus.store.rdbms.table.TableImpl.validate(TableImpl.java:175) > > at > > > org.datanucleus.store.rdbms.RDBMSStoreManager$ClassAdder.performTablesValidation(RDBMSStoreManager.java:2711) > > at > > > org.datanucleus.store.rdbms.RDBMSStoreManager$ClassAdder.addClassTablesAndValidate(RDBMSStoreManager.java:2503) > > at > > > org.datanucleus.store.rdbms.RDBMSStoreManager$ClassAdder.run(RDBMSStoreManager.java:2148) > > at > > > org.datanucleus.store.rdbms.AbstractSchemaTransaction.execute(AbstractSchemaTransaction.java:113) > > at > > > org.datanucleus.store.rdbms.RDBMSStoreManager.addClasses(RDBMSStoreManager.java:986) > > at > > > org.datanucleus.store.rdbms.RDBMSStoreManager.addClasses(RDBMSStoreManager.java:952) > > at > > > org.datanucleus.store.AbstractStoreManager.addClass(AbstractStoreManager.java:919) > > at > > > org.datanucleus.store.mapped.MappedStoreManager.getDatastoreClass(MappedStoreManager.java:356) > > at > > > org.datanucleus.store.rdbms.query.legacy.ExtentHelper.getExtent(ExtentHelper.java:48) > > at > > > org.datanucleus.store.rdbms.RDBMSStoreManager.getExtent(RDBMSStoreManager.java:1332) > > at > > org.datanucleus.ObjectManagerImpl.getExtent(ObjectManagerImpl.java:4149) > > at > > > org.datanucleus.store.rdbms.query.legacy.JDOQLQueryCompiler.compileCandidates(JDOQLQueryCompiler.java:411) > > at > > > org.datanucleus.store.rdbms.query.legacy.QueryCompiler.executionCompile(QueryCompiler.java:312) > > at > > > org.datanucleus.store.rdbms.query.legacy.JDOQLQueryCompiler.compile(JDOQLQueryCompiler.java:225) > > at > > > org.datanucleus.store.rdbms.query.legacy.JDOQLQuery.compileInternal(JDOQLQuery.java:175) > > at org.datanucleus.store.query.Query.executeQuery(Query.java:1628) > > at > > > org.datanucleus.store.rdbms.query.legacy.JDOQLQuery.executeQuery(JDOQLQuery.java:245) > > at org.datanucleus.store.query.Query.executeWithArray(Query.java:1499) > > at org.datanucleus.jdo.JDOQuery.execute(JDOQuery.java:266) > > at > > > org.apache.hadoop.hive.metastore.ObjectStore.getMTable(ObjectStore.java:775) > > at > > > org.apache.hadoop.hive.metastore.ObjectStore.getTable(ObjectStore.java:709) > > at > > > org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler$17.run(HiveMetaStore.java:1076) > > at > > > org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler$17.run(HiveMetaStore.java:1073) > > at > > > org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.executeWithRetry(HiveMetaStore.java:307) > > at > > > org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table(HiveMetaStore.java:1073) > > at > > > org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_schema(HiveMetaStore.java:1785) > > at > > > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getSchema(HiveMetaStoreClient.java:857) > > at > > > HiveMetaStoreClientTest.shouldGetSchemaFromMetaStore(HiveMetaStoreClientTest.java:10) > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > at > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > > at > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > > at java.lang.reflect.Method.invoke(Method.java:597) > > at > > > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44) > > at > > > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) > > at > > > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41) > > at > > > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) > > at > > > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) > > at > > > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31) > > at > > > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:73) > > at > > > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:46) > > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:180) > > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:41) > > at org.junit.runners.ParentRunner$1.evaluate(ParentRunner.java:173) > > at > > > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) > > at > > > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31) > > at org.junit.runners.ParentRunner.run(ParentRunner.java:220) > > at org.junit.runner.JUnitCore.run(JUnitCore.java:137) > > at > > > com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:94) > > at > > > com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:196) > > at > com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:65) > > > > 11/03/14 14:22:02 ERROR DataNucleus.Datastore: An exception was thrown > > while adding/validating class(es) : Expected primary key for table > > `BUCKETING_COLS` PRIMARY KEY (`SD_ID`,`IDX`) not found in existing keys > > PRIMARY KEY (`SD_ID`) > > Expected primary key for table `BUCKETING_COLS` PRIMARY KEY > (`SD_ID`,`IDX`) > > not found in existing keys PRIMARY KEY (`SD_ID`) > > org.datanucleus.store.rdbms.exceptions.WrongPrimaryKeyException: Expected > > primary key for table `BUCKETING_COLS` PRIMARY KEY (`SD_ID`,`IDX`) not > found > > in existing keys PRIMARY KEY (`SD_ID`) > > at > > > org.datanucleus.store.rdbms.table.TableImpl.validatePrimaryKey(TableImpl.java:368) > > at > org.datanucleus.store.rdbms.table.TableImpl.validate(TableImpl.java:180) > > at > > > org.datanucleus.store.rdbms.RDBMSStoreManager$ClassAdder.performTablesValidation(RDBMSStoreManager.java:2711) > > at > > > org.datanucleus.store.rdbms.RDBMSStoreManager$ClassAdder.addClassTablesAndValidate(RDBMSStoreManager.java:2503) > > at > > > org.datanucleus.store.rdbms.RDBMSStoreManager$ClassAdder.run(RDBMSStoreManager.java:2148) > > at > > > org.datanucleus.store.rdbms.AbstractSchemaTransaction.execute(AbstractSchemaTransaction.java:113) > > at > > > org.datanucleus.store.rdbms.RDBMSStoreManager.addClasses(RDBMSStoreManager.java:986) > > at > > > org.datanucleus.store.rdbms.RDBMSStoreManager.addClasses(RDBMSStoreManager.java:952) > > at > > > org.datanucleus.store.AbstractStoreManager.addClass(AbstractStoreManager.java:919) > > at > > > org.datanucleus.store.mapped.MappedStoreManager.getDatastoreClass(MappedStoreManager.java:356) > > at > > > org.datanucleus.store.rdbms.query.legacy.ExtentHelper.getExtent(ExtentHelper.java:48) > > at > > > org.datanucleus.store.rdbms.RDBMSStoreManager.getExtent(RDBMSStoreManager.java:1332) > > at > > org.datanucleus.ObjectManagerImpl.getExtent(ObjectManagerImpl.java:4149) > > at > > > org.datanucleus.store.rdbms.query.legacy.JDOQLQueryCompiler.compileCandidates(JDOQLQueryCompiler.java:411) > > at > > > org.datanucleus.store.rdbms.query.legacy.QueryCompiler.executionCompile(QueryCompiler.java:312) > > at > > > org.datanucleus.store.rdbms.query.legacy.JDOQLQueryCompiler.compile(JDOQLQueryCompiler.java:225) > > at > > > org.datanucleus.store.rdbms.query.legacy.JDOQLQuery.compileInternal(JDOQLQuery.java:175) > > at org.datanucleus.store.query.Query.executeQuery(Query.java:1628) > > at > > > org.datanucleus.store.rdbms.query.legacy.JDOQLQuery.executeQuery(JDOQLQuery.java:245) > > at org.datanucleus.store.query.Query.executeWithArray(Query.java:1499) > > at org.datanucleus.jdo.JDOQuery.execute(JDOQuery.java:266) > > at > > > org.apache.hadoop.hive.metastore.ObjectStore.getMTable(ObjectStore.java:775) > > at > > > org.apache.hadoop.hive.metastore.ObjectStore.getTable(ObjectStore.java:709) > > at > > > org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler$17.run(HiveMetaStore.java:1076) > > at > > > org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler$17.run(HiveMetaStore.java:1073) > > at > > > org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.executeWithRetry(HiveMetaStore.java:307) > > at > > > org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table(HiveMetaStore.java:1073) > > at > > > org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_schema(HiveMetaStore.java:1785) > > at > > > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getSchema(HiveMetaStoreClient.java:857) > > at > > > HiveMetaStoreClientTest.shouldGetSchemaFromMetaStore(HiveMetaStoreClientTest.java:10) > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > at > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > > at > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > > at java.lang.reflect.Method.invoke(Method.java:597) > > at > > > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44) > > at > > > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) > > at > > > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41) > > at > > > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) > > at > > > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) > > at > > > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31) > > at > > > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:73) > > at > > > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:46) > > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:180) > > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:41) > > at org.junit.runners.ParentRunner$1.evaluate(ParentRunner.java:173) > > at > > > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) > > at > > > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31) > > at org.junit.runners.ParentRunner.run(ParentRunner.java:220) > > at org.junit.runner.JUnitCore.run(JUnitCore.java:137) > > at > > > com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:94) > > at > > > com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:196) > > at > com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:65) > > > > I've debugged through the code and found that those missing columns were > > added by datanucleus JDO code. I'm not sure what this means, rest of the > > hive applications are working perfectly fine. > > > > Is there anything that I should try to figure out what's going on? or > just > > generally is it a right way to get a schema from hive table? > > > > J > > >
