Jason Dere created HIVE-20298: --------------------------------- Summary: Illegal null value in column `TBLS`.`WRITE_ID` Key: HIVE-20298 URL: https://issues.apache.org/jira/browse/HIVE-20298 Project: Hive Issue Type: Bug Components: Metastore Reporter: Jason Dere
Manually upgraded my existing local metastore using upgrade-3.0.0-to-3.1.0.mysql.sql, upgrade-3.1.0-to-3.2.0.mysql.sql, upgrade-3.2.0-to-4.0.0.mysql.sql. When running DESCRIBE EXTENDED of an existing table, I was getting the following error in hive.log. It looks like the ObjectStore/MTable classes don't seem to be able to support null values in the new writeId column that was added to the TBLS table in the metastore. cc [~sershe] [~ekoifman] {noformat} Caused by: javax.jdo.JDODataStoreException: Illegal null value in column `TBLS`.`WRITE_ID` NestedThrowables: org.datanucleus.store.rdbms.exceptions.NullValueException: Illegal null value in column `TBLS`.`WRITE_ID` at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:553) at org.datanucleus.api.jdo.JDOQuery.executeInternal(JDOQuery.java:391) at org.datanucleus.api.jdo.JDOQuery.execute(JDOQuery.java:255) at org.apache.hadoop.hive.metastore.ObjectStore.getMTable(ObjectStore.java:1802) at org.apache.hadoop.hive.metastore.ObjectStore.getMTable(ObjectStore.java:1838) at org.apache.hadoop.hive.metastore.ObjectStore.getTable(ObjectStore.java:1424) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:97) at com.sun.proxy.$Proxy39.getTable(Unknown Source) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table_core(HiveMetaStore.java:2950) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getTableInternal(HiveMetaStore.java:2898) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table_req(HiveMetaStore.java:2882) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147) ... 36 more Caused by: org.datanucleus.store.rdbms.exceptions.NullValueException: Illegal null value in column `TBLS`.`WRITE_ID` at org.datanucleus.store.rdbms.mapping.datastore.BigIntRDBMSMapping.getLong(BigIntRDBMSMapping.java:140) at org.datanucleus.store.rdbms.mapping.java.SingleFieldMapping.getLong(SingleFieldMapping.java:155) at org.datanucleus.store.rdbms.fieldmanager.ResultSetGetter.fetchLongField(ResultSetGetter.java:124) at org.datanucleus.state.AbstractStateManager.replacingLongField(AbstractStateManager.java:1549) at org.datanucleus.state.StateManagerImpl.replacingLongField(StateManagerImpl.java:120) at org.apache.hadoop.hive.metastore.model.MTable.dnReplaceField(MTable.java) at org.apache.hadoop.hive.metastore.model.MTable.dnReplaceFields(MTable.java) at org.datanucleus.state.StateManagerImpl.replaceFields(StateManagerImpl.java:3109) at org.datanucleus.store.rdbms.query.PersistentClassROF$1.fetchFields(PersistentClassROF.java:465) at org.datanucleus.state.StateManagerImpl.loadFieldValues(StateManagerImpl.java:2238) at org.datanucleus.state.StateManagerImpl.initialiseForHollow(StateManagerImpl.java:263) at org.datanucleus.state.ObjectProviderFactoryImpl.newForHollow(ObjectProviderFactoryImpl.java:112) at org.datanucleus.ExecutionContextImpl.findObject(ExecutionContextImpl.java:3097) at org.datanucleus.store.rdbms.query.PersistentClassROF.getObjectForDatastoreId(PersistentClassROF.java:460) at org.datanucleus.store.rdbms.query.PersistentClassROF.getObject(PersistentClassROF.java:385) at org.datanucleus.store.rdbms.query.ForwardQueryResult.nextResultSetElement(ForwardQueryResult.java:188) at org.datanucleus.store.rdbms.query.ForwardQueryResult$QueryResultIterator.next(ForwardQueryResult.java:416) at org.datanucleus.store.rdbms.query.ForwardQueryResult.processNumberOfResults(ForwardQueryResult.java:143) at org.datanucleus.store.rdbms.query.ForwardQueryResult.advanceToEndOfResultSet(ForwardQueryResult.java:171) at org.datanucleus.store.rdbms.query.ForwardQueryResult.getSizeUsingMethod(ForwardQueryResult.java:527) at org.datanucleus.store.query.AbstractQueryResult.size(AbstractQueryResult.java:357) at org.datanucleus.store.query.Query.executeQuery(Query.java:1874) at org.datanucleus.store.query.Query.executeWithArray(Query.java:1744) at org.datanucleus.api.jdo.JDOQuery.executeInternal(JDOQuery.java:368) ... 54 more {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)