[
https://issues.apache.org/jira/browse/IMPALA-10372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Quanlong Huang updated IMPALA-10372:
------------------------------------
Attachment: hive-metastore.log.gz
> Data loading fails with foreign key constraint violation in postgresql
> ----------------------------------------------------------------------
>
> Key: IMPALA-10372
> URL: https://issues.apache.org/jira/browse/IMPALA-10372
> Project: IMPALA
> Issue Type: Bug
> Reporter: Quanlong Huang
> Priority: Major
> Attachments: hive-metastore.log.gz
>
>
> Saw a failure in data loading in a private build:
> {code}
> ERROR: DROP TABLE IF EXISTS
> functional_seq_record_bzip.insert_only_transactional_table
> Traceback (most recent call last):
> File
> "/data/jenkins/workspace/impala-private-parameterized/repos/Impala/bin/load-data.py",
> line 208, in exec_impala_query_from_file
> result = impala_client.execute(query)
> File
> "/data/jenkins/workspace/impala-private-parameterized/repos/Impala/tests/beeswax/impala_beeswax.py",
> line 187, in execute
> handle = self.__execute_query(query_string.strip(), user=user)
> File
> "/data/jenkins/workspace/impala-private-parameterized/repos/Impala/tests/beeswax/impala_beeswax.py",
> line 363, in __execute_query
> handle = self.execute_query_async(query_string, user=user)
> File
> "/data/jenkins/workspace/impala-private-parameterized/repos/Impala/tests/beeswax/impala_beeswax.py",
> line 357, in execute_query_async
> handle = self.__do_rpc(lambda: self.imp_service.query(query,))
> File
> "/data/jenkins/workspace/impala-private-parameterized/repos/Impala/tests/beeswax/impala_beeswax.py",
> line 520, in __do_rpc
> raise ImpalaBeeswaxException(self.__build_error_message(b), b)
> ImpalaBeeswaxException: ImpalaBeeswaxException:
> INNER EXCEPTION: <class 'beeswaxd.ttypes.BeeswaxException'>
> MESSAGE: ImpalaRuntimeException: Error making 'dropTable' RPC to Hive
> Metastore:
> CAUSED BY: MetaException: Exception thrown flushing changes to datastore
> {code}
> The root exception is
> {code}
> ERROR: update or delete on table "TBLS" violates foreign key constraint
> "MV_TABLES_USED_FK2" on table "MV_TABLES_USED"
> {code}
> Found it in hive-metastore.log:
> {code}
> 2020-12-01T04:37:32,946 INFO [pool-6-thread-18] metastore.HiveMetaStore: 27:
> source:127.0.0.1 get_table :
> tbl=hive.functional_seq_record_bzip.insert_only_transactional_table
> 2020-12-01T04:37:32,946 INFO [pool-6-thread-18] HiveMetaStore.audit:
> ugi=jenkins ip=127.0.0.1 cmd=source:127.0.0.1 get_table :
> tbl=hive.functional_seq_record_bzip.insert_only_transactional_table
> 2020-12-01T04:37:32,955 INFO [pool-6-thread-18]
> metastore.MetastoreDefaultTransformer: Starting translation for processor
> Impala4.0.0-SNAPSHOT@localhost on list 1
> 2020-12-01T04:37:32,955 INFO [pool-6-thread-18]
> metastore.MetastoreDefaultTransformer: Table
> insert_only_transactional_table,#bucket=0,isBucketed:false,tableType=MANAGED_TABLE,tableCapabilities=HIVEMANAGEDINSERTREAD,HIVEMANAGEDINSERTWRITE
> 2020-12-01T04:37:32,955 INFO [pool-6-thread-18]
> metastore.MetastoreDefaultTransformer: Abilities for match: Table
> type=MANAGED_TABLE,accesstype is RW
> 2020-12-01T04:37:32,955 INFO [pool-6-thread-18]
> metastore.MetastoreDefaultTransformer: Transformer return list of 1
> 2020-12-01T04:37:32,961 INFO [pool-6-thread-18] metastore.HiveMetaStore: 27:
> source:127.0.0.1 drop_table :
> tbl=hive.functional_seq_record_bzip.insert_only_transactional_table
> 2020-12-01T04:37:32,961 INFO [pool-6-thread-18] HiveMetaStore.audit:
> ugi=jenkins ip=127.0.0.1 cmd=source:127.0.0.1 drop_table :
> tbl=hive.functional_seq_record_bzip.insert_only_transactional_table
> 2020-12-01T04:37:33,143 ERROR [pool-6-thread-18]
> metastore.RetryingHMSHandler: Retrying HMSHandler after 2000 ms (attempt 1 of
> 10) with error: javax.jdo.JDODataStoreException: Exception thrown flushing
> changes to datastore
> ......
> 2020-12-01T04:37:53,690 INFO [pool-6-thread-18] metastore.HiveMetaStore: 27:
> source:127.0.0.1 drop_table :
> tbl=hive.functional_seq_record_bzip.insert_only_transactional_table
> 2020-12-01T04:37:53,690 INFO [pool-6-thread-18] HiveMetaStore.audit:
> ugi=jenkins ip=127.0.0.1 cmd=source:127.0.0.1 drop_table :
> tbl=hive.functional_seq_record_bzip.insert_only_transactional_table
> 2020-12-01T04:37:53,709 ERROR [pool-6-thread-18]
> metastore.RetryingHMSHandler: HMSHandler Fatal error:
> javax.jdo.JDODataStoreException: Exception thrown flushing changes to
> datastore
> at
> org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:543)
> at
> org.datanucleus.api.jdo.JDOPersistenceManager.flush(JDOPersistenceManager.java:2058)
> at
> org.apache.hadoop.hive.metastore.ObjectStore.addNotificationEvent(ObjectStore.java:10827)
> at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:97)
> at com.sun.proxy.$Proxy27.addNotificationEvent(Unknown Source)
> at
> org.apache.hive.hcatalog.listener.DbNotificationListener.process(DbNotificationListener.java:1123)
> at
> org.apache.hive.hcatalog.listener.DbNotificationListener.onDropTable(DbNotificationListener.java:234)
> at
> org.apache.hadoop.hive.metastore.MetaStoreListenerNotifier$19.notify(MetaStoreListenerNotifier.java:103)
> at
> org.apache.hadoop.hive.metastore.MetaStoreListenerNotifier.notifyEvent(MetaStoreListenerNotifier.java:285)
> at
> org.apache.hadoop.hive.metastore.MetaStoreListenerNotifier.notifyEvent(MetaStoreListenerNotifier.java:347)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.drop_table_core(HiveMetaStore.java:2991)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.drop_table_with_environment_context(HiveMetaStore.java:3231)
> at sun.reflect.GeneratedMethodAccessor33.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
> at com.sun.proxy.$Proxy28.drop_table_with_environment_context(Unknown
> Source)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$drop_table_with_environment_context.getResult(ThriftHiveMetastore.java:16875)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$drop_table_with_environment_context.getResult(ThriftHiveMetastore.java:16859)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:111)
> at
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:107)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1898)
> at
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:119)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> NestedThrowablesStackTrace:
> java.sql.BatchUpdateException: Batch entry 0 DELETE FROM "TBLS" WHERE
> "TBL_ID"=713 was aborted: ERROR: update or delete on table "TBLS" violates
> foreign key constraint "MV_TABLES_USED_FK2" on table "MV_TABLES_USED"
> Detail: Key (TBL_ID)=(713) is still referenced from table "MV_TABLES_USED".
> Call getNextException to see other errors in the batch.
> at
> org.postgresql.jdbc.BatchResultHandler.handleError(BatchResultHandler.java:159)
> at
> org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2268)
> at
> org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:510)
> at
> org.postgresql.jdbc.PgStatement.internalExecuteBatch(PgStatement.java:851)
> at org.postgresql.jdbc.PgStatement.executeBatch(PgStatement.java:874)
> at
> org.postgresql.jdbc.PgPreparedStatement.executeBatch(PgPreparedStatement.java:1563)
> at
> com.zaxxer.hikari.pool.ProxyStatement.executeBatch(ProxyStatement.java:125)
> at
> com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeBatch(HikariProxyPreparedStatement.java)
> at
> org.datanucleus.store.rdbms.ParamLoggingPreparedStatement.executeBatch(ParamLoggingPreparedStatement.java:366)
> at
> org.datanucleus.store.rdbms.SQLController.processConnectionStatement(SQLController.java:676)
> at
> org.datanucleus.store.rdbms.SQLController.processStatementsForConnection(SQLController.java:644)
> at
> org.datanucleus.store.rdbms.SQLController$1.transactionFlushed(SQLController.java:731)
> at
> org.datanucleus.store.connection.AbstractManagedConnection.transactionFlushed(AbstractManagedConnection.java:89)
> at
> org.datanucleus.store.connection.ConnectionManagerImpl$2.transactionFlushed(ConnectionManagerImpl.java:450)
> at org.datanucleus.TransactionImpl.flush(TransactionImpl.java:210)
> at
> org.datanucleus.ExecutionContextImpl.flushInternal(ExecutionContextImpl.java:4060)
> at
> org.datanucleus.ExecutionContextThreadedImpl.flushInternal(ExecutionContextThreadedImpl.java:450)
> at
> org.datanucleus.ExecutionContextImpl.flush(ExecutionContextImpl.java:4001)
> at
> org.datanucleus.ExecutionContextThreadedImpl.flush(ExecutionContextThreadedImpl.java:436)
> at
> org.datanucleus.api.jdo.JDOPersistenceManager.flush(JDOPersistenceManager.java:2040)
> at
> org.apache.hadoop.hive.metastore.ObjectStore.addNotificationEvent(ObjectStore.java:10827)
> at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:97)
> at com.sun.proxy.$Proxy27.addNotificationEvent(Unknown Source)
> at
> org.apache.hive.hcatalog.listener.DbNotificationListener.process(DbNotificationListener.java:1123)
> at
> org.apache.hive.hcatalog.listener.DbNotificationListener.onDropTable(DbNotificationListener.java:234)
> at
> org.apache.hadoop.hive.metastore.MetaStoreListenerNotifier$19.notify(MetaStoreListenerNotifier.java:103)
> at
> org.apache.hadoop.hive.metastore.MetaStoreListenerNotifier.notifyEvent(MetaStoreListenerNotifier.java:285)
> at
> org.apache.hadoop.hive.metastore.MetaStoreListenerNotifier.notifyEvent(MetaStoreListenerNotifier.java:347)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.drop_table_core(HiveMetaStore.java:2991)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.drop_table_with_environment_context(HiveMetaStore.java:3231)
> at sun.reflect.GeneratedMethodAccessor33.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
> at com.sun.proxy.$Proxy28.drop_table_with_environment_context(Unknown
> Source)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$drop_table_with_environment_context.getResult(ThriftHiveMetastore.java:16875)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$drop_table_with_environment_context.getResult(ThriftHiveMetastore.java:16859)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:111)
> at
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:107)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1898)
> at
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:119)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.postgresql.util.PSQLException: ERROR: update or delete on
> table "TBLS" violates foreign key constraint "MV_TABLES_USED_FK2" on table
> "MV_TABLES_USED"
> Detail: Key (TBL_ID)=(713) is still referenced from table "MV_TABLES_USED".
> at
> org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2532)
> at
> org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2267)
> ... 50 more
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
