[
https://issues.apache.org/jira/browse/HIVE-12937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sushanth Sowmyan updated HIVE-12937:
------------------------------------
Status: Patch Available (was: Open)
> DbNotificationListener unable to clean up old notification events
> -----------------------------------------------------------------
>
> Key: HIVE-12937
> URL: https://issues.apache.org/jira/browse/HIVE-12937
> Project: Hive
> Issue Type: Bug
> Affects Versions: 2.0.0, 1.2.1, 1.3.0, 2.1.0
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
> Attachments: HIVE-12937.patch
>
>
> There is a bug in ObjectStore, where we use pm.deletePersistent instead of
> pm.deletePersistentAll, which causes the persistenceManager to try and drop a
> org.datanucleus.store.rdbms.query.ForwardQueryResult instead of the
> appropriate associated
> org.apache.hadoop.hive.metastore.model.MNotificationLog.
> This results in an error that looks like this:
> {noformat}
> Exception in thread "CleanerThread"
> org.datanucleus.api.jdo.exceptions.ClassNotPersistenceCapableException: The
> class "org.datanucleus.store.rdbms.query.ForwardQueryResult" is not
> persistable. This means that it either hasnt been enhanced, or that the
> enhanced version of the file is not in the CLASSPATH (or is hidden by an
> unenhanced version), or the Meta-Data/annotations for the class are not found.
> at
> org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:380)
> at
> org.datanucleus.api.jdo.JDOPersistenceManager.jdoDeletePersistent(JDOPersistenceManager.java:807)
> at
> org.datanucleus.api.jdo.JDOPersistenceManager.deletePersistent(JDOPersistenceManager.java:820)
> at
> org.apache.hadoop.hive.metastore.ObjectStore.cleanNotificationEvents(ObjectStore.java:7149)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:114)
> at com.sun.proxy.$Proxy0.cleanNotificationEvents(Unknown Source)
> at
> org.apache.hive.hcatalog.listener.DbNotificationListener$CleanerThread.run(DbNotificationListener.java:277)
> NestedThrowablesStackTrace:
> The class "org.datanucleus.store.rdbms.query.ForwardQueryResult" is not
> persistable. This means that it either hasnt been enhanced, or that the
> enhanced version of the file is not in the CLASSPATH (or is hidden by an
> unenhanced version), or the Meta-Data/annotations for the class are not found.
> org.datanucleus.exceptions.ClassNotPersistableException: The class
> "org.datanucleus.store.rdbms.query.ForwardQueryResult" is not persistable.
> This means that it either hasnt been enhanced, or that the enhanced version
> of the file is not in the CLASSPATH (or is hidden by an unenhanced version),
> or the Meta-Data/annotations for the class are not found.
> at
> org.datanucleus.ExecutionContextImpl.assertClassPersistable(ExecutionContextImpl.java:5698)
> at
> org.datanucleus.ExecutionContextImpl.deleteObjectInternal(ExecutionContextImpl.java:2495)
> at
> org.datanucleus.ExecutionContextImpl.deleteObjectWork(ExecutionContextImpl.java:2466)
> at
> org.datanucleus.ExecutionContextImpl.deleteObject(ExecutionContextImpl.java:2417)
> at
> org.datanucleus.ExecutionContextThreadedImpl.deleteObject(ExecutionContextThreadedImpl.java:245)
> at
> org.datanucleus.api.jdo.JDOPersistenceManager.jdoDeletePersistent(JDOPersistenceManager.java:802)
> at
> org.datanucleus.api.jdo.JDOPersistenceManager.deletePersistent(JDOPersistenceManager.java:820)
> at
> org.apache.hadoop.hive.metastore.ObjectStore.cleanNotificationEvents(ObjectStore.java:7149)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:114)
> at com.sun.proxy.$Proxy0.cleanNotificationEvents(Unknown Source)
> at
> org.apache.hive.hcatalog.listener.DbNotificationListener$CleanerThread.run(DbNotificationListener.java:277)
> {noformat}
> The end result of this bug is that users of DbNotificationListener will have
> an evergrowing number of notification events that are not cleaned up as they
> age. This is an easy enough fix, but shows that we have a lack of test
> coverage here.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
