[
https://issues.apache.org/jira/browse/HIVE-12937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sushanth Sowmyan updated HIVE-12937:
------------------------------------
Description:
There is a bug in ObjectStore, where we use pm.deletePersistent instead of
pm.deletePersistentAll, which causes the persistenceManager to try and drop a
org.datanucleus.store.rdbms.query.ForwardQueryResult instead of the appropriate
associated org.apache.hadoop.hive.metastore.model.MNotificationLog.
This results in an error that looks like this:
{noformat}
Exception in thread "CleanerThread"
org.datanucleus.api.jdo.exceptions.ClassNotPersistenceCapableException: The
class "org.datanucleus.store.rdbms.query.ForwardQueryResult" is not
persistable. This means that it either hasnt been enhanced, or that the
enhanced version of the file is not in the CLASSPATH (or is hidden by an
unenhanced version), or the Meta-Data/annotations for the class are not found.
at
org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:380)
at
org.datanucleus.api.jdo.JDOPersistenceManager.jdoDeletePersistent(JDOPersistenceManager.java:807)
at
org.datanucleus.api.jdo.JDOPersistenceManager.deletePersistent(JDOPersistenceManager.java:820)
at
org.apache.hadoop.hive.metastore.ObjectStore.cleanNotificationEvents(ObjectStore.java:7149)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:114)
at com.sun.proxy.$Proxy0.cleanNotificationEvents(Unknown Source)
at
org.apache.hive.hcatalog.listener.DbNotificationListener$CleanerThread.run(DbNotificationListener.java:277)
NestedThrowablesStackTrace:
The class "org.datanucleus.store.rdbms.query.ForwardQueryResult" is not
persistable. This means that it either hasnt been enhanced, or that the
enhanced version of the file is not in the CLASSPATH (or is hidden by an
unenhanced version), or the Meta-Data/annotations for the class are not found.
org.datanucleus.exceptions.ClassNotPersistableException: The class
"org.datanucleus.store.rdbms.query.ForwardQueryResult" is not persistable. This
means that it either hasnt been enhanced, or that the enhanced version of the
file is not in the CLASSPATH (or is hidden by an unenhanced version), or the
Meta-Data/annotations for the class are not found.
at
org.datanucleus.ExecutionContextImpl.assertClassPersistable(ExecutionContextImpl.java:5698)
at
org.datanucleus.ExecutionContextImpl.deleteObjectInternal(ExecutionContextImpl.java:2495)
at
org.datanucleus.ExecutionContextImpl.deleteObjectWork(ExecutionContextImpl.java:2466)
at
org.datanucleus.ExecutionContextImpl.deleteObject(ExecutionContextImpl.java:2417)
at
org.datanucleus.ExecutionContextThreadedImpl.deleteObject(ExecutionContextThreadedImpl.java:245)
at
org.datanucleus.api.jdo.JDOPersistenceManager.jdoDeletePersistent(JDOPersistenceManager.java:802)
at
org.datanucleus.api.jdo.JDOPersistenceManager.deletePersistent(JDOPersistenceManager.java:820)
at
org.apache.hadoop.hive.metastore.ObjectStore.cleanNotificationEvents(ObjectStore.java:7149)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:114)
at com.sun.proxy.$Proxy0.cleanNotificationEvents(Unknown Source)
at
org.apache.hive.hcatalog.listener.DbNotificationListener$CleanerThread.run(DbNotificationListener.java:277)
{noformat}
The end result of this bug is that users of DbNotificationListener will have an
evergrowing number of notification events that are not cleaned up as they age.
This is an easy enough fix, but shows that we have a lack of test coverage here.
was:
There is a bug in ObjectStore, where we use pm.deletePersistent instead of
pm.deletePersistentAll, which causes the persistenceManager to try and drop a
org.datanucleus.store.rdbms.query.ForwardQueryResult instead of the appropriate
associated org.apache.hadoop.hive.metastore.model.MNotificationLog.
This results in an error that looks like this:
{noformat}
Exception in thread "CleanerThread"
org.datanucleus.api.jdo.exceptions.ClassNotPersistenceCapableException: The
class "org.datanucleus.store.rdbms.query.ForwardQueryResult" is not
persistable. This means that it either hasnt been enhanced, or that the
enhanced version of the file is not in the CLASSPATH (or is hidden by an
unenhanced version), or the Meta-Data/annotations for the class are not found.
at
org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:380)
at
org.datanucleus.api.jdo.JDOPersistenceManager.jdoDeletePersistent(JDOPersistenceManager.java:807)
at
org.datanucleus.api.jdo.JDOPersistenceManager.deletePersistent(JDOPersistenceManager.java:820)
at
org.apache.hadoop.hive.metastore.ObjectStore.cleanNotificationEvents(ObjectStore.java:7149)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:114)
at com.sun.proxy.$Proxy0.cleanNotificationEvents(Unknown Source)
at
org.apache.hive.hcatalog.listener.DbNotificationListener$CleanerThread.run(DbNotificationListener.java:277)
NestedThrowablesStackTrace:
The class "org.datanucleus.store.rdbms.query.ForwardQueryResult" is not
persistable. This means that it either hasnt been enhanced, or that the
enhanced version of the file is not in the CLASSPATH (or is hidden by an
unenhanced version), or the Meta-Data/annotations for the class are not found.
org.datanucleus.exceptions.ClassNotPersistableException: The class
"org.datanucleus.store.rdbms.query.ForwardQueryResult" is not persistable. This
means that it either hasnt been enhanced, or that the enhanced version of the
file is not in the CLASSPATH (or is hidden by an unenhanced version), or the
Meta-Data/annotations for the class are not found.
at
org.datanucleus.ExecutionContextImpl.assertClassPersistable(ExecutionContextImpl.java:5698)
at
org.datanucleus.ExecutionContextImpl.deleteObjectInternal(ExecutionContextImpl.java:2495)
at
org.datanucleus.ExecutionContextImpl.deleteObjectWork(ExecutionContextImpl.java:2466)
at
org.datanucleus.ExecutionContextImpl.deleteObject(ExecutionContextImpl.java:2417)
at
org.datanucleus.ExecutionContextThreadedImpl.deleteObject(ExecutionContextThreadedImpl.java:245)
at
org.datanucleus.api.jdo.JDOPersistenceManager.jdoDeletePersistent(JDOPersistenceManager.java:802)
at
org.datanucleus.api.jdo.JDOPersistenceManager.deletePersistent(JDOPersistenceManager.java:820)
at
org.apache.hadoop.hive.metastore.ObjectStore.cleanNotificationEvents(ObjectStore.java:7149)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:114)
at com.sun.proxy.$Proxy0.cleanNotificationEvents(Unknown Source)
at
org.apache.hive.hcatalog.listener.DbNotificationListener$CleanerThread.run(DbNotificationListener.java:277)
{noformat}
The end result of this bug is that users of DbNotificationListener will have an
evergrowing number of notification events that are not cleaned up as they age.
This is an easy enough fix, but also shows that we have a lack of code coverage
here.
> DbNotificationListener unable to clean up old notification events
> -----------------------------------------------------------------
>
> Key: HIVE-12937
> URL: https://issues.apache.org/jira/browse/HIVE-12937
> Project: Hive
> Issue Type: Bug
> Affects Versions: 1.3.0, 1.2.1, 2.0.0, 2.1.0
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
>
> There is a bug in ObjectStore, where we use pm.deletePersistent instead of
> pm.deletePersistentAll, which causes the persistenceManager to try and drop a
> org.datanucleus.store.rdbms.query.ForwardQueryResult instead of the
> appropriate associated
> org.apache.hadoop.hive.metastore.model.MNotificationLog.
> This results in an error that looks like this:
> {noformat}
> Exception in thread "CleanerThread"
> org.datanucleus.api.jdo.exceptions.ClassNotPersistenceCapableException: The
> class "org.datanucleus.store.rdbms.query.ForwardQueryResult" is not
> persistable. This means that it either hasnt been enhanced, or that the
> enhanced version of the file is not in the CLASSPATH (or is hidden by an
> unenhanced version), or the Meta-Data/annotations for the class are not found.
> at
> org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:380)
> at
> org.datanucleus.api.jdo.JDOPersistenceManager.jdoDeletePersistent(JDOPersistenceManager.java:807)
> at
> org.datanucleus.api.jdo.JDOPersistenceManager.deletePersistent(JDOPersistenceManager.java:820)
> at
> org.apache.hadoop.hive.metastore.ObjectStore.cleanNotificationEvents(ObjectStore.java:7149)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:114)
> at com.sun.proxy.$Proxy0.cleanNotificationEvents(Unknown Source)
> at
> org.apache.hive.hcatalog.listener.DbNotificationListener$CleanerThread.run(DbNotificationListener.java:277)
> NestedThrowablesStackTrace:
> The class "org.datanucleus.store.rdbms.query.ForwardQueryResult" is not
> persistable. This means that it either hasnt been enhanced, or that the
> enhanced version of the file is not in the CLASSPATH (or is hidden by an
> unenhanced version), or the Meta-Data/annotations for the class are not found.
> org.datanucleus.exceptions.ClassNotPersistableException: The class
> "org.datanucleus.store.rdbms.query.ForwardQueryResult" is not persistable.
> This means that it either hasnt been enhanced, or that the enhanced version
> of the file is not in the CLASSPATH (or is hidden by an unenhanced version),
> or the Meta-Data/annotations for the class are not found.
> at
> org.datanucleus.ExecutionContextImpl.assertClassPersistable(ExecutionContextImpl.java:5698)
> at
> org.datanucleus.ExecutionContextImpl.deleteObjectInternal(ExecutionContextImpl.java:2495)
> at
> org.datanucleus.ExecutionContextImpl.deleteObjectWork(ExecutionContextImpl.java:2466)
> at
> org.datanucleus.ExecutionContextImpl.deleteObject(ExecutionContextImpl.java:2417)
> at
> org.datanucleus.ExecutionContextThreadedImpl.deleteObject(ExecutionContextThreadedImpl.java:245)
> at
> org.datanucleus.api.jdo.JDOPersistenceManager.jdoDeletePersistent(JDOPersistenceManager.java:802)
> at
> org.datanucleus.api.jdo.JDOPersistenceManager.deletePersistent(JDOPersistenceManager.java:820)
> at
> org.apache.hadoop.hive.metastore.ObjectStore.cleanNotificationEvents(ObjectStore.java:7149)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:114)
> at com.sun.proxy.$Proxy0.cleanNotificationEvents(Unknown Source)
> at
> org.apache.hive.hcatalog.listener.DbNotificationListener$CleanerThread.run(DbNotificationListener.java:277)
> {noformat}
> The end result of this bug is that users of DbNotificationListener will have
> an evergrowing number of notification events that are not cleaned up as they
> age. This is an easy enough fix, but shows that we have a lack of test
> coverage here.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
