Zhihua Deng created HIVE-28604:
----------------------------------
Summary: Allow HMS to configure the DataNucleus level 1 cache
Key: HIVE-28604
URL: https://issues.apache.org/jira/browse/HIVE-28604
Project: Hive
Issue Type: Improvement
Security Level: Public (Viewable by anyone)
Reporter: Zhihua Deng
Assignee: Zhihua Deng
Under heavy memory load, the HMS could see some opaque NPE, like:
{noformat}
ERROR org.apache.hadoop.hive.metastore.RetryingHMSHandler: [TThreadPoolServer
WorkerProcess-196]: MetaException(message:java.lang.NullPointerException)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newMetaException(
HiveMetaStore.java:8883)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.throwMetaException(
HiveMetaStore.java:10267)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.
drop_table_with_environment_context(HiveMetaStore.java:3566){noformat}
{noformat}
EC.preCommit L1Cache op IS NULL!
{noformat}
In current DataNucleus, the default L1 cache is SoftRefCache, it doesn't allow
null key or op(value) by design. However the op(value) type in SoftRefCache is
a SoftReference, means the entry can be reclaimed in case of GC, which could to
result to such the puzzling problem.
Add datanucleus.cache.level1.type in MetastoreConf as a workaround, so the HMS
can configure the type of L1 cache.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
