[
https://issues.apache.org/jira/browse/IMPALA-11260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17552417#comment-17552417
]
Vincent Tran commented on IMPALA-11260:
---------------------------------------
!image-2022-06-09-16-17-39-445.png!
In the same way that this can hang *ImpalaServer::Start()*, this can also hang
queries in *ImpalaServer::ExecuteInternal()*, since the call over JNI never
returns because the Java threads in
*org.apache.impala.service.JniFrontend.getCatalogMetrics()* are stuck here
waiting for a *Future* that will never complete:
{noformat}
"Thread-17" #59 prio=5 os_prio=0 cpu=1404.89ms elapsed=5312.17s
tid=0x000000000acee000 nid=0xae02 waiting on condition [0x00007f6443484000]
java.lang.Thread.State: WAITING (parking)
at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
- parking to wait for <0x00007f6faf520b78> (a
java.util.concurrent.CompletableFuture$Signaller)
at
java.util.concurrent.locks.LockSupport.park([email protected]/LockSupport.java:194)
at
java.util.concurrent.CompletableFuture$Signaller.block([email protected]/CompletableFuture.java:1796)
at
java.util.concurrent.ForkJoinPool.managedBlock([email protected]/ForkJoinPool.java:3128)
at
java.util.concurrent.CompletableFuture.waitingGet([email protected]/CompletableFuture.java:1823)
at
java.util.concurrent.CompletableFuture.get([email protected]/CompletableFuture.java:1998)
at
com.google.common.util.concurrent.Uninterruptibles.getUninterruptibly(Uninterruptibles.java:237)
at
org.apache.impala.catalog.local.CatalogdMetaProvider.loadWithCaching(CatalogdMetaProvider.java:509)
at
org.apache.impala.catalog.local.CatalogdMetaProvider.loadTableList(CatalogdMetaProvider.java:649)
at
org.apache.impala.catalog.local.LocalDb.loadTableNames(LocalDb.java:170)
at
org.apache.impala.catalog.local.LocalDb.getAllTableNames(LocalDb.java:155)
at
org.apache.impala.service.Frontend.getCatalogMetrics(Frontend.java:779)
at
org.apache.impala.service.JniFrontend.getCatalogMetrics(JniFrontend.java:221)
Locked ownable synchronizers:
- None
{noformat}
> Catalog cache item sizes of CatalogdMetaProvider are underestimated on Java9+
> -----------------------------------------------------------------------------
>
> Key: IMPALA-11260
> URL: https://issues.apache.org/jira/browse/IMPALA-11260
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
> Affects Versions: Impala 3.1.0, Impala 3.2.0, Impala 4.0.0, Impala 3.3.0,
> Impala 3.4.0, Impala 3.4.1
> Reporter: Quanlong Huang
> Priority: Critical
> Attachments: image-2022-06-09-16-17-39-445.png
>
>
> When running local catalog mode on Java11, the Ehcache sizeof lib complains
> that cache sizes may be underestimated:
> {code:java}
> W0421 20:50:44.238312 9819 ObjectGraphWalker.java:251]
> 744e548159a57cb5:879ee74c00000000] The JVM is preventing Ehcache from
> accessing the subgraph beneath 'final jdk.internal.loader.URLClassPath
> jdk.internal.loader.ClassLoaders$AppClassLoader.ucp' - cache sizes may be
> underestimated as a result
> Java exception follows:
> java.lang.reflect.InaccessibleObjectException: Unable to make field final
> jdk.internal.loader.URLClassPath
> jdk.internal.loader.ClassLoaders$AppClassLoader.ucp accessible: module
> java.base does not "opens jdk.internal.loader" to unnamed module @6ba7383d
> at
> java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:340)
> at
> java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:280)
> at
> java.base/java.lang.reflect.Field.checkCanSetAccessible(Field.java:176)
> at java.base/java.lang.reflect.Field.setAccessible(Field.java:170)
> at
> org.ehcache.sizeof.ObjectGraphWalker.getAllFields(ObjectGraphWalker.java:245)
> at
> org.ehcache.sizeof.ObjectGraphWalker.getFilteredFields(ObjectGraphWalker.java:204)
> at
> org.ehcache.sizeof.ObjectGraphWalker.walk(ObjectGraphWalker.java:159)
> at org.ehcache.sizeof.SizeOf.deepSizeOf(SizeOf.java:74)
> at
> org.apache.impala.catalog.local.CatalogdMetaProvider$SizeOfWeigher.weigh(CatalogdMetaProvider.java:1999)
> at
> com.google.common.cache.LocalCache$Segment.setValue(LocalCache.java:2010)
> at
> com.google.common.cache.LocalCache$Segment.replace(LocalCache.java:2956)
> at com.google.common.cache.LocalCache.replace(LocalCache.java:4258)
> at
> org.apache.impala.catalog.local.CatalogdMetaProvider.loadWithCaching(CatalogdMetaProvider.java:540)
> at
> org.apache.impala.catalog.local.CatalogdMetaProvider.loadIcebergApiTable(CatalogdMetaProvider.java:1056)
> at
> org.apache.impala.catalog.local.LocalIcebergTable.loadIcebergTableViaMetaProvider(LocalIcebergTable.java:87)
> at
> org.apache.impala.catalog.local.LocalTable.load(LocalTable.java:107)
> at org.apache.impala.catalog.local.LocalDb.getTable(LocalDb.java:127)
> at
> org.apache.impala.analysis.StmtMetadataLoader.getMissingTables(StmtMetadataLoader.java:310)
> at
> org.apache.impala.analysis.StmtMetadataLoader.loadTables(StmtMetadataLoader.java:165)
> at
> org.apache.impala.analysis.StmtMetadataLoader.loadTables(StmtMetadataLoader.java:141)
> at
> org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:2014)
> at
> org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:1926)
> at
> org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1750)
> at
> org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:164){code}
> Similar errors on other classes:
> {code}
> The JVM is preventing Ehcache from accessing the subgraph beneath 'final
> jdk.internal.loader.AbstractClassLoaderValue
> jdk.internal.loader.AbstractClassLoaderValue$Sub.this$0' - cache sizes may be
> underestimated as a result
> The JVM is preventing Ehcache from accessing the subgraph beneath 'final
> jdk.internal.loader.URLClassPath
> jdk.internal.loader.ClassLoaders$AppClassLoader.ucp' - cache sizes may be
> underestimated as a result
> The JVM is preventing Ehcache from accessing the subgraph beneath 'private
> final java.lang.Object jdk.internal.loader.AbstractClassLoaderValue$Sub.key'
> - cache sizes may be underestimated as a result
> The JVM is preventing Ehcache from accessing the subgraph beneath 'private
> final java.lang.String java.lang.module.Configuration.targetPlatform' - cache
> sizes may be underestimated as a result
> The JVM is preventing Ehcache from accessing the subgraph beneath 'private
> final java.lang.String java.lang.module.ModuleDescriptor.mainClass' - cache
> sizes may be underestimated as a result
> The JVM is preventing Ehcache from accessing the subgraph beneath 'private
> final java.lang.String java.lang.module.ModuleDescriptor.name' - cache sizes
> may be underestimated as a result
> The JVM is preventing Ehcache from accessing the subgraph beneath 'private
> final java.lang.String java.lang.module.ModuleDescriptor.rawVersionString' -
> cache sizes may be underestimated as a result
> The JVM is preventing Ehcache from accessing the subgraph beneath 'private
> final java.lang.module.ModuleDescriptor$Version
> java.lang.module.ModuleDescriptor.version' - cache sizes may be
> underestimated as a result
> The JVM is preventing Ehcache from accessing the subgraph beneath 'private
> final java.util.List java.lang.module.Configuration.parents' - cache sizes
> may be underestimated as a result
> The JVM is preventing Ehcache from accessing the subgraph beneath 'private
> final java.util.Map java.lang.module.Configuration.graph' - cache sizes may
> be underestimated as a result
> The JVM is preventing Ehcache from accessing the subgraph beneath 'private
> final java.util.Map java.lang.module.Configuration.nameToModule' - cache
> sizes may be underestimated as a result
> The JVM is preventing Ehcache from accessing the subgraph beneath 'private
> final java.util.Map jdk.internal.loader.BuiltinClassLoader.moduleToReader' -
> cache sizes may be underestimated as a result
> The JVM is preventing Ehcache from accessing the subgraph beneath 'private
> final java.util.Map jdk.internal.loader.BuiltinClassLoader.nameToModule' -
> cache sizes may be underestimated as a result
> The JVM is preventing Ehcache from accessing the subgraph beneath 'private
> final java.util.Map jdk.internal.module.ServicesCatalog.map' - cache sizes
> may be underestimated as a result
> The JVM is preventing Ehcache from accessing the subgraph beneath 'private
> final java.util.Set java.lang.module.Configuration.modules' - cache sizes may
> be underestimated as a result
> The JVM is preventing Ehcache from accessing the subgraph beneath 'private
> final java.util.Set java.lang.module.ModuleDescriptor.exports' - cache sizes
> may be underestimated as a result
> The JVM is preventing Ehcache from accessing the subgraph beneath 'private
> final java.util.Set java.lang.module.ModuleDescriptor.modifiers' - cache
> sizes may be underestimated as a result
> The JVM is preventing Ehcache from accessing the subgraph beneath 'private
> final java.util.Set java.lang.module.ModuleDescriptor.opens' - cache sizes
> may be underestimated as a result
> The JVM is preventing Ehcache from accessing the subgraph beneath 'private
> final java.util.Set java.lang.module.ModuleDescriptor.packages' - cache sizes
> may be underestimated as a result
> The JVM is preventing Ehcache from accessing the subgraph beneath 'private
> final java.util.Set java.lang.module.ModuleDescriptor.provides' - cache sizes
> may be underestimated as a result
> The JVM is preventing Ehcache from accessing the subgraph beneath 'private
> final java.util.Set java.lang.module.ModuleDescriptor.requires' - cache sizes
> may be underestimated as a result
> The JVM is preventing Ehcache from accessing the subgraph beneath 'private
> final java.util.Set java.lang.module.ModuleDescriptor.uses' - cache sizes may
> be underestimated as a result
> The JVM is preventing Ehcache from accessing the subgraph beneath 'private
> final jdk.internal.loader.BuiltinClassLoader
> jdk.internal.loader.BuiltinClassLoader.parent' - cache sizes may be
> underestimated as a result
> The JVM is preventing Ehcache from accessing the subgraph beneath 'private
> final jdk.internal.loader.URLClassPath
> jdk.internal.loader.BuiltinClassLoader.ucp' - cache sizes may be
> underestimated as a result
> The JVM is preventing Ehcache from accessing the subgraph beneath 'private
> jdk.internal.reflect.ConstructorAccessorImpl
> jdk.internal.reflect.DelegatingConstructorAccessorImpl.delegate' - cache
> sizes may be underestimated as a result
> The JVM is preventing Ehcache from accessing the subgraph beneath 'private
> volatile java.lang.ref.SoftReference
> jdk.internal.loader.BuiltinClassLoader.resourceCache' - cache sizes may be
> underestimated as a result
> The JVM is preventing Ehcache from accessing the subgraph beneath 'private
> volatile java.util.List java.lang.module.Configuration.allConfigurations' -
> cache sizes may be underestimated as a result
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
