[ https://issues.apache.org/jira/browse/FLINK-14985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16986771#comment-16986771 ]
Gary Yao commented on FLINK-14985: ---------------------------------- [~Rockey Cui] I am not able to reproduce the issue. I tried submitting your job on AWS emr-5.28 with Hadoop 2.8.5, and it actually finished. Judging by the stacktrace, the ehcache ServiceLocator is able to find Store Providers but is not able to resolve their dependencies [1]. You can try to remote debug a task manager to see what happens inside the ehcache library [2]. You can also try with a different Hadoop distribution (Note that we do not run tests against Hadoop 3.x at the moment). [1] https://github.com/ehcache/ehcache3/blob/17576a1c3b41033ff01031623a6b3cf7642a3940/core/src/main/java/org/ehcache/core/spi/ServiceLocator.java#L401 [2] https://cwiki.apache.org/confluence/display/FLINK/Remote+Debugging+of+Flink+Clusters > FLNIK YARN per job with ehcache Exception > ----------------------------------------- > > Key: FLINK-14985 > URL: https://issues.apache.org/jira/browse/FLINK-14985 > Project: Flink > Issue Type: Bug > Components: Deployment / YARN > Affects Versions: 1.9.1 > Environment: openjdk-1.8.0.102-4.b14.el7.x86_64 > flink1.9.1 > Hadoop 3.0.0-cdh6.2.0 > Reporter: Rockey Cui > Priority: Major > Attachments: flink-ehcache-test.zip > > > I used ehcache in my project,I get the following exception when using per job > mode: > ---- > java.lang.IllegalStateException: UserManagedCacheBuilder failed to > build.java.lang.IllegalStateException: UserManagedCacheBuilder failed to > build. at > org.ehcache.config.builders.UserManagedCacheBuilder.build(UserManagedCacheBuilder.java:182) > at > org.ehcache.config.builders.UserManagedCacheBuilder.build(UserManagedCacheBuilder.java:404) > at lookupabletable.cache.AllCache.init(AllCache.java:46) at > lookupabletable.function.AbstractTableFunction.initCache(AbstractTableFunction.java:83) > at > lookupabletable.function.AbstractTableFunction.open(AbstractTableFunction.java:65) > at LookupFunction$7.open(Unknown Source) at > org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:36) > at > org.apache.flink.table.runtime.operators.join.lookup.LookupJoinRunner.open(LookupJoinRunner.java:68) > at > org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:36) > at > org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.open(AbstractUdfStreamOperator.java:102) > at > org.apache.flink.streaming.api.operators.ProcessOperator.open(ProcessOperator.java:56) > at > org.apache.flink.streaming.runtime.tasks.StreamTask.openAllOperators(StreamTask.java:532) > at > org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:396) > at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:705) at > org.apache.flink.runtime.taskmanager.Task.run(Task.java:530) at > java.lang.Thread.run(Thread.java:748)Caused by: > java.lang.IllegalStateException: > org.ehcache.core.spi.ServiceLocator$DependencyException: Failed to find > provider with satisfied dependency set for interface > org.ehcache.core.spi.store.Store$Provider [candidates > [org.ehcache.impl.internal.store.heap.OnHeapStoreProviderFactory@5a58dd22, > org.ehcache.impl.internal.store.offheap.OffHeapStoreProviderFactory@4d766540, > org.ehcache.impl.internal.store.disk.OffHeapDiskStoreProviderFactory@784d016d, > org.ehcache.impl.internal.store.tiering.TieredStoreProviderFactory@25b63026, > org.ehcache.impl.internal.store.loaderwriter.LoaderWriterStoreProviderFactory@776ed22b]] > at > org.ehcache.core.spi.ServiceLocator$DependencySet.build(ServiceLocator.java:350) > at > org.ehcache.config.builders.UserManagedCacheBuilder.build(UserManagedCacheBuilder.java:179) > ... 15 moreCaused by: > org.ehcache.core.spi.ServiceLocator$DependencyException: Failed to find > provider with satisfied dependency set for interface > org.ehcache.core.spi.store.Store$Provider [candidates > [org.ehcache.impl.internal.store.heap.OnHeapStoreProviderFactory@5a58dd22, > org.ehcache.impl.internal.store.offheap.OffHeapStoreProviderFactory@4d766540, > org.ehcache.impl.internal.store.disk.OffHeapDiskStoreProviderFactory@784d016d, > org.ehcache.impl.internal.store.tiering.TieredStoreProviderFactory@25b63026, > org.ehcache.impl.internal.store.loaderwriter.LoaderWriterStoreProviderFactory@776ed22b]] > at > org.ehcache.core.spi.ServiceLocator$DependencySet.lookupService(ServiceLocator.java:401) > at > org.ehcache.core.spi.ServiceLocator$DependencySet.build(ServiceLocator.java:322) > ... 16 more > ---- > but No problem in standalone mode. > Any help ? > > My code--> > public class AllCache extends AbstractCache<String, Row> { > private static final long serialVersionUID = -3639314692125067324L; > private PersistentUserManagedCache<String, Row> cache; > private String tempDir; > private LocalPersistenceService persistenceService; > public AllCache(CacheInfo cacheInfo) { > super(cacheInfo); > } > @Override > public void init() { > tempDir = System.getProperty("java.io.tmpdir") + File.separator > + cacheInfo.getCacheTmpDir() + File.separator > + UUID.randomUUID().toString() + File.separator > + Thread.currentThread().getId(); > persistenceService = new DefaultLocalPersistenceService( > new DefaultPersistenceConfiguration(new File(tempDir))); > cache = UserManagedCacheBuilder.newUserManagedCacheBuilder(String.class, > Row.class) > .with(new UserManagedPersistenceContext<>("persistentCache", > persistenceService)) > .withResourcePools(ResourcePoolsBuilder.newResourcePoolsBuilder() > .heap(cacheInfo.getCacheMaxMb(), MemoryUnit.MB) > .disk(10, MemoryUnit.GB, false) > ).build(true); > } > @Override > public Row get(String key) { > return cache.get(key); > } > @Override > public void put(String key, Row value) { > cache.put(key, value); > } > @Override > public void putAll(Map<String, Row> data) { > cache.putAll(data); > } > @Override > public void close() { > try { > cache.close(); > cache.destroy(); > persistenceService.stop(); > } catch (Exception ignore) { > } > } > } -- This message was sent by Atlassian Jira (v8.3.4#803005)