I'm currently trying to integrate ignite client within apache flink
taskmanagers(jvm) to fast lookup cache. Although my thoughput without ignite
is about 30k/s when i add ignite it drops to 2k/s.
- Cache configuration was set with defaults expect for off-heap.
- I have 3 different caches with one records each (testing purposes).
- I've defined also near cache for clients.
- Both flink and ignite ara managed by yarn.
Is there any better and fastest way to access data? Am I doing something
wrong with the following configurations?
Thanks
------------------ Data access in client
this.nearCache.get(key);
------------------ Cache configuration
CacheConfiguration<String, Object[]> cacheConfig = new
CacheConfiguration<>();
cacheConfig.setName(table.getName());
cacheConfig.setCopyOnRead(false);
cacheConfig.setMemoryMode(CacheMemoryMode.OFFHEAP_TIERED);
cacheConfig.setOffHeapMaxMemory(0L);
------------------ Loading from client
IgniteDataStreamer<String, Object[]> streamer =
ignite.dataStreamer(cacheConfig.getName())
------------------ Near cache configuration
NearCacheConfiguration<String, Object[]> nearCacheConfig = new
NearCacheConfiguration<>();
nearCacheConfig.setNearEvictionPolicy(new LruEvictionPolicy(10000));
IgniteCache<String, Object[]> nearCache =
ignite.getOrCreateNearCache("near-" + cacheConfig.getName(),
nearCacheConfig);
------------------ Ignite configuration
<bean class="org.apache.ignite.configuration.IgniteConfiguration">
<property name="discoverySpi">
<bean
class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
<property name="ipFinder">
<bean
class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
<property name="addresses">
<list>
<value>xx.xxx.xxx.111:47500..47510</value>
<value>xx.xxx.xxx.112:47500..47510</value>
</list>
</property>
</bean>
</property>
<property name="joinTimeout" value="60000"/>
</bean>
</property>
<property name="communicationSpi">
<bean
class="org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi">
<property name="sharedMemoryPort" value="-1"/>
</bean>
</property>
</bean>
--
View this message in context:
http://apache-ignite-users.70518.x6.nabble.com/Client-near-cache-with-Apache-Flink-tp11627.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
