Apache Ignite cannot access cache from listener after event “EVT_CACHE_STARTED” fired
I want to be able to monitor events of cache creation in Apache Ignite.
Whenever such events happen - I want to be able to do something with those
caches, after they are created, but before anyone else could inserts
something.
So I used local listener. Below is all the code:
@Configuration
public class ServerConfig {
public ServerConfig(Environment e) throws Exception {
IgniteConfiguration cfg = new IgniteConfiguration();
cfg.setIncludeEventTypes(EventType.EVT_CACHE_STARTED);
Ignite ignite = Ignition.start(cfg);
String cacheName = "test";
registerCacheCreationListener(ignite);
IgniteCache cache =
ignite.getOrCreateCache(cacheName);
}
private void registerCacheCreationListener(Ignite ignite){
IgnitePredicate locLsnr = new
IgnitePredicate(){
@IgniteInstanceResource
private Ignite ignite;
@Override
public boolean apply(CacheEvent evt) {
System.out.println("Received event [evt=" + evt.name() + "
cacheName=" + evt.cacheName());
IgniteCache cache =
ignite.cache(evt.cacheName()); // CANNOT ACCESS evt.cacheName() -
STUCKS HERE
System.out.println("finish listener");
return true;
}
};
ignite.events().localListen(locLsnr, EventType.EVT_CACHE_STARTED);
}
}
So when I do:
ignite.cache(evt.cacheName())
inside IgnitePredicate - it is not yet available as I understand.
Please help me find out where can I be wrong. Thanks.
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Apache Ignite cannot access cache from listener after event “EVT_CACHE_STARTED” fired
I want to be able to monitor events of cache creation in Apache Ignite.
Whenever such events happen - I want to be able to do something with those
caches, after they are created, but before anyone else could inserts
something.
So I used local listener. Below is all the code:
@Configuration
public class ServerConfig {
public ServerConfig(Environment e) throws Exception {
IgniteConfiguration cfg = new IgniteConfiguration();
cfg.setIncludeEventTypes(EventType.EVT_CACHE_STARTED);
Ignite ignite = Ignition.start(cfg);
String cacheName = "test";
registerCacheCreationListener(ignite);
IgniteCache cache =
ignite.getOrCreateCache(cacheName);
}
private void registerCacheCreationListener(Ignite ignite){
IgnitePredicate locLsnr = new
IgnitePredicate(){
@IgniteInstanceResource
private Ignite ignite;
@Override
public boolean apply(CacheEvent evt) {
System.out.println("Received event [evt=" + evt.name() + "
cacheName=" + evt.cacheName());
IgniteCache cache =
ignite.cache(evt.cacheName()); // CANNOT ACCESS evt.cacheName() -
STUCKS HERE
System.out.println("finish listener");
return true;
}
};
ignite.events().localListen(locLsnr, EventType.EVT_CACHE_STARTED);
}
}
So when I do:
ignite.cache(evt.cacheName())
inside IgnitePredicate - it is not yet available as I understand.
Please help me find out where can I be wrong. Thanks.
Re: Ways to improve re-balancing of partitions and how to monitor re-balance progress
Add to above I also see below, completed rebalancing and cancelled rebalancing logs after any client joins connects to the cluster. Is it normal behavior of re balance [2018-11-08T20:06:36,936][INFO ][disco-event-worker-#341%EDIFCustomer%][GridDiscoveryManager] ^-- Node [id=B5C2D0EE-72CB-49A6-848F-83DC7CBDCCD7, clusterState=ACTIVE] [2018-11-08T20:06:36,936][INFO ][disco-event-worker-#341%EDIFCustomer%][GridDiscoveryManager] ^-- Baseline [id=1, size=5, online=5, offline=0] [2018-11-08T20:06:36,936][INFO ][disco-event-worker-#341%EDIFCustomer%][GridDiscoveryManager] Data Regions Configured: [2018-11-08T20:06:36,936][INFO ][disco-event-worker-#341%EDIFCustomer%][GridDiscoveryManager] ^-- default [initSize=256.0 MiB, maxSize=1.5 TiB, persistenceEnabled=true] [2018-11-08T20:06:36,937][INFO ][exchange-worker-#342%EDIFCustomer%][time] Started exchange init [topVer=AffinityTopologyVersion [topVer=23, minorTopVer=0], crd=false, evt=NODE_JOINED, evtNode=c3394ec2-a71a-4108-b720-0b2a10501fcc, customEvt=null, allowMerge=true] [2018-11-08T20:06:36,939][INFO ][exchange-worker-#342%EDIFCustomer%][GridDhtPartitionsExchangeFuture] Finish exchange future [startVer=AffinityTopologyVersion [topVer=23, minorTopVer=0], resVer=AffinityTopologyVersion [topVer=23, minorTopVer=0], err=null] [2018-11-08T20:06:36,939][INFO ][exchange-worker-#342%EDIFCustomer%][time] Finished exchange init [topVer=AffinityTopologyVersion [topVer=23, minorTopVer=0], crd=false] [2018-11-08T20:06:36,949][INFO ][exchange-worker-#342%EDIFCustomer%][GridDhtPartitionDemander] Cancelled rebalancing from all nodes [topology=AffinityTopologyVersion [topVer=22, minorTopVer=1]] [2018-11-08T20:06:36,949][INFO ][exchange-worker-#342%EDIFCustomer%][GridDhtPartitionDemander] Completed rebalance future: RebalanceFuture [grp=CacheGroupContext [grp=MapServCustCache], topVer=AffinityTopologyVersion [topVer=22, minorTopVer=1], rebalanceId=1509] [2018-11-08T20:06:36,949][INFO ][exchange-worker-#342%EDIFCustomer%][GridDhtPartitionDemander] Cancelled rebalancing from all nodes [topology=AffinityTopologyVersion [topVer=22, minorTopVer=1]] -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Ways to improve re-balancing of partitions and how to monitor re-balance progress
Hi We are using Ignite 2.6 I have used rebalanceThreadPool to 16 and rebalanceBatchSize to 2 MB and we have total data around 12 TB with 6 nodes cluster. We are using very high end machines which are 128 CPU and 2 TB RAM etc, we are finding ways to get re-balancing to be completed fast, right now with the above configuration it takes very long time. We have each cache loaded with 500M records, taking more than 24 hours to re-balance single cache. Its more than 3 days rebalancing has started, still its not yet over. I only could completed (final) rebalancing log for only few caches around 4 to 5, rest of the 40 caches not completed. And, what I observed is, after removing a node from baseline and re-balancing of data has started, there is no increase in the disk utilization, I could only see RAM utilization has increased (from 200 GB to 500 GB). Does it mean that data gets re-balanced only in RAM and RAM updates not getting reflected on DISK ?? Is there any better ways to monitor progress of re-balancing activity ? And, also ways to speed up re-balance provided we have enough system resources Thanks Naveen -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Transformation and sort dataset
Hi I currently store Vectors in ignite. I need tout compute thé distance between a référence Vector and all the stored one and After return thé 10 smallest. If thé référence dataset is Big (thé one that store Vector) , let Say 1,000,000,000 Whats thé best way to do this like with spark on a stream itérative way? Thx
Re: How could imporeve UNION ALL performance
Hi, Can you share a query plan? Do you have any indices on last and lastprice columns? Have you tried to run queries separately without Union? чт, 8 нояб. 2018 г., 23:47 wengyao04 wengya...@gmail.com: > Hi we use ignite as our searching backend. > Our cache is in replicated mode and primary_sync. Our query is more like > the > following. We find that that the query time is the sum of two sub-select > query. Since our two sub select query is independent, is there a way to run > them in parallel ? Thanks > > SELECT market.id, market.name, market.lastPrice > FROM (( >SELECT marketR.id, marketR.name, marketR.lastPrice >FROM STOCK marketR >JOIN TABLE(exchange int = ?) ON i.exchange = marketR.exchange >WHERE marketR.subscription = 'TRUE' AND marketR.last > 50. >) >UNION ALL >( > SELECT marketD.id, marketD.name, marketD.lastPrice > FROM STOCK marketD > JOIN TABLE(exchange int = ?) ON i.exchange = marketD.exchange > WHERE marketD.subscription = 'FALSE' AND marketD.lastPrice > > 50. >)) > market ORDER BY market.lastPrice DESC LIMIT 5000 > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
