You don't need to handle it, as it mentioned in exception message - "DataStreamer will retry data transfer at stable topology ". It will retry right after topology will be updated. If you don't have any other exceptions - data loading finished successfully
2017-04-21 15:54 GMT+03:00 Alper Tekinalp <[email protected]>: > Hi, > > So how should I handle that exception? Should I just ignore it? Is > there a way I can know when the data streamer successfully deliver > after getting that error? I just want to know if data > loading/rebalancing finished succesfully. > > On Fri, Apr 21, 2017 at 2:10 PM, Evgenii Zhuravlev > <[email protected]> wrote: > > Hi, > > > > Minor topology version changes when you creating or removing caches and > at > > late affinity. > > > > Yes, you see this error in log because minor topoly was changed while > > datastreamer was running, but you didn't lose any data. > > > > According to javadoc: > > > >>Note that there is no guarantee that data will be delivered after this > >> concrete attempt (e.g., it can fail when topology is changing), but it > won't > >> be lost anyway. > > > > Evgenii > > > > > > 2017-04-21 13:24 GMT+03:00 Alper Tekinalp <[email protected]>: > >> > >> Hi. > >> > >> First of all what is minor topology version and when does it chage? > >> > >> Below we got an error while loading data with data streamer: > >> > >> 18/Apr/2017 13:50:45 INFO 34391455 [exchange-worker-#31%null%] > >> org.apache.ignite.internal.processors.cache.GridCacheProcessor(L:475) - > >> Started cache [name=RECORD_CACHE_XX, mode=PARTITIONED] > >> .. > >> 18/Apr/2017 13:50:49 ERROR 34394884 [DeploymentWorker-0] > >> com.intellica.evam.engine.dynamic.helpers.DeploymentHelper(L:359) - > Scenario > >> data could not be preloaded in distributed deployment for scenario [XX]. > >> javax.cache.CacheException: class > >> org.apache.ignite.IgniteCheckedException: Failed to finish operation > (too > >> many remaps): 32 > >> at > >> org.apache.ignite.internal.processors.cache.GridCacheUtils. > convertToCacheException(GridCacheUtils.java:1465) > >> at > >> org.apache.ignite.internal.processors.datastreamer. > DataStreamerImpl.close(DataStreamerImpl.java:1160) > >> at > >> org.apache.ignite.internal.processors.datastreamer. > DataStreamerImpl.close(DataStreamerImpl.java:1244) > >> at > >> com.intellica.evam.engine.db.dao.ScenarioRecordDao. > preloadScenarioRecords(ScenarioRecordDao.java:134) > >> at > >> com.intellica.evam.engine.dynamic.helpers.cache.PreloadHelper. > preloadScenarioRecordData(PreloadHelper.java:58) > >> at > >> com.intellica.evam.engine.dynamic.helpers.cache.PreloadHelper. > preloadScenarioData(PreloadHelper.java:24) > >> at > >> com.intellica.evam.engine.dynamic.helpers.DeploymentHelper. > distributedDeploy(DeploymentHelper.java:827) > >> at > >> com.intellica.evam.engine.dynamic.DeploymentWorker. > handleDeploymentEvent(DeploymentWorker.java:83) > >> at > >> com.intellica.evam.engine.dynamic.DeploymentWorker.run( > DeploymentWorker.java:62) > >> Caused by: class org.apache.ignite.IgniteCheckedException: Failed to > >> finish operation (too many remaps): 32 > >> at > >> org.apache.ignite.internal.processors.datastreamer. > DataStreamerImpl$5.apply(DataStreamerImpl.java:863) > >> at > >> org.apache.ignite.internal.processors.datastreamer. > DataStreamerImpl$5.apply(DataStreamerImpl.java:828) > >> at > >> org.apache.ignite.internal.util.future.GridFutureAdapter$ > ArrayListener.apply(GridFutureAdapter.java:456) > >> at > >> org.apache.ignite.internal.util.future.GridFutureAdapter$ > ArrayListener.apply(GridFutureAdapter.java:439) > >> at > >> org.apache.ignite.internal.util.future.GridFutureAdapter. > notifyListener(GridFutureAdapter.java:271) > >> at > >> org.apache.ignite.internal.util.future.GridFutureAdapter. > notifyListeners(GridFutureAdapter.java:259) > >> at > >> org.apache.ignite.internal.util.future.GridFutureAdapter. > onDone(GridFutureAdapter.java:389) > >> at > >> org.apache.ignite.internal.util.future.GridFutureAdapter. > onDone(GridFutureAdapter.java:355) > >> at > >> org.apache.ignite.internal.processors.datastreamer. > DataStreamerImpl$Buffer.onResponse(DataStreamerImpl.java:1789) > >> at > >> org.apache.ignite.internal.processors.datastreamer. > DataStreamerImpl$3.onMessage(DataStreamerImpl.java:335) > >> at > >> org.apache.ignite.internal.managers.communication. > GridIoManager.invokeListener(GridIoManager.java:1215) > >> at > >> org.apache.ignite.internal.managers.communication.GridIoManager. > processRegularMessage0(GridIoManager.java:843) > >> at > >> org.apache.ignite.internal.managers.communication. > GridIoManager.access$2100(GridIoManager.java:108) > >> at > >> org.apache.ignite.internal.managers.communication.GridIoManager$6.run( > GridIoManager.java:783) > >> at > >> java.util.concurrent.ThreadPoolExecutor.runWorker( > ThreadPoolExecutor.java:1145) > >> at > >> java.util.concurrent.ThreadPoolExecutor$Worker.run( > ThreadPoolExecutor.java:615) > >> at java.lang.Thread.run(Thread.java:745) > >> Caused by: class org.apache.ignite.IgniteCheckedException: DataStreamer > >> request failed [node=6fe302a3-f353-4235-af5a-3708bf240750] > >> at > >> org.apache.ignite.internal.processors.datastreamer. > DataStreamerImpl$Buffer.onResponse(DataStreamerImpl.java:1777) > >> ... 8 more > >> Caused by: class org.apache.ignite.IgniteCheckedException: DataStreamer > >> will retry data transfer at stable topology [reqTop= > AffinityTopologyVersion > >> [topVer=11, minorTopVer=14], topVer=AffinityTopologyVersion [topVer=11, > >> minorTopVer=15], node=remote] > >> at > >> org.apache.ignite.internal.processors.datastreamer.DataStreamProcessor. > localUpdate(DataStreamProcessor.java:339) > >> at > >> org.apache.ignite.internal.processors.datastreamer.DataStreamProcessor. > processRequest(DataStreamProcessor.java:297) > >> at > >> org.apache.ignite.internal.processors.datastreamer. > DataStreamProcessor.access$000(DataStreamProcessor.java:56) > >> at > >> org.apache.ignite.internal.processors.datastreamer. > DataStreamProcessor$1.onMessage(DataStreamProcessor.java:86) > >> ... 7 more > >> 18/Apr/2017 13:50:49 INFO 34394950 [exchange-worker-#31%null%] > >> org.apache.ignite.internal.processors.cache.GridCacheProcessor(L:475) - > >> Started cache [name=FEC_XX, mode=PARTITIONED] > >> > >> > >> I guess the reason seems that when one of the server nodes loads data > >> through data streamer to cache RECORD_CACHE_XX, other server creates > cache > >> FEC_XX. Bacause cache creation increases the minor topology version. > >> > >> Is my understanding true? > >> If it is what is the reason? > >> > >> -- > >> Alper Tekinalp > >> > >> Software Developer > >> Evam Streaming Analytics > >> > >> Atatürk Mah. Turgut Özal Bulv. > >> Gardenya 5 Plaza K:6 Ataşehir > >> 34758 İSTANBUL > >> > >> Tel: +90 216 455 01 53 Fax: +90 216 455 01 54 > >> www.evam.com.tr > > > > > > > > -- > Alper Tekinalp > > Software Developer > Evam Streaming Analytics > > Atatürk Mah. Turgut Özal Bulv. > Gardenya 5 Plaza K:6 Ataşehir > 34758 İSTANBUL > > Tel: +90 216 455 01 53 Fax: +90 216 455 01 54 > www.evam.com.tr >
