[
https://issues.apache.org/jira/browse/IGNITE-993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602897#comment-14602897
]
Semen Boikov commented on IGNITE-993:
-------------------------------------
In current implementation information about caches
(GridCacheProcessor.registeredCaches) is accessed from multiple threads:
- new descriptor is put from tcp-disco-msg-worker
(GridCacheProcessor.onCacheChangeRequested)
- then to check if exchange is needed 'registeredCaches' is accessed
from disco-event-worker (GridCacheProcessor.dynamicCacheRegistered)
- for stopped cache descriptor is removed on exchange end from system
pool thread
Start/stop logic can be easily broken for concurrent star/stop requests for
cache with the same name. To fix this issue need to refactor cache code to
update/access 'registeredCaches' only from discovery thread. Information
required for exchange (see ogic in GridCacheProcessor.dynamicCacheRegistered)
should be computed in discovery thread and can be passed to exchange as special
transient field in DynamicCacheChangeRequest.
> IllegalArgumentException if two nodes create and drop caches independently
> ---------------------------------------------------------------------------
>
> Key: IGNITE-993
> URL: https://issues.apache.org/jira/browse/IGNITE-993
> Project: Ignite
> Issue Type: Bug
> Components: cache
> Affects Versions: sprint-5
> Reporter: Sergey Kozlov
> Assignee: Semen Boikov
> Fix For: sprint-7
>
> Attachments: CacheStreamerStartStopCacheTest.java, log.zip
>
>
> Note: use attached file
> The logic of test:
> Start two nodes and each node does in loop following: create cache, load 10k
> keys via streamer, drop cache.
> Steps:
> 1. Start 1st node by CacheStreamerStartStopCache
> 2. Wait "Iteration 100" message in output
> 3. Start 2nd node by CacheStreamerStartStopCache
> 1st node output:
> {noformat}
> [15:58:33,395][INFO ][exchange-worker-#49%null%][GridDhtPreloader]
> <server_cache_1209_884> Starting rebalancing in ASYNC mode:
> server_cache_1209_884
> [15:58:33,396][INFO ][preloader-demand-worker-#824%null%][GridDhtPreloader]
> <server_cache_1209_884> Completed rebalancing in ASYNC mode
> [cache=server_cache_1209_884, time=0 ms]
> [15:58:33,433][INFO ][disco-event-worker-#46%null%][GridDiscoveryManager]
> Added new node to topology: TcpDiscoveryNode
> [id=407e2385-405d-4ce1-85c1-2641fd81b217, addrs=[0:0:0:0:0:0:0:1, 127.0.0.1,
> 10.0.0.5, 192.168.56.1, 2001:db8:85a3:0:0:8a2e:370:7334],
> sockAddrs=[/0:0:0:0:0:0:0:1:47501, /127.0.0.1:47501, rr/192.168.56.1:47501,
> /10.0.0.5:47501, rr/192.168.56.1:47501, /192.168.56.1:47501,
> rr/192.168.56.1:47501, /2001:db8:85a3:0:0:8a2e:370:7334:47501],
> discPort=47501, order=2, intOrder=2, loc=false,
> ver=1.1.1#20150604-sha1:a6ea325c, isClient=false]
> [15:58:33,434][INFO ][disco-event-worker-#46%null%][GridDiscoveryManager]
> Topology snapshot [ver=2, nodes=2, CPUs=8, heap=2.0GB]
> [15:58:33] (err) Failed to execute compound future reducer: Compound future
> listener: GridCompoundFuture [lsnrCalls=0, finished=false, rdc=null,
> init=true, res=java.util.concurrent.atomic.AtomicMarkableReference@78237daf,
> err=null, done=false, cancelled=false, err=null, futs=[true]]class
> org.apache.ignite.IgniteCheckedException: Cache is not configured:
> server_cache_1209_884
> at
> org.apache.ignite.internal.util.IgniteUtils.cast(IgniteUtils.java:6732)
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:124)
> at
> org.apache.ignite.internal.util.future.GridCompoundFuture$Listener.apply(GridCompoundFuture.java:304)
> at
> org.apache.ignite.internal.util.future.GridCompoundFuture$Listener.apply(GridCompoundFuture.java:295)
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter$ArrayListener.apply(GridFutureAdapter.java:419)
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter$ArrayListener.apply(GridFutureAdapter.java:402)
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:241)
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListeners(GridFutureAdapter.java:229)
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:359)
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:325)
> at
> org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$Buffer.onResponse(DataStreamerImpl.java:1256)
> at
> org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$3.onMessage(DataStreamerImpl.java:235)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:690)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.access$1500(GridIoManager.java:58)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager$5.run(GridIoManager.java:653)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.IllegalArgumentException: Cache is not configured:
> server_cache_1209_884
> at
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.jcache(GridCacheProcessor.java:2715)
> at
> org.apache.ignite.internal.processors.datastreamer.DataStreamerUpdateJob.call(DataStreamerUpdateJob.java:91)
> at
> org.apache.ignite.internal.processors.datastreamer.DataStreamProcessor.processRequest(DataStreamProcessor.java:280)
> at
> org.apache.ignite.internal.processors.datastreamer.DataStreamProcessor.access$000(DataStreamProcessor.java:44)
> at
> org.apache.ignite.internal.processors.datastreamer.DataStreamProcessor$4$1.run(DataStreamProcessor.java:204)
> at
> org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6180)
> at
> org.apache.ignite.internal.processors.closure.GridClosureProcessor$1.body(GridClosureProcessor.java:752)
> at
> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:108)
> ... 3 more
> {noformat}
> 2nd node output:
> {noformat}
> [15:58:33,945][INFO ][main][GridDhtPreloader] <ignite-sys-cache> Starting
> rebalancing in SYNC mode: ignite-sys-cache
> [15:58:33,946][INFO ][main][GridDhtPreloader] <ignite-atomics-sys-cache>
> Starting rebalancing in SYNC mode: ignite-atomics-sys-cache
> [15:58:33,946][INFO ][main][GridDhtPreloader] <server_cache_1209_773>
> Starting rebalancing in ASYNC mode: server_cache_1209_773
> [15:58:33,946][INFO ][main][GridDhtPreloader] <ignite-marshaller-sys-cache>
> Starting rebalancing in SYNC mode: ignite-marshaller-sys-cache
> [15:58:33,952][ERROR][ignite-#9%pub-null%][DataStreamProcessor] Failed to
> finish update job.
> java.lang.IllegalArgumentException: Cache is not configured:
> server_cache_1209_884
> at
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.jcache(GridCacheProcessor.java:2715)
> at
> org.apache.ignite.internal.processors.datastreamer.DataStreamerUpdateJob.call(DataStreamerUpdateJob.java:91)
> at
> org.apache.ignite.internal.processors.datastreamer.DataStreamProcessor.processRequest(DataStreamProcessor.java:280)
> at
> org.apache.ignite.internal.processors.datastreamer.DataStreamProcessor.access$000(DataStreamProcessor.java:44)
> at
> org.apache.ignite.internal.processors.datastreamer.DataStreamProcessor$4$1.run(DataStreamProcessor.java:204)
> at
> org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6180)
> at
> org.apache.ignite.internal.processors.closure.GridClosureProcessor$1.body(GridClosureProcessor.java:752)
> at
> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:108)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> [15:58:33,990][ERROR][ignite-#10%pub-null%][DataStreamProcessor] Failed to
> finish update job.
> java.lang.IllegalArgumentException: Cache is not configured:
> server_cache_1209_884
> at
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.jcache(GridCacheProcessor.java:2715)
> at
> org.apache.ignite.internal.processors.datastreamer.DataStreamerUpdateJob.call(DataStreamerUpdateJob.java:91)
> at
> org.apache.ignite.internal.processors.datastreamer.DataStreamProcessor.processRequest(DataStreamProcessor.java:280)
> at
> org.apache.ignite.internal.processors.datastreamer.DataStreamProcessor.access$000(DataStreamProcessor.java:44)
> at
> org.apache.ignite.internal.processors.datastreamer.DataStreamProcessor$1.onMessage(DataStreamProcessor.java:70)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:690)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.access$1500(GridIoManager.java:58)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager$5.run(GridIoManager.java:653)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> [15:58:34,012][INFO ][preloader-demand-worker-#67%null%][GridDhtPreloader]
> <ignite-marshaller-sys-cache> Completed rebalancing in SYNC mode
> [cache=ignite-marshaller-sys-cache, time=70 ms]
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
