Hi VM, This sounds like a good use case for Compute Grid [1]. It will allow you to do parallel processing with proper failover and load balancing.
The issue with the distributed queue is that any operation is actually the update of a single entry in cache, so this will be a single point of contention. Most likely it will have negative effect on performance. [1] https://apacheignite.readme.io/docs/compute-grid -Val -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Implementing-a-distributed-crawler-tp6654p6682.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
