As I already mentioned, I would use compute grid for this. As for the back pressure, you can limit the number of jobs executed at the same time and define different scheduling strategies [1]. The queue just looks like an unnecessary piece here.
[1] https://apacheignite.readme.io/docs/job-scheduling -Val -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Implementing-a-distributed-crawler-tp6654p6689.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
