Hi, Thanks for the insight Marius.
> the first usecase was to implement a multi-datacenter sync... > [publish instances use active queues and author] saved the packages in a > passive queue for other datacenters to pull the updates. > second usecase was to pause delivery of some queues [in case one endpoint is > unavailable for a long time] Above is indeed helpful to know! I also understood from above that ResourceQueue was not implemented as an Active Queue because there was no such use case. Do you think there's merit in developing an active variant of ResourceQueue implementation? This will enable o.a.s.d.agent.impl.ForwardDistributionAgentFactory to use queues persisted in JCR without relying in Sling Jobs (for the reasons mentioned in SLING-7754). The caveat here would be that a queue-processor attached to the "Active" variant of ResourceQueue impl can only work on a single instance in the cluster [0], but this would still be helpful since Resource-backed queues (in contrast to a "In Memory" or "On Disk" queue) will survive terminations of Sling instances running in containers without any affinity to disk. I've opened SLING-8853 for that and I hope to be able to work on it. Please let me know in case this sounds like a bad idea at the outset. Regards -Ashish [0] https://sling.apache.org/documentation/bundles/scheduler-service-commons-scheduler.html#scheduling-the-job-just-once-in-a-cluster [1] https://issues.apache.org/jira/browse/SLING-8853 -----Original Message----- From: Marius Petria <[email protected]> Sent: Monday, November 18, 2019 8:10 PM To: [email protected] Subject: Re: Resource-backed Passive Queue impl in Content Distribution Hi, I introduced the passive queues in SLING-4629 and I am trying to recall the exact context __. An active queue is a queue for which a processor was defined and a passive queue is one for which no processor was defined. If I recall correctly the first usecase was to implement a multi-datacenter sync. In a datacenter there was an author instance that coordinated the sync between publish instances in that datacenter using active queues but also saved the packages in a passive queue for other datacenters to pull the updates. The second usecase was to pause delivery of some queues. Assume that one endpoint is down for a long time and you want to pause delivery to it, but still build the queue, then you make the queue for that endpoint passive. Marius https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fsling-org-apache-sling-distribution-core%2Fcommit%2F0a6508e42034721a97eb2f86f9ede591f09db7fe&data=02%7C01%7Cashishc%40adobe.com%7C1b128bd347084327e2b308d76c354016%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C1%7C637096848275000581&sdata=r%2F3hTgHQ0Ba8J4hT9zLxstzRIdKZRNQxAVTq0l5AhFA%3D&reserved=0 On 18/11/2019, 12:23, "Ashish Chopra" <[email protected]> wrote: Hi, I'm trying to better understand agent instances created via QueueDistributionAgentFactory [0], and the Resource-backed queues they leverage [1]. The Sling issue through which this implementation was added makes a mention of Active and Passive queues (as defined as [2]), with [1] being a Passive queue primarily useful for Reverse Distribution. I hope to be able to leverage Resource-backed queues for forward-distribution scenarios, but currently it is not possible (given [1] and [2]). I'm working to understand existing "active" queue impls in sling-dist-core bundle [3] to have a better sense of what would it take to develop an "active" variant of ResourceQueue impl. I have a few queries on the topic above: * is there any documentation (beyond [2], that is) which explains the concept of Passive and Active queues (in context of SCD) in more detail? * is there any background information on why ResourceQueue impl [1] is currently Passive? i.e., was it due to any known limitations, or a case of YAGNI? I read through SLING-7754 and searched the archives, but couldn't get any pointers. Would be great if the community can help me discover more information on this topic. Thanks Ashish [0] https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fsling-org-apache-sling-distribution-core%2Fblob%2Fmaster%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fsling%2Fdistribution%2Fagent%2Fimpl%2FQueueDistributionAgentFactory.java&data=02%7C01%7Cashishc%40adobe.com%7C1b128bd347084327e2b308d76c354016%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C1%7C637096848275000581&sdata=K0f%2BgG7ERQ99CcSniWrY5hw3g6XtuyMCUkXwTbKTig8%3D&reserved=0 [1] https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fsling-org-apache-sling-distribution-core%2Fblob%2Fmaster%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fsling%2Fdistribution%2Fqueue%2Fimpl%2Fresource%2FResourceQueue.java&data=02%7C01%7Cashishc%40adobe.com%7C1b128bd347084327e2b308d76c354016%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C1%7C637096848275000581&sdata=9X2WblTXcCUZLZjRyHlY%2F017WOplq2EwDvlzRqcgpLo%3D&reserved=0 [2] https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FSLING-7754%3FfocusedCommentId%3D16522146&data=02%7C01%7Cashishc%40adobe.com%7C1b128bd347084327e2b308d76c354016%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C1%7C637096848275010574&sdata=y%2FEkVWuIL4o%2FwRHaBty%2FyVDwqRhrY0tOoEKcD%2FjniUk%3D&reserved=0 [3] https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fsling-org-apache-sling-distribution-core%2Fblob%2Fdeb3d2ae%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fsling%2Fdistribution%2Fagent%2Fimpl%2FForwardDistributionAgentFactory.java%23L171-L177&data=02%7C01%7Cashishc%40adobe.com%7C1b128bd347084327e2b308d76c354016%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C1%7C637096848275010574&sdata=33t61wifNoixJDcF8maGreR2mtoN5PAmsbTT8UrlN08%3D&reserved=0
