Re: Sling integration with Etcd
Hi all, Thanks for your input. Yes, operations is something to consider, that's something we discuss internally. A small Go process also has its appeal when compared to a JVM deployment. I'll give Etcd a go as it seems nobody has done it already. Regards, Timothee 2015-02-04 15:56 GMT+00:00 Bertrand Delacretaz bdelacre...@apache.org: On Wed, Feb 4, 2015 at 4:45 PM, Bertrand Delacretaz bdelacre...@apache.org wrote: ...I haven't integrated etcd with the Sling discovery but I have used it for my https://github.com/bdelacretaz/docker-sling-cluster prototype and found it quite nice Note also that Zookeeper running in a JVM is an advantage over etcd being AFAIK a Linux thing - operations familiarity might be a criteria to consider. -Bertrand
Re: Sling integration with Etcd
I made an Ubuntu package out of etcd for an unrelated project. If anyone wants a copy let me know. -Bruce From: Bertrand Delacretaz bdelacre...@apache.org Reply-To: dev dev@sling.apache.org Date: Wednesday, February 4, 2015 at 7:45 AM To: dev dev@sling.apache.org Subject: Re: Sling integration with Etcd Hi, On Wed, Feb 4, 2015 at 2:40 PM, Timothée Maret timothee.ma...@gmail.com wrote: ...Has someone already implemented it ?... I haven't integrated etcd with the Sling discovery but I have used it for my https://github.com/bdelacretaz/docker-sling-cluster prototype and found it quite nice. -Bertrand
Re: Sling integration with Etcd
On Wed, Feb 4, 2015 at 4:45 PM, Bertrand Delacretaz bdelacre...@apache.org wrote: ...I haven't integrated etcd with the Sling discovery but I have used it for my https://github.com/bdelacretaz/docker-sling-cluster prototype and found it quite nice Note also that Zookeeper running in a JVM is an advantage over etcd being AFAIK a Linux thing - operations familiarity might be a criteria to consider. -Bertrand
Sling integration with Etcd
Hi, I found some time to look closer to the Sling Discovery implementation on AWS S3 proposal in [0]. Doing so, I stumbled upon Etcd [1] based on Raft [2] which seems to covers exactly what I wanted to do. Indeed, Etcd provides both persistent and temporal keys a well as primitives to perform leader election. Therefor, the Sling discovery API could be implemented on top of Etcd with the notable addition of * multi-level leader election (e.g. at deploymen level, region level, or any arbitrary segmentation). * persistent properties Etcd would require to be deployed on dedicated HW (not embedded) but offers HTTP API for Sling instances to communicate with it. I have added this option in SLING-2939. Has someone already implemented it ? If not, I think it would make much more sense to implement this integration rather than the idea in [0] and I would give it a go. Regards, Timothee [0] http://apache-sling.73963.n3.nabble.com/Sling-Discovery-implementation-on-AWS-S3-td4033160.html [1] https://github.com/coreos/etcd [2] https://raftconsensus.github.io
Re: Sling integration with Etcd
Hi Timothée, As discussed yesterday privately I absolutely support the idea of having an additional, 3rd-party based implementation of discovery.api. Whether it be etcd or zookeeper (or even something else) is up to feature comparison/discussion/etc.. One thing I thought was mentioned is that we use ZooKeeper elsewhere already and there might thus be synergies. But I haven't done a direct comparison of etcd vs zookeeper so far, so don't know if there is a clear winner anyway.. One thing to remember when implementing: one feature of discovery.api is that it groups instances hooked to the same repository into the same ClusterView and provides it with a stable (cluster)Id. In discovery.impl that is achieved by chatting and voting via said repository. When using an external mechanism, the fact that an instance belongs to a certain cluster needs special treatment as well in one way or another: either by still storing the id in the repository for example or via some configuration mechanism maybe. (Just something to keep in mind..) Cheers, Stefan On 2/4/15 2:40 PM, Timothée Maret timothee.ma...@gmail.com wrote: Hi, I found some time to look closer to the Sling Discovery implementation on AWS S3 proposal in [0]. Doing so, I stumbled upon Etcd [1] based on Raft [2] which seems to covers exactly what I wanted to do. Indeed, Etcd provides both persistent and temporal keys a well as primitives to perform leader election. Therefor, the Sling discovery API could be implemented on top of Etcd with the notable addition of * multi-level leader election (e.g. at deploymen level, region level, or any arbitrary segmentation). * persistent properties Etcd would require to be deployed on dedicated HW (not embedded) but offers HTTP API for Sling instances to communicate with it. I have added this option in SLING-2939. Has someone already implemented it ? If not, I think it would make much more sense to implement this integration rather than the idea in [0] and I would give it a go. Regards, Timothee [0] http://apache-sling.73963.n3.nabble.com/Sling-Discovery-implementation-on- AWS-S3-td4033160.html [1] https://github.com/coreos/etcd [2] https://raftconsensus.github.io
Re: Sling integration with Etcd
Hi, On Wed, Feb 4, 2015 at 2:40 PM, Timothée Maret timothee.ma...@gmail.com wrote: ...Has someone already implemented it ?... I haven't integrated etcd with the Sling discovery but I have used it for my https://github.com/bdelacretaz/docker-sling-cluster prototype and found it quite nice. -Bertrand
