Hi Lewis, The documentation related to distributed configuration management is at: https://cwiki.apache.org/confluence/display/OODT/OODT+Distributed+Configuration+Management. But it is not available in 1.2.x releases.
This tool lets you upload configuration (say policies, properties files, etc) to filemanagers, resource managers and workflow managers without knowing their IPs. If you have a multi node cluster and want to publish common configuration to all of them, this tool can be used. The difference compared to something like ansible and this is, you don't need to know the IPs of the nodes. Furthermore, I have tested the ability to listen for configuration change and apply them accordingly to file manager. Try them out and let me know what you think or any improvement is required. Thanks, Imesha On Fri, 30 Nov 2018 at 01:38, Tom Barber <t...@spicule.co.uk> wrote: > Imesha is the king of the distributed config stuff, I believe some docs > went in the wiki although I can’t check right now. > > Interesting stuff on Brooklyn somehow I’d missed that, gonna check it out! > > > On 29 November 2018 at 19:18:06, Lewis John McGibbney (lewi...@apache.org) > wrote: > > Some recent work on my end on a similar thread, for COAL-SDS [0] which is > essentially a customized RADiX deployment, right now I'm working on writing > Apache Brooklyn [1] blueprints which will really help me to deploy COAL-SDS > to any cloud platform with the necessary dependencies installed so I can > execute the algorithms which run in my OODT workflow. > I didn't want to tie this to any particular cloud provider hence the > Brooklyn blueprints. > > Back to the topic of conversation here, I think we should start with the > basic services e.g. file, workflow and resource mgr. These can be > Dockerized very easily with necessary flags for service configuration. > Admitedly I > ve not tried out the distributed configuration management yet. Is there any > documentation I can swot up on it please? > > [0] https://github.com/capstone-coal/coal-sds/ > [1] https://brooklyn.apache.org > > On 2018/11/24 22:22:06, Tom Barber <t...@spicule.co.uk> wrote: > > Hey Imesha, et al, > > > > I think the ideas are pretty good. Radix is a good starting point, but > > there needs to be a useful recipe for people wanting to customise it with > > their own stuff. > > > > Docker Compose is a good way for spinning up a multi container solution, > > although without Swarm, you don’t get any redundancy. Kubernetes will > give > > users better scale out support. I’m also testing some cool Juju support > for > > Kubernetes which might make the deployment of this stuff easier on K8S. > > > > I’d certainly like to see an out of the box solution which uses the ZK > > stuff you did, the Avro stuff thats going in and OODT, even if its a way > > for us to discover the ZK and scale up limitations. I think using Docker > > will make larger deployments of OODT easier and something we should > > certainly put some thought into. > > > > > > On 18 November 2018 at 11:53:53, Imesha Sudasingha ( > imesha...@cse.mrt.ac.lk) > > wrote: > > > > Hi All, > > > > Recently we had an interesting discussion > > < > > > > https://lists.apache.org/thread.html/759f2145d9eacf87f3ed1255236354d713633883b47b91249b1f5c75@ > > <dev.oodt.apache.org>> > > [1] on making OODT easy to use out of the box through containerization. > As > > a summary I suggest following steps to move on. > > > > 1. Implement ability to build component wise docker images. Tom's idea > was > > to add a docker build profile to RADIX build. I think this will be a good > > starting point. > > > > 2. Add easy deployment capability through docker-compose > > <https://docs.docker.com/compose/> and kubernetes. We can keep > kubernetes > > for later depending on the effort we can put. This will provide any new > > comer to run a working example of OODT with a single command. A big plus > > point from new comers' point of view. > > > > 3. Since we have distributed configuration management > > < > > > > https://cwiki.apache.org/confluence/display/OODT/OODT+Distributed+Configuration+Management > > > > > > > [2] already implemented, may be we don't need to put any burden of > building > > docker images to users. Instead, they can simply start a generic OODT > > deployment (say using docker compose) and publish custom configuration > > (properties, policies, etc) to those running components. However, need to > > discuss further on this. > > > > What do you think? Comments? > > > > New contributors are also welcome to participate in this discussion (and > of > > course to contribute ;-) ) > > > > Cheers, > > Imesha > > > > [1] > > > > https://lists.apache.org/thread.html/759f2145d9eacf87f3ed1255236354d713633883b47b91249b1f5c75@%3Cdev.oodt.apache.org%3E > > [2] > > > > https://cwiki.apache.org/confluence/display/OODT/OODT+Distributed+Configuration+Management > > > > -- > > > > > > Spicule Limited is registered in England & Wales. Company Number: > > 09954122. Registered office: First Floor, Telecom House, 125-135 Preston > > Road, Brighton, England, BN1 6AF. VAT No. 251478891. > > > > > > > > > > All engagements > > are subject to Spicule Terms and Conditions of Business. This email and > its > > contents are intended solely for the individual to whom it is addressed > and > > may contain information that is confidential, privileged or otherwise > > protected from disclosure, distributing or copying. Any views or opinions > > presented in this email are solely those of the author and do not > > necessarily represent those of Spicule Limited. The company accepts no > > liability for any damage caused by any virus transmitted by this email. > If > > you have received this message in error, please notify us immediately by > > reply email before deleting it from your system. Service of legal notice > > cannot be effected on Spicule Limited by email. > > > > -- > > > Spicule Limited is registered in England & Wales. Company Number: > 09954122. Registered office: First Floor, Telecom House, 125-135 Preston > Road, Brighton, England, BN1 6AF. VAT No. 251478891. > > > > > All engagements > are subject to Spicule Terms and Conditions of Business. This email and > its > contents are intended solely for the individual to whom it is addressed > and > may contain information that is confidential, privileged or otherwise > protected from disclosure, distributing or copying. Any views or opinions > presented in this email are solely those of the author and do not > necessarily represent those of Spicule Limited. The company accepts no > liability for any damage caused by any virus transmitted by this email. If > you have received this message in error, please notify us immediately by > reply email before deleting it from your system. Service of legal notice > cannot be effected on Spicule Limited by email. >
