Imesha is the king of the distributed config stuff, I believe some docs went in the wiki although I can’t check right now.
Interesting stuff on Brooklyn somehow I’d missed that, gonna check it out! On 29 November 2018 at 19:18:06, Lewis John McGibbney (lewi...@apache.org) wrote: Some recent work on my end on a similar thread, for COAL-SDS [0] which is essentially a customized RADiX deployment, right now I'm working on writing Apache Brooklyn [1] blueprints which will really help me to deploy COAL-SDS to any cloud platform with the necessary dependencies installed so I can execute the algorithms which run in my OODT workflow. I didn't want to tie this to any particular cloud provider hence the Brooklyn blueprints. Back to the topic of conversation here, I think we should start with the basic services e.g. file, workflow and resource mgr. These can be Dockerized very easily with necessary flags for service configuration. Admitedly I ve not tried out the distributed configuration management yet. Is there any documentation I can swot up on it please? [0] https://github.com/capstone-coal/coal-sds/ [1] https://brooklyn.apache.org On 2018/11/24 22:22:06, Tom Barber <t...@spicule.co.uk> wrote: > Hey Imesha, et al, > > I think the ideas are pretty good. Radix is a good starting point, but > there needs to be a useful recipe for people wanting to customise it with > their own stuff. > > Docker Compose is a good way for spinning up a multi container solution, > although without Swarm, you don’t get any redundancy. Kubernetes will give > users better scale out support. I’m also testing some cool Juju support for > Kubernetes which might make the deployment of this stuff easier on K8S. > > I’d certainly like to see an out of the box solution which uses the ZK > stuff you did, the Avro stuff thats going in and OODT, even if its a way > for us to discover the ZK and scale up limitations. I think using Docker > will make larger deployments of OODT easier and something we should > certainly put some thought into. > > > On 18 November 2018 at 11:53:53, Imesha Sudasingha ( imesha...@cse.mrt.ac.lk) > wrote: > > Hi All, > > Recently we had an interesting discussion > < > https://lists.apache.org/thread.html/759f2145d9eacf87f3ed1255236354d713633883b47b91249b1f5c75@ > <dev.oodt.apache.org>> > [1] on making OODT easy to use out of the box through containerization. As > a summary I suggest following steps to move on. > > 1. Implement ability to build component wise docker images. Tom's idea was > to add a docker build profile to RADIX build. I think this will be a good > starting point. > > 2. Add easy deployment capability through docker-compose > <https://docs.docker.com/compose/> and kubernetes. We can keep kubernetes > for later depending on the effort we can put. This will provide any new > comer to run a working example of OODT with a single command. A big plus > point from new comers' point of view. > > 3. Since we have distributed configuration management > < > https://cwiki.apache.org/confluence/display/OODT/OODT+Distributed+Configuration+Management> > > [2] already implemented, may be we don't need to put any burden of building > docker images to users. Instead, they can simply start a generic OODT > deployment (say using docker compose) and publish custom configuration > (properties, policies, etc) to those running components. However, need to > discuss further on this. > > What do you think? Comments? > > New contributors are also welcome to participate in this discussion (and of > course to contribute ;-) ) > > Cheers, > Imesha > > [1] > https://lists.apache.org/thread.html/759f2145d9eacf87f3ed1255236354d713633883b47b91249b1f5c75@%3Cdev.oodt.apache.org%3E > [2] > https://cwiki.apache.org/confluence/display/OODT/OODT+Distributed+Configuration+Management > > -- > > > Spicule Limited is registered in England & Wales. Company Number: > 09954122. Registered office: First Floor, Telecom House, 125-135 Preston > Road, Brighton, England, BN1 6AF. VAT No. 251478891. > > > > > All engagements > are subject to Spicule Terms and Conditions of Business. This email and its > contents are intended solely for the individual to whom it is addressed and > may contain information that is confidential, privileged or otherwise > protected from disclosure, distributing or copying. Any views or opinions > presented in this email are solely those of the author and do not > necessarily represent those of Spicule Limited. The company accepts no > liability for any damage caused by any virus transmitted by this email. If > you have received this message in error, please notify us immediately by > reply email before deleting it from your system. Service of legal notice > cannot be effected on Spicule Limited by email. > -- Spicule Limited is registered in England & Wales. Company Number: 09954122. Registered office: First Floor, Telecom House, 125-135 Preston Road, Brighton, England, BN1 6AF. VAT No. 251478891. All engagements are subject to Spicule Terms and Conditions of Business. This email and its contents are intended solely for the individual to whom it is addressed and may contain information that is confidential, privileged or otherwise protected from disclosure, distributing or copying. Any views or opinions presented in this email are solely those of the author and do not necessarily represent those of Spicule Limited. The company accepts no liability for any damage caused by any virus transmitted by this email. If you have received this message in error, please notify us immediately by reply email before deleting it from your system. Service of legal notice cannot be effected on Spicule Limited by email.
