Some recent work on my end on a similar thread, for COAL-SDS [0] which is essentially a customized RADiX deployment, right now I'm working on writing Apache Brooklyn [1] blueprints which will really help me to deploy COAL-SDS to any cloud platform with the necessary dependencies installed so I can execute the algorithms which run in my OODT workflow. I didn't want to tie this to any particular cloud provider hence the Brooklyn blueprints.
Back to the topic of conversation here, I think we should start with the basic services e.g. file, workflow and resource mgr. These can be Dockerized very easily with necessary flags for service configuration. Admitedly I ve not tried out the distributed configuration management yet. Is there any documentation I can swot up on it please? [0] https://github.com/capstone-coal/coal-sds/ [1] https://brooklyn.apache.org On 2018/11/24 22:22:06, Tom Barber <t...@spicule.co.uk> wrote: > Hey Imesha, et al, > > I think the ideas are pretty good. Radix is a good starting point, but > there needs to be a useful recipe for people wanting to customise it with > their own stuff. > > Docker Compose is a good way for spinning up a multi container solution, > although without Swarm, you don’t get any redundancy. Kubernetes will give > users better scale out support. I’m also testing some cool Juju support for > Kubernetes which might make the deployment of this stuff easier on K8S. > > I’d certainly like to see an out of the box solution which uses the ZK > stuff you did, the Avro stuff thats going in and OODT, even if its a way > for us to discover the ZK and scale up limitations. I think using Docker > will make larger deployments of OODT easier and something we should > certainly put some thought into. > > > On 18 November 2018 at 11:53:53, Imesha Sudasingha (imesha...@cse.mrt.ac.lk) > wrote: > > Hi All, > > Recently we had an interesting discussion > < > https://lists.apache.org/thread.html/759f2145d9eacf87f3ed1255236354d713633883b47b91249b1f5c75@ > <dev.oodt.apache.org>> > [1] on making OODT easy to use out of the box through containerization. As > a summary I suggest following steps to move on. > > 1. Implement ability to build component wise docker images. Tom's idea was > to add a docker build profile to RADIX build. I think this will be a good > starting point. > > 2. Add easy deployment capability through docker-compose > <https://docs.docker.com/compose/> and kubernetes. We can keep kubernetes > for later depending on the effort we can put. This will provide any new > comer to run a working example of OODT with a single command. A big plus > point from new comers' point of view. > > 3. Since we have distributed configuration management > < > https://cwiki.apache.org/confluence/display/OODT/OODT+Distributed+Configuration+Management> > > [2] already implemented, may be we don't need to put any burden of building > docker images to users. Instead, they can simply start a generic OODT > deployment (say using docker compose) and publish custom configuration > (properties, policies, etc) to those running components. However, need to > discuss further on this. > > What do you think? Comments? > > New contributors are also welcome to participate in this discussion (and of > course to contribute ;-) ) > > Cheers, > Imesha > > [1] > https://lists.apache.org/thread.html/759f2145d9eacf87f3ed1255236354d713633883b47b91249b1f5c75@%3Cdev.oodt.apache.org%3E > [2] > https://cwiki.apache.org/confluence/display/OODT/OODT+Distributed+Configuration+Management > > -- > > > Spicule Limited is registered in England & Wales. Company Number: > 09954122. Registered office: First Floor, Telecom House, 125-135 Preston > Road, Brighton, England, BN1 6AF. VAT No. 251478891. > > > > > All engagements > are subject to Spicule Terms and Conditions of Business. This email and its > contents are intended solely for the individual to whom it is addressed and > may contain information that is confidential, privileged or otherwise > protected from disclosure, distributing or copying. Any views or opinions > presented in this email are solely those of the author and do not > necessarily represent those of Spicule Limited. The company accepts no > liability for any damage caused by any virus transmitted by this email. If > you have received this message in error, please notify us immediately by > reply email before deleting it from your system. Service of legal notice > cannot be effected on Spicule Limited by email. >
