Hi all, Looks like a very interesting plan. I have been working with Docker and docker-compose for about a year now and I would like to contribute. I am fairly new to Kubernetes though but still, I might be able to help on that part as well.
Kind regards, Madhawa *Madhawa Vidanapathirana* Student Department of Computer Science and Engineering University of Moratuwa Sri Lanka Mobile: (+94) 716874425 Email: madhawavidanapathir...@gmail.com Linked-In: *https://www.linkedin.com/in/madhawav/ <https://www.linkedin.com/in/madhawav/>* On Fri, Nov 30, 2018 at 4:00 PM Imesha Sudasingha <imesha...@cse.mrt.ac.lk> wrote: > Hi Lewis, > > The documentation related to distributed configuration management is at: > > https://cwiki.apache.org/confluence/display/OODT/OODT+Distributed+Configuration+Management > . > But it is not available in 1.2.x releases. > > This tool lets you upload configuration (say policies, properties files, > etc) to filemanagers, resource managers and workflow managers without > knowing their IPs. If you have a multi node cluster and want to publish > common configuration to all of them, this tool can be used. The difference > compared to something like ansible and this is, you don't need to know the > IPs of the nodes. > > Furthermore, I have tested the ability to listen for configuration change > and apply them accordingly to file manager. Try them out and let me know > what you think or any improvement is required. > > Thanks, > Imesha > > > > > On Fri, 30 Nov 2018 at 01:38, Tom Barber <t...@spicule.co.uk> wrote: > > > Imesha is the king of the distributed config stuff, I believe some docs > > went in the wiki although I can’t check right now. > > > > Interesting stuff on Brooklyn somehow I’d missed that, gonna check it > out! > > > > > > On 29 November 2018 at 19:18:06, Lewis John McGibbney ( > lewi...@apache.org) > > wrote: > > > > Some recent work on my end on a similar thread, for COAL-SDS [0] which is > > essentially a customized RADiX deployment, right now I'm working on > writing > > Apache Brooklyn [1] blueprints which will really help me to deploy > COAL-SDS > > to any cloud platform with the necessary dependencies installed so I can > > execute the algorithms which run in my OODT workflow. > > I didn't want to tie this to any particular cloud provider hence the > > Brooklyn blueprints. > > > > Back to the topic of conversation here, I think we should start with the > > basic services e.g. file, workflow and resource mgr. These can be > > Dockerized very easily with necessary flags for service configuration. > > Admitedly I > > ve not tried out the distributed configuration management yet. Is there > any > > documentation I can swot up on it please? > > > > [0] https://github.com/capstone-coal/coal-sds/ > > [1] https://brooklyn.apache.org > > > > On 2018/11/24 22:22:06, Tom Barber <t...@spicule.co.uk> wrote: > > > Hey Imesha, et al, > > > > > > I think the ideas are pretty good. Radix is a good starting point, but > > > there needs to be a useful recipe for people wanting to customise it > with > > > their own stuff. > > > > > > Docker Compose is a good way for spinning up a multi container > solution, > > > although without Swarm, you don’t get any redundancy. Kubernetes will > > give > > > users better scale out support. I’m also testing some cool Juju support > > for > > > Kubernetes which might make the deployment of this stuff easier on K8S. > > > > > > I’d certainly like to see an out of the box solution which uses the ZK > > > stuff you did, the Avro stuff thats going in and OODT, even if its a > way > > > for us to discover the ZK and scale up limitations. I think using > Docker > > > will make larger deployments of OODT easier and something we should > > > certainly put some thought into. > > > > > > > > > On 18 November 2018 at 11:53:53, Imesha Sudasingha ( > > imesha...@cse.mrt.ac.lk) > > > wrote: > > > > > > Hi All, > > > > > > Recently we had an interesting discussion > > > < > > > > > > > > https://lists.apache.org/thread.html/759f2145d9eacf87f3ed1255236354d713633883b47b91249b1f5c75@ > > > <dev.oodt.apache.org>> > > > [1] on making OODT easy to use out of the box through containerization. > > As > > > a summary I suggest following steps to move on. > > > > > > 1. Implement ability to build component wise docker images. Tom's idea > > was > > > to add a docker build profile to RADIX build. I think this will be a > good > > > starting point. > > > > > > 2. Add easy deployment capability through docker-compose > > > <https://docs.docker.com/compose/> and kubernetes. We can keep > > kubernetes > > > for later depending on the effort we can put. This will provide any new > > > comer to run a working example of OODT with a single command. A big > plus > > > point from new comers' point of view. > > > > > > 3. Since we have distributed configuration management > > > < > > > > > > > > https://cwiki.apache.org/confluence/display/OODT/OODT+Distributed+Configuration+Management > > > > > > > > > > > [2] already implemented, may be we don't need to put any burden of > > building > > > docker images to users. Instead, they can simply start a generic OODT > > > deployment (say using docker compose) and publish custom configuration > > > (properties, policies, etc) to those running components. However, need > to > > > discuss further on this. > > > > > > What do you think? Comments? > > > > > > New contributors are also welcome to participate in this discussion > (and > > of > > > course to contribute ;-) ) > > > > > > Cheers, > > > Imesha > > > > > > [1] > > > > > > > > https://lists.apache.org/thread.html/759f2145d9eacf87f3ed1255236354d713633883b47b91249b1f5c75@%3Cdev.oodt.apache.org%3E > > > [2] > > > > > > > > https://cwiki.apache.org/confluence/display/OODT/OODT+Distributed+Configuration+Management > > > > > > -- > > > > > > > > > Spicule Limited is registered in England & Wales. Company Number: > > > 09954122. Registered office: First Floor, Telecom House, 125-135 > Preston > > > Road, Brighton, England, BN1 6AF. VAT No. 251478891. > > > > > > > > > > > > > > > All engagements > > > are subject to Spicule Terms and Conditions of Business. This email and > > its > > > contents are intended solely for the individual to whom it is addressed > > and > > > may contain information that is confidential, privileged or otherwise > > > protected from disclosure, distributing or copying. Any views or > opinions > > > presented in this email are solely those of the author and do not > > > necessarily represent those of Spicule Limited. The company accepts no > > > liability for any damage caused by any virus transmitted by this email. > > If > > > you have received this message in error, please notify us immediately > by > > > reply email before deleting it from your system. Service of legal > notice > > > cannot be effected on Spicule Limited by email. > > > > > > > -- > > > > > > Spicule Limited is registered in England & Wales. Company Number: > > 09954122. Registered office: First Floor, Telecom House, 125-135 Preston > > Road, Brighton, England, BN1 6AF. VAT No. 251478891. > > > > > > > > > > All engagements > > are subject to Spicule Terms and Conditions of Business. This email and > > its > > contents are intended solely for the individual to whom it is addressed > > and > > may contain information that is confidential, privileged or otherwise > > protected from disclosure, distributing or copying. Any views or opinions > > presented in this email are solely those of the author and do not > > necessarily represent those of Spicule Limited. The company accepts no > > liability for any damage caused by any virus transmitted by this email. > If > > you have received this message in error, please notify us immediately by > > reply email before deleting it from your system. Service of legal notice > > cannot be effected on Spicule Limited by email. > > >
