The solution is interesting, maybe AWX is not the right tool but I would like to hear what Marco thinks about this.
The most important phase is going to be the initial one: the prototype will help us to refine the requirements. Fed On Tue, Mar 26, 2019 at 4:28 PM A Stanley <[email protected]> wrote: > That's what made me think to mention Terraform. You can still leverage > your ansible playbooks for configuration and use Terraform's Kubernetes > integration to handle the infrastructure complexity. Of course this is all > easier said than done. I've done something sort of similar with Terraform > and docker-compose on different cloud providers but haven't made the jump > to Kubernetes yet. A simple non-scaling example is here; > > https://github.com/2stacks/terraform-openvpn > > Just wanted to add a suggestion. I have not used AWX but I intend to > check it out. > > As for the pain with integrating multiple applications across multiple > OSs, code bases, dependencies etc. I feel your pain. I recently started > writing integration tests for Travis-CI for all of my containers. Not just > for code builds but for how the containers interact once they're built. > This has saved me some headaches lately. > > Ill be honest I haven't tried customizing containers with ansible. I > usually use ansible to customize my Docker Hosts. I like baking custom > containers that can be modified as needed with environment variables. The > best examples I've seen come from cookiecutter-django. I usually start > there and then publish the container with the app preloaded for reuse > later. Not to drag on but this subject is very broad and there are lots of > options. > > > On Tue, Mar 26, 2019, 3:44 PM Federico Capoano <[email protected]> > wrote: > >> Marco Giuntini also has some experience with Terraform, is it right Marco? >> >> I am not the expert on this front, I'm open to suggestions. We need a >> tool which allows us to configure an instance. >> I like ansible-openwisp2 because I use a playbook or a series of >> playbooks for an installation, for example: >> >> - a playbook for all the openwisp configurations, with additional tasks >> and roles for postgresql, freeradius, login page etc. >> - a playbook for the openvpn server >> >> I like the fact that most of the configurations are in YAML format and I >> can easily understand the configuration of an installation by looking at >> the YAML. >> >> The problem is that it is not simple to support horizontal scaling and >> that it become a pain to maintain over time. >> For example, if you have installed an OpenWISP instance on Ubuntu 16 LTS >> and then you have to migrate to Ubuntu 18 LTS, you create a new VM, run the >> playbooks there, pray that it works, most of the time it won't work out of >> the box and then you have to spend 1 day or more to fix the issues that >> come up, which may happen in other roles that you don't control and then >> you have to fork them and patch them, send pull requests to the maintainers >> which will take 1 month on average to merge it. >> Or if you have a custom module, maybe to deploy it you have to execute >> additional tasks that may conflict with other previous tasks and you find >> out only when you deploy. >> And if someone with access to the VM has made some manual editing... you >> risk overwriting something they did. >> >> With docker instead, if you migrate to a new OS, you work on the images >> (which can be also painful, but if something breaks you find out soon >> during testing, not when you deploy) and when the images compile >> successfully you can run a full instance locally and test it before >> deploying. >> Same applies when you add custom modules by extending a docker image (eg: >> use it as a base). >> The good thing is the immutability of the result and the fact that as >> soon as a system package or a python dependency needs to be upgraded, you >> generate a new image, you bring up new containers and the old ones die. >> No manual configurations can be made, because the services are ephemeral >> (will be thrown away) by definition, everything can be and must be replaced >> easily, which means can also be replicated and scaled up very easily. >> >> So the ideal situation is one in which: >> >> - we can manage most configurations of the services in some textual format >> - we can manage the django settings easily for a single installation (in >> which different containers may have mostly identical django settings with >> some minor differences, django has a way to specify default settings, we >> could use that low level feature for example) >> - we can store all these configs under git in private repos >> >> On Tue, Mar 26, 2019 at 2:14 PM A Stanley <[email protected]> wrote: >> >>> Terraform.io may also be an option. I've had some success with it and >>> Docker/Ansible and it has support for Kubernetes. >>> >>> On Tue, Mar 26, 2019, 1:56 PM Marco Giuntini <[email protected]> >>> wrote: >>> >>>> Hi, >>>> >>>> https://github.com/ansible/awx is not a tool but it is an example on >>>> how Ansible AWX project uses docker and ansible to deploy and configure the >>>> application. >>>> >>>> Regards, >>>> >>>> -- >>>> Marco Giuntini >>>> GTalk: [email protected] >>>> Skype: hispanico70420 >>>> Twitter: @Hispanico81 >>>> PGP Key ID: BD774009 >>>> >>>> >>>> On Tue, 26 Mar 2019 at 17:25, Federico Capoano < >>>> [email protected]> wrote: >>>> >>>>> A demo on Kubernetes and using some kind of tool that makes it easy to >>>>> provision (Marco Giuntini suggested https://github.com/ansible/awx, >>>>> if you know something equivalent you could suggest) would be a big win. >>>>> >>>>> Helping out in writing tests (using some sort of mock SSH server) for >>>>> https://github.com/openwisp/openwisp-controller/pull/31 is also a big >>>>> plus, because if the students really understand the code of that branch it >>>>> will have an easier time to understand how to deploy it, I have some >>>>> automated test samples that I can show if needed. >>>>> >>>>> Federico >>>>> >>>>> >>>>> On Tue, Mar 26, 2019 at 11:31 AM Ajay Tripathi <[email protected]> >>>>> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> On Monday, March 25, 2019 at 7:25:05 PM UTC+5:30, Federico Capoano >>>>>> wrote: >>>>>>> >>>>>>> Yes, but we also want to allow to scale horizontally more easily, so >>>>>>> the traffic can be dispatched to more instances of the same containers >>>>>>> if >>>>>>> needed (imagine a kubernetes cluster spread on multiple nodes). >>>>>>> >>>>>> >>>>>> Great. I made a docker swarm stack and tested horizontal scaling of >>>>>> my prototype of the docker-compose. I understand the requirement better >>>>>> now. >>>>>> **The mentioned docker-compose stack is a simple openwisp container >>>>>> and a redis container. >>>>>> >>>>>> >>>>>>> The Admin interface is used ony by administrators, not very often, >>>>>>> so we likley won't need more instances of the same containers to scale. >>>>>>> The load on the other services increases with the size of the >>>>>>> network (number of devices or also number of users in the case of the >>>>>>> radius module), so we will need to scale up with those. >>>>>>> >>>>>> >>>>>> Thanks for clearing that up. >>>>>> >>>>>> >>>>>>> each module has its own URLs and APIs that are not admin related >>>>>>> which are used to provide configurations, update the network topology, >>>>>>> communicate with freeradius and so on, each one of these group of >>>>>>> features >>>>>>> should run in isolation. >>>>>>> >>>>>> A more realistic scenario is a user who wants to use only admin + >>>>>>> radius module, or admin + controller module. >>>>>>> But we should not add restrictions on which containers the users >>>>>>> want to use because we should consider this to be a base on which users >>>>>>> and >>>>>>> developers can build a solution tailored to their needs with custom >>>>>>> modules. >>>>>>> >>>>>> django-channels is the framework with which you build the websocket >>>>>>> server, but the actual logic that allows you to do anything with the >>>>>>> websocket is in OpenWISP. >>>>>>> At the moment only OpenWISP Controller has some websocket logic >>>>>>> (inherited from django-loci) but in the future we will have more. >>>>>>> >>>>>> >>>>>>> The duty of this container is to serve the websocket server and >>>>>>> process data coming from websocket clients. >>>>>>> >>>>>>> Immagine the same installation we have today, but instead of having >>>>>>> it on a single VM, we have it spread on different containers, each >>>>>>> container dedicated to a single service: the containers which receive >>>>>>> more >>>>>>> traffic can be scaled up, either vertically with more resources (RAM, >>>>>>> CPU) >>>>>>> or horizontally with more containers if possible (if a load balancer in >>>>>>> front of the containers is available to distribute traffic, we can do >>>>>>> this >>>>>>> with nginx for all the containers which serve HTTP or WebSocket >>>>>>> requests, >>>>>>> with the celery containers we don't need a load balancer because they >>>>>>> read >>>>>>> from the broker service which in our case is redis by default). >>>>>>> >>>>>> >>>>>> That explained some crucial points to help me understand! :) >>>>>> >>>>>> >>>>>>> I hope is clearer now! >>>>>>> >>>>>> >>>>>> Yes, thank you. Currently, I am reading more on the same. :) >>>>>> I am making a prototype with some of the features for understanding >>>>>> further, I am working on the features in whichever order I feel is most >>>>>> important for complete understanding. >>>>>> Please let me know if there is a specific requirement of the >>>>>> prototype that you'd like to see before the deadline so that I can focus >>>>>> on >>>>>> that first. >>>>>> >>>>>> Ajay >>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "OpenWISP" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to [email protected]. >>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "OpenWISP" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "OpenWISP" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "OpenWISP" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> For more options, visit https://groups.google.com/d/optout. >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "OpenWISP" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> For more options, visit https://groups.google.com/d/optout. >> > -- > You received this message because you are subscribed to the Google Groups > "OpenWISP" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "OpenWISP" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
