I admit I'm not as knowledgable about the Kolla codebase as I'd like to be, so most of what you're saying is going over my head. I think mainly I don't understand the problem statement. It looks like you're pulling all the "hard coded" things out of the docker files, and making them user replaceable? So the dockerfiles just become a list of required steps, and the user can change how each step is implemented? Would this also unify the dockefiles so there wouldn't be a huge if statements between Centos and Ubuntu?
Thx, Britt On 5/27/16, 1:58 AM, "Steven Dake (stdake)" <std...@cisco.com> wrote: > > >On 5/26/16, 8:45 PM, "Swapnil Kulkarni (coolsvap)" <m...@coolsvap.net> wrote: > >>On Fri, May 27, 2016 at 8:35 AM, Steven Dake (stdake) <std...@cisco.com> >>wrote: >>> Hey folks, >>> >>> While Swapnil has been busy churning the dockerfile.j2 files to all >>>match >>> the same style, and we also had summit where we declared we would solve >>>the >>> plugin problem, I have decided to begin work on a DSL prototype. >>> >>> Here are the problems I want to solve in order of importance by this >>>work: >>> >>> Build CentOS, Ubuntu, Oracle Linux, Debian, Fedora containers >>> Provide a programmatic way to manage Dockerfile construction rather >>>then a >>> manual (with vi or emacs or the like) mechanism >>> Allow complete overrides of every facet of Dockerfile construction, most >>> especially repositories per container (rather than in the base >>>container) to >>> permit the use case of dependencies from one version with dependencies >>>in >>> another version of a different service >>> Get out of the business of maintaining 100+ dockerfiles but instead >>>maintain >>> one master file which defines the data that needs to be used to >>>construct >>> Dockerfiles >>> Permit different types of optimizations or Dockerfile building by >>>changing >>> around the parser implementation to allow layering of each operation, >>>or >>> alternatively to merge layers as we do today >>> >>> I don't believe we can proceed with both binary and source plugins >>>given our >>> current implementation of Dockerfiles in any sane way. >>> >>> I further don't believe it is possible to customize repositories & >>>installed >>> files per container, which I receive increasing requests for offline. >>> >>> To that end, I've created a very very rough prototype which builds the >>>base >>> container as well as a mariadb container. The mariadb container builds >>>and >>> I suspect would work. >>> >>> An example of the DSL usage is here: >>> https://review.openstack.org/#/c/321468/4/dockerdsl/dsl.yml >>> >>> A very poorly written parser is here: >>> https://review.openstack.org/#/c/321468/4/dockerdsl/load.py >>> >>> I played around with INI as a format, to take advantage of oslo.config >>>and >>> kolla-build.conf, but that didn't work out. YML is the way to go. >>> >>> I'd appreciate reviews on the YML implementation especially. >>> >>> How I see this work progressing is as follows: >>> >>> A yml file describing all docker containers for all distros is placed in >>> kolla/docker >>> The build tool adds an option ‹use-yml which uses the YML file >>> A parser (such as load.py above) is integrated into build.py to lay >>>down he >>> Dockerfiles >>> Wait 4-6 weeks for people to find bugs and complain >>> Make the ‹use-yml the default for 4-6 weeks >>> Once we feel confident in the yml implementation, remove all >>>Dockerfile.j2 >>> files >>> Remove ‹use-yml option >>> Remove all jinja2-isms from build.py >>> >>> This is similar to the work that took place to convert from raw >>>Dockerfiles >>> to Dockerfile.j2 files. We are just reusing that pattern. Hopefully >>>this >>> will be the last major refactor of the dockerfiles unless someone has >>>some >>> significant complaints about the approach. >>> >>> Regards >>> -steve >>> >>> >>> >>>_________________________________________________________________________ >>>_ >>> OpenStack Development Mailing List (not for usage questions) >>> Unsubscribe: >>>openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>> >> >>The DSL template to generate the Dockerfile seems way better than the >>jinja templates in terms of extension which is currently a major >>bottleneck in the plugin implementation. I am +2+W on this plan of >>action to test it for next 4-6 weeks and see thereon. >> >>Swapnil >> > >Agree. > >Customization and plugins are the trigger for the work. I was thinking of >the following: > >Elemental.yml (ships with Kolla) >Elemental-merge.yml (operator provides in /etc/kolla, this file is yaml >merged with elemental.yml) >Elemental-override.yml (operator provides in /etc/kolla, this file >overrides any YAML sections defined) > >I think merging and overriding the yaml files should be pretty easy, >compared to jinja2, where I don't even know where to begin in a way that >the operator doesn't have to have deep knowledge of Kolla's internal >implementation. > >Regards >-steve > >>__________________________________________________________________________ >>OpenStack Development Mailing List (not for usage questions) >>Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > >__________________________________________________________________________ >OpenStack Development Mailing List (not for usage questions) >Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev