On 19 October 2017 at 13:32, Joshua Harlow <harlo...@fastmail.com> wrote: > This reminded me of something I wanted to ask. > > Is it true to state that only way to get 'fully' shared-base layers is to > have `kolla-build` build all the projects (that a person/company/other may > use) in one invocation? (in part because of the jinja2 template generation > which would cause differences in dockerfiles?...)
Well jinja2 should render same dockerfile no matter when you call it, so it should be fine. Alternatively you can run something like kolla-build nova --skip-parents - this call will try to build all images with "nova" in them while not rebuilding openstack-base and base image. > I was pretty sure this was the case (unless things have changed), but just > wanting to check since that question seems (somewhat) on-topic... > > At godaddy we build individual projects using `kolla-build` (in part because > it makes it easy to rebuild + test + deply a single project with either an > update or a patch or ...) and I suspect others are doing this also (after > all the kolla-build command does take a regex of projects to build) - though > doing it in this way does seem like it would not reuse (all the layers > outside of the base operating system) layers 'optimally'? > > Thoughts? > > -Josh > > Sam Yaple wrote: >> >> docker_image wouldn't be the best place for that. Buf if you are looking >> for a quicker solution, kolla_docker was written specifically to be >> license compatible for openstack. its structure should make it easily >> adapted to delete an image. And you can copy it and cut it up thanks to >> the license. >> >> Are you pushing images with no shared base layers at all (300MB >> compressed image is no shared base layers)? With shared base layers a >> full image set of Kolla images should be much smaller than the numbers >> you posted. >> >> Thanks, >> SamYaple >> >> On Thu, Oct 19, 2017 at 11:03 AM, Gabriele Cerami <gcer...@redhat.com >> <mailto:gcer...@redhat.com>> wrote: >> >> Hi, >> >> our CI scripts are now automatically building, testing and pushing >> approved openstack/RDO services images to public repositories in >> dockerhub using ansible docker_image module. >> >> Promotions have had some hiccups, but we're starting to regularly >> upload >> new images every 4 hours. >> >> When we'll get at full speed, we'll potentially have >> - 3-4 different sets of images, one per release of openstack (counting >> a >> EOL release grace period) >> - 90-100 different services images per release >> - 4-6 different versions of the same image ( keeping older promoted >> images for a while ) >> >> At around 300MB per image a possible grand total is around 650GB of >> space used. >> >> We don't know if this is acceptable usage of dockerhub space and for >> this we already sent a similar email the to docker support to ask >> specifically if the user would get penalty in any way (e.g. enforcing >> quotas, rete limiting, blocking). We're still waiting for a reply. >> >> In any case it's critical to keep the usage around the estimate, and >> to >> achieve this we need a way to automatically delete the older images. >> docker_image module does not provide this functionality, and we think >> the only way is issuing direct calls to dockerhub API >> >> https://docs.docker.com/registry/spec/api/#deleting-an-image >> <https://docs.docker.com/registry/spec/api/#deleting-an-image> >> >> docker_image module structure doesn't seem to encourage the addition >> of >> such functionality directly in it, so we may be forced to use the uri >> module. >> With new images uploaded potentially every 4 hours, this will become a >> problem to be solved within the next two weeks. >> >> We'd appreciate any input for an existing, in progress and/or better >> solution for bulk deletion, and issues that may arise with our space >> usage in dockerhub >> >> Thanks >> >> >> __________________________________________________________________________ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: >> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> <http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev> >> >> >> __________________________________________________________________________ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
