[
https://issues.apache.org/jira/browse/SOLR-15250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17315875#comment-17315875
]
Chris M. Hostetter commented on SOLR-15250:
-------------------------------------------
{quote}Perfectly happy to move the docker-scripts to a different directory in
the TGZ, but I would prefer for them to not live under server/
{quote}
I split this out into SOLR-15322 since it seems like something that can be
hashed out & committed independently of how exactly we generated "official"
Dockerfiles
----
{quote}I’m not sure it’s necessary to have all of that logic in the Dockerfile
that isn’t used until release time. When we were talking about templating, I
was imagining that the Keys/Download/Verification logic would be injected,
along with all of those ARGs. It certainly works in the patch that you have,
but it’s hard to tell what’s used for each build style.
{quote}
The idea was everything in the first "input" stage is really just for the
official releases – otherwise all it does is copy the (pre-existing solr.tgz)
so that it's available in the second stage (which is identical regardless of
any downloading or validation done for official releases).
To back up for a minute, the crux of the idea i was going for is what i
mentioned in the jira description...
{quote}The driving goal being that the Dockerfile used for official {{_/solr}}
builds should be as close as possible to identical to the Dockerfile used for
"local" builds by users – given the constraints put on us by the docker-library
team.
{quote}
...i was really trying to keep things such that the "template" used to build
the official Dockerfile *IS* the local Dockerfile, just with some simple
replacement of the ARG lines...
{noformat}
# psuedo-shell "templating" logic
Dockerfile.local
# used as is for local builds
cat Dockerfile.local | replace_blank_args_with_values.sh > Dockerfile.official
# used for _/solr
{noformat}
I was really hoping to keep the "Dockerfile.local" – commited in GIT – to be
fully readable and understandable and usable "as is" just by passing sufficient
build args ... it might include logic that was unnecessary for local builds,
but it would still be obvious from reading it how *BOTH* builds worked, w/o
needing to understand any special template logic (or look at other "snippet
files" that got included, etc...)
I had _initially_ hoped we could just "pre-pend" an extra build stage in front
of Dockerfile.local (along the lines of your comment about "injecting" that
additional logic) that would keep the files simple in a similar way, something
like...
{noformat}
# psuedo-shell "templating" logic
Dockerfile.local #
used as is for local builds
cat Dockerfile.official-snippet Dockerfile.local > Dockerfile.official #
used for _/solr
{noformat}
...but that isn't really viable because of how multi-stage builds work, there
is no way to "inherit" any context from a previous-stage w/o explicitly {{COPY
--from=...}} 'ing the data.
Which means if we need more complex templating / injecting, I don't really see
any way of doing it that would still allow us to have a 'usable'
Dockerfile.local in the GIT repo ... it seems like we'd have to have both
Dockerfile.local and Dockerfile.official be "generated" by templating logic...
{noformat}
# psuedo-shell "templating" logic
cat Dockerfile.common.stub | replace_vars_with_values.sh > Dockerfile.local
# used as is for local builds
cat Dockerfile.official.stub Dockerfile.common.stub |
replace_vars_with_values.sh > Dockerfile.official # used for _/solr
{noformat}
...which is fine, just not what i was originally hoping we could achieve.
----
{quote}Actually we probably want to create a folder (not in the build
directory) that stores it, so that we can commit it when we cut the official
release in Git. Therefore we have somewhere to point to here:
[https://github.com/docker-library/official-images/blob/master/library/solr] We
would need to remove the folder after upping the branch version (8.8.0 ->
8.8.1-SNAPSHOT). But we do need it in the git history
{quote}
I spun this off into SOLR-15321 since it seems like a larger conversation about
the overall process that needs to be better fleshed out independent of how
exactly we handle the templating.
----
{quote}I’m +1 to downloading the keys, but it wouldn’t be hard to add the key
in the release logic. It’s something the release wizard already knows about.
Not sure if the docker people would be ok with it though, they seem to suggest
putting the key in the Dockerfile
[https://github.com/docker-library/official-images#security]
{quote}
I'm fine with embedding the RM's key fingerprint in the Dockerfile since
docker-library seems to want that – but wouldn't it make more sense to download
the KEYS file direct from apache.org (to resolve that fingerprint to usable
key) then to rely on third-party key servers?
----
{quote}We need to change FROM $BASE_IMAGE to FROM <BASE_IMAGE>. The Docker team
has scripts that go and check the FROMs to make sure that they are approved
images.
{quote}
Ok ... I thought the important thing was the the (effective) base image *USED*
had to be an official (approved) image ... not that they cared what the
*LITERAL* {{FROM XXXX}} line in the Dockerfile ... but poking around the source
code for bashbrew I see that it won't resolve/allow build {{$ARG}} syntax in
the FROM line – and based on the 'TODO' comment above that logic it's
intentional and they just haven't added a specific error message about it yet...
[https://github.com/docker-library/bashbrew/blob/9df92226ae134cac637af04e6d466b82f0508c84/cmd/bashbrew/docker.go#L126]
...which puts another nail in the coffin of my idea to minimize the amount of
templating needed via build args. alas.
----
{quote}All of the other args should go after the FROM, I believe
{quote}
Any ARG that we want to use in multiple stages _with the same default_ has to
be pre-defined before the first FROM line (otherwise you have to redundently
declare the same default, and risk changing one later w/o remembering to change
all of them). I alluded to this in a comment...
{code:java}
# NOTE: All build args are pre-declared here (with applicable defaults), before
the first FROM line, for simplicity and
# documentation purposes.
#
# Each stage that needs one of these ARGs re-declares them (w/o any default,
ie: no '=') to inherit the default value.
{code}
> Dockerfile for local builds that can also serve as template for 'official'
> docker images
> ----------------------------------------------------------------------------------------
>
> Key: SOLR-15250
> URL: https://issues.apache.org/jira/browse/SOLR-15250
> Project: Solr
> Issue Type: Sub-task
> Reporter: Chris M. Hostetter
> Assignee: Chris M. Hostetter
> Priority: Major
> Attachments: SOLR-15250.patch
>
>
> This issue tracks PoC work experimenting with the idea of the following
> workflow:
> For Users:
> * a {{Dockerfile}} (or {{Dockerfile.local}}) in our git repo that can be
> used (directly or via gradle) to build docker images directly from a local
> solr.tgz (in the docker build context)
> For Release Manager:
> * The exact same {{Dockerfile}} serves as a "template" that gradle tasks use
> to generate a {{build/Dockerfile.official}} via some very simple
> substitutions to fill in ARG defaults based on the "official"
> solr-VERSION.tgz for this release
> * This {{Dockerfile.official}} can then be committed to the docker-solr
> github repo (or some similar new 9.x+ repo) and can be build with a a
> completely empty build context – in which it downloads (and validates) the
> official solr-VERSION.tgz (based on the ARG values that were filled in during
> the release)
> * Automated tests can help us "validate" that the generated
> {{Dockerfile.official}} will work _prior_ to officially publishing release
> artifacts, by using a "mock" download server to host the local
> solr-VERSION.tgz file
> The driving goal being that the Dockerfile used for official {{_/solr}}
> builds should be as close as possible to identical to the Dockerfile used for
> "local" builds by users – given the constraints put on us by the
> docker-library team.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
