[
https://issues.apache.org/jira/browse/HDDS-1495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16840870#comment-16840870
]
Craig Condit commented on HDDS-1495:
------------------------------------
I realize I'm probably a little late to this party, but maybe that means I'll
have a fresh perspective on this...
I think it bears looking at what the purpose of the various Docker images we're
creating are. First, we have the Docker image which is created by
_*start-build-env.sh*_ in the root of the Hadoop project. This is extremely
useful for ensuring a consisten_t *build*_ ** environment, but is not suitable
for public distribution. Additionally, I think there are valid uses for other
images which include local testing, integration testing, pre-commit testing,
etc. Finally, we have public images which are meant for end users to build
upon.**
How exactly each of these images is created is IMO not that important – this
should probably be dictated by those who are acting as release managers. As a
local developer, I'd prefer a solution that doesn't involve automatic creation
of downstream images on every build (even on dist builds). I think using
something *-Pdocker* to activate building is fine, as it's opt-in rather than
opt-out.
All of the above applies to Hadoop proper, so now for the Ozone-specific bits...
Since Ozone is meant to run as a plugin to an existing Hadoop installation, I
think it's worth considering publishing multiple images with different
underlying Hadoop base versions. Today this might include:
* ozone:0.4-hadoop-3.0.3
* ozone:0.4-hadoop-3.1.2
* ozone:0.4-hadoop-3.2.0
These could serve as reference implementations, suitable for publishing on
DockerHub,etc. (along with associated Dockerfiles). Distributions and users who
are more security conscious would probably rebuild based on security patches in
underlying OS, etc. This pattern tracks closely with how many other complex
open source projects release images (even openjdk typically comes in several
different flavors (alpine, ubuntu, slim, etc.).
> Create hadoop/ozone docker images with inline build process
> -----------------------------------------------------------
>
> Key: HDDS-1495
> URL: https://issues.apache.org/jira/browse/HDDS-1495
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
> Reporter: Elek, Marton
> Assignee: Eric Yang
> Priority: Major
> Attachments: HADOOP-16091.001.patch, HADOOP-16091.002.patch,
> HDDS-1495.003.patch, HDDS-1495.004.patch, HDDS-1495.005.patch,
> HDDS-1495.006.patch, HDDS-1495.007.patch, Hadoop Docker Image inline build
> process.pdf
>
>
> This is proposed by [~eyang] in
> [this|https://lists.apache.org/thread.html/33ac54bdeacb4beb023ebd452464603aaffa095bd104cb43c22f484e@%3Chdfs-dev.hadoop.apache.org%3E]
> mailing thread.
> {quote}1, 3. There are 38 Apache projects hosting docker images on Docker hub
> using Apache Organization. By browsing Apache github mirror. There are only 7
> projects using a separate repository for docker image build. Popular projects
> official images are not from Apache organization, such as zookeeper, tomcat,
> httpd. We may not disrupt what other Apache projects are doing, but it looks
> like inline build process is widely employed by majority of projects such as
> Nifi, Brooklyn, thrift, karaf, syncope and others. The situation seems a bit
> chaotic for Apache as a whole. However, Hadoop community can decide what is
> best for Hadoop. My preference is to remove ozone from source tree naming, if
> Ozone is intended to be subproject of Hadoop for long period of time. This
> enables Hadoop community to host docker images for various subproject without
> having to check out several source tree to trigger a grand build. However,
> inline build process seems more popular than separated process. Hence, I
> highly recommend making docker build inline if possible.
> {quote}
> The main challenges are also discussed in the thread:
> {code:java}
> 3. Technically it would be possible to add the Dockerfile to the source
> tree and publish the docker image together with the release by the
> release manager but it's also problematic:
> {code}
> a) there is no easy way to stage the images for the vote
> c) it couldn't be flagged as automated on dockerhub
> d) It couldn't support the critical updates.
> * Updating existing images (for example in case of an ssl bug, rebuild
> all the existing images with exactly the same payload but updated base
> image/os environment)
> * Creating image for older releases (We would like to provide images,
> for hadoop 2.6/2.7/2.7/2.8/2.9. Especially for doing automatic testing
> with different versions).
> {code:java}
> {code}
> The a) can be solved (as [~eyang] suggested) with using a personal docker
> image during the vote and publish it to the dockerhub after the vote (in case
> the permission can be set by the INFRA)
> Note: based on LEGAL-270 and linked discussion both approaches (inline build
> process / external build process) are compatible with the apache release.
> Note: HDDS-851 and HADOOP-14898 contains more information about these
> problems.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
