[
https://issues.apache.org/jira/browse/HDDS-1495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16839804#comment-16839804
]
Eric Yang commented on HDDS-1495:
---------------------------------
{quote} a) As I wrote earler the statement is not true, the source of
hadoop-runner is part of hadoop repository as discussed on the mailing
list{quote}
Hadoop-runner is not building from hadoop-docker-ozone. Build.sh builds
apache/ozone:latest, not apache/hadoop-runner:latest. I am confused by your
statement above.
{quote} b) As discussed earlier it can be risky for all the downstream tests
where we need to set the classpath. This change should be very carefully
tested. And this is just not required because in containerized world the
version is defined by the container not the content.{quote}
We can continue to track CLASSPATH issue in HDDS-1520. I don't believe closing
it as can not reproduce is the right solution because some of the issues with
CLASSPATH bloat were identified and have solutions and recommendation in
HADOOP-7939, HADOOP-6255 and even earlier that I have lost track.
{quote} d) I think it's very important to define the scope of the provided
docker images. Until now it was clearly defined as developer tools and not for
production. I think for production use case most of the enterprise users have
own processes to create standard images (according to defined security
policies). To providing production-ready docker images are harder than removing
default sudo privileges (IMHO).{quote}
Hadoop already define developer tools by running start-build-env.sh in
HADOOP-11843. Attempt to redefine development process seems redundant.
Official Ozone image has been publishing to docker hub as official Ozone docker
build in HDDS-851. HDDS-1526 also intend to create Ozone 0.4.0 docker image
while maintain a separate code change and branch process from the 0.4.0 release
vote. This puts the release process at risk that we may have difficulty to
reproduce Ozone 0.4.0 docker image, when someone forget to tag multiple source
code repositories with the same version or the released code didn't get voted
on.
The goal is to fold [hadoop-docker-ozone
repository|https://github.com/apache/hadoop-docker-ozone] and
[hadoop-runner-latest
branch|https://github.com/apache/hadoop/tree/docker-hadoop-runner-latest] into
hadoop trunk. The consecutive versioning of Ozone releases can apply to docker
submodule in ozone-0.4 branches. I admit that I didn't like ozone-0.4 branch
in Hadoop source code, but I did not ask for separating repository. In my
view, a separate repository means a separate incubation project in Apache. I
did not predict that hadoop-docker-ozone repository to be created, but I would
be ok if hadoop-ozone repository contains all Ozone related code. I am
inclined to agree that ozone-0.4 branch is still far better than having
hadoop-docker-ozone repository or hadoop-runner-latest branch for now. [~elek]
Do we agree on bring those code into [ozone-0.4
branch|https://github.com/apache/hadoop/tree/ozone-0.4] and
[trunk|https://github.com/apache/hadoop/]?
> Create hadoop/ozone docker images with inline build process
> -----------------------------------------------------------
>
> Key: HDDS-1495
> URL: https://issues.apache.org/jira/browse/HDDS-1495
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
> Reporter: Elek, Marton
> Assignee: Eric Yang
> Priority: Major
> Attachments: HADOOP-16091.001.patch, HADOOP-16091.002.patch,
> HDDS-1495.003.patch, HDDS-1495.004.patch, HDDS-1495.005.patch,
> HDDS-1495.006.patch, HDDS-1495.007.patch, Hadoop Docker Image inline build
> process.pdf
>
>
> This is proposed by [~eyang] in
> [this|https://lists.apache.org/thread.html/33ac54bdeacb4beb023ebd452464603aaffa095bd104cb43c22f484e@%3Chdfs-dev.hadoop.apache.org%3E]
> mailing thread.
> {quote}1, 3. There are 38 Apache projects hosting docker images on Docker hub
> using Apache Organization. By browsing Apache github mirror. There are only 7
> projects using a separate repository for docker image build. Popular projects
> official images are not from Apache organization, such as zookeeper, tomcat,
> httpd. We may not disrupt what other Apache projects are doing, but it looks
> like inline build process is widely employed by majority of projects such as
> Nifi, Brooklyn, thrift, karaf, syncope and others. The situation seems a bit
> chaotic for Apache as a whole. However, Hadoop community can decide what is
> best for Hadoop. My preference is to remove ozone from source tree naming, if
> Ozone is intended to be subproject of Hadoop for long period of time. This
> enables Hadoop community to host docker images for various subproject without
> having to check out several source tree to trigger a grand build. However,
> inline build process seems more popular than separated process. Hence, I
> highly recommend making docker build inline if possible.
> {quote}
> The main challenges are also discussed in the thread:
> {code:java}
> 3. Technically it would be possible to add the Dockerfile to the source
> tree and publish the docker image together with the release by the
> release manager but it's also problematic:
> {code}
> a) there is no easy way to stage the images for the vote
> c) it couldn't be flagged as automated on dockerhub
> d) It couldn't support the critical updates.
> * Updating existing images (for example in case of an ssl bug, rebuild
> all the existing images with exactly the same payload but updated base
> image/os environment)
> * Creating image for older releases (We would like to provide images,
> for hadoop 2.6/2.7/2.7/2.8/2.9. Especially for doing automatic testing
> with different versions).
> {code:java}
> {code}
> The a) can be solved (as [~eyang] suggested) with using a personal docker
> image during the vote and publish it to the dockerhub after the vote (in case
> the permission can be set by the INFRA)
> Note: based on LEGAL-270 and linked discussion both approaches (inline build
> process / external build process) are compatible with the apache release.
> Note: HDDS-851 and HADOOP-14898 contains more information about these
> problems.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
