[
https://issues.apache.org/jira/browse/HADOOP-16092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16765128#comment-16765128
]
Elek, Marton edited comment on HADOOP-16092 at 2/11/19 4:30 PM:
----------------------------------------------------------------
Thanks [~eyang] the comment.
It works together with HADOOP-16091. But it's independent from HADOOP-16091:
not all the containers can be created from maven. Eg. we have build/base-runner
images.
Until now there was a strong limitation on dockerhub. If you defined a
branch->dockertag mapping it was not possible to use the backref of the regular
expression.
Let's say we have the following branch-tag mapping for the same repository:
||Branch name||container tag||
|hadooprunner-(.*)|{sourceref}|
|hadoop-(.*)|{sourceref}|
With these settings, a hadoop-2.7.0 branch was used to create a docker image
with tag hadoop-2.7.0. And instead of apache/hadoop:2.7.0 we got an
apache/hadoop:hadoop-2.7.0. As a workaround we started to use fixed mapping
without regular expression but it made very hard to support multiple versions
(that's the reason why we have only hadoop:2 and hadoop:3).
I tested it again recently, and with the latest dockerhub version there is no
such limitation fortunately. Now we can also use the \{\1} ref.
||Branch name||container tag||
|hadooprunner-(.*)|{\1}|
|hadoop-(.*)|{\1}|
Long story short, with this improvement we can move all the ozone images (from
hadoop-docker-ozone repo) and hadoop images (from 3 branches of hadoop repo) to
the same dedicated repository (hadoop-docker).
hadoop-docker.it repository is not yet created, but if you create it with self
service (can be created only by members), I would be happy to move the existing
images to there under this jira.
was (Author: elek):
Thanks [~eyang] the comment.
It works together with HADOOP-16091. But it's independent from HADOOP-16091:
not all the containers can be created from maven. Eg. we have build/base-runner
images.
Until now there was a strong limitation on dockerhub. If you defined a
branch->dockertag mapping it was not possible to use the backref of the regular
expression.
Let's say we have the following branch-tag mapping for the same repository:
||Branch name||container tag||
|hadooprunner-(.*)|{sourceref}|
|ozonerunner-(.*)|{sourceref}|
|hadoop-(.*)|{sourceref}|
|ozone-(.*)|{sourceref}|
With these settings, a hadoop-2.7.0 branch was used to create a docker image
with tag hadoop-2.7.0. And instead of apache/hadoop:2.7.0 we got an
apache/hadoop:hadoop-2.7.0. As a workaround we started to use fixed mapping
without regular expression but it made very hard to support multiple versions
(that's the reason why we have only hadoop:2 and hadoop:3).
I tested it again recently, and with the latest dockerhub version there is no
such limitation fortunately. Now we can also use the \{\1} ref.
||Branch name||container tag||final tag||
|hadooprunner-(.*)|{\1}|
|ozonerunner-(.*)|{\1}|
|hadoop-(.*)|{\1}|
|ozone-(.*)|{\1}|
Long story short, with this improvement we can move all the ozone images (from
hadoop-docker-ozone repo) and hadoop images (from 3 branches of hadoop repo) to
the same dedicated repository (hadoop-docker).
hadoop-docker repository is not yet created, but if you create it with self
service, I would be happy to move the existing images to there under this jira.
> Move the source of hadoop/ozone containers to the same repository
> -----------------------------------------------------------------
>
> Key: HADOOP-16092
> URL: https://issues.apache.org/jira/browse/HADOOP-16092
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Elek, Marton
> Priority: Major
>
> This is proposed by [~eyang] in
> [this|https://lists.apache.org/thread.html/33ac54bdeacb4beb023ebd452464603aaffa095bd104cb43c22f484e@%3Chdfs-dev.hadoop.apache.org%3E]
> mailing thread.
> bq. Hadoop community can decide what is best for Hadoop. My preference is to
> remove ozone from source tree naming, if Ozone is intended to be subproject
> of Hadoop for long period of time. This enables Hadoop community to host
> docker images for various subproject without having to check out several
> source tree to trigger a grand build
> As of now the source of hadoop docker images are stored in the hadoop git
> repository (docker-* branches) for hadoop and in hadoop-docker-ozone git
> repository for ozone (all branches).
> As it's discussed in HDDS-851 the biggest challenge to solve here is the
> mapping between git branches and dockerhub tags. It's not possible to use the
> captured part of a github branch.
> For example it's not possible to define a rule to build all the ozone-(.*)
> branches and use a tag $1 for it. Without this support we need to create a
> new mapping for all the releases manually (with the help of the INFRA).
> Note: HADOOP-16091 can solve this problem as it doesn't require branch
> mapping any more.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
