[
https://issues.apache.org/jira/browse/YARN-9244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16760294#comment-16760294
]
Suma Shivaprasad edited comment on YARN-9244 at 2/4/19 11:09 PM:
-----------------------------------------------------------------
Configure docker registry to use S3 storage driver with
/etc/docker/registry/config.yml having the following configs
{noformat}
version: 0.1
log:
fields:
service: registry
http:
addr: :5000
storage:
cache:
layerinfo: inmemory
s3:
accesskey: #AWS_KEY#
secretkey: #AWS_SECRET#
region: #AWS_REGION#
bucket: #AWS_BUCKET#
encrypt: #ENCRYPT#
secure: #SECURE#
chunksize: 5242880
multipartcopychunksize: 33554432
multipartcopymaxconcurrency: 100
multipartcopythresholdsize: 33554432
rootdirectory: #STORAGE_PATH#
{noformat}
docker run -e AWS_BUCKET=<BUCKET_NAME> -e AWS_KEY=accessKey -e
AWS_SECRET=secretKey -e AWS_REGION="aws-region" -e STORAGE_PATH=/registry
-e SECURE false -e ENCRYPT true -p 5000:5000 registry
{noformat}
time docker push localhost:5000/centos:latest
{noformat}
The push refers to repository [localhost:5000/centos]
071d8bd76517: Pushed
real 0m13.188s
user 0m0.085s
sys 0m0.007s
{noformat}
docker tag centos:latest localhost:5000/centos:latest
time docker pull localhost:5000/centos:latest
{noformat}
latest: Pulling from centos
a02a4930cb5d: Pull complete
Status: Downloaded newer image for localhost:5000/centos:latest
*real 0m2.918s*
user 0m0.048s
sys 0m0.023s
was (Author: suma.shivaprasad):
Configure docker registry to use S3 storage driver with
/etc/docker/registry/config.yml having the following configs
{noformat}
version: 0.1
log:
fields:
service: registry
http:
addr: :5000
storage:
cache:
layerinfo: inmemory
s3:
accesskey: #AWS_KEY#
secretkey: #AWS_SECRET#
region: #AWS_REGION#
bucket: #AWS_BUCKET#
encrypt: #ENCRYPT#
secure: #SECURE#
chunksize: 5242880
multipartcopychunksize: 33554432
multipartcopymaxconcurrency: 100
multipartcopythresholdsize: 33554432
rootdirectory: #STORAGE_PATH#
{noformat}
docker run -e AWS_BUCKET=<BUCKET_NAME> -e AWS_KEY=accessKey -e
AWS_SECRET=secretKey -e AWS_REGION="aws-region" -e STORAGE_PATH=/registry
-e SECURE false -e ENCRYPT true -p 5000:5000 registry
time docker push localhost:5000/centos:latest
The push refers to repository [localhost:5000/centos]
071d8bd76517: Pushed
real 0m13.188s
user 0m0.085s
sys 0m0.007s
{noformat}
docker tag centos:latest localhost:5000/centos:latest
time docker pull localhost:5000/centos:latest
{noformat}
latest: Pulling from centos
a02a4930cb5d: Pull complete
Status: Downloaded newer image for localhost:5000/centos:latest
*real 0m2.918s*
user 0m0.048s
sys 0m0.023s
> Document docker registry deployment with direct S3 driver
> ---------------------------------------------------------
>
> Key: YARN-9244
> URL: https://issues.apache.org/jira/browse/YARN-9244
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Eric Yang
> Assignee: Suma Shivaprasad
> Priority: Major
>
> On Amazon, it would be possible to deploy docker registry in YARN using
> Docker's own S3 driver. The same micro-benchmark can also be applied to show
> the docker registry performance difference. The sample YARN service
> configuration looks like this:
> {code}
> {
> "name": "docker-registry",
> "version": "1.0",
> "kerberos_principal" : {
> "principal_name" : "hbase/_h...@example.com",
> "keytab" : "file:///etc/security/keytabs/hbase.service.keytab"
> },
> "components" :
> [
> {
> "name": "registry",
> "number_of_containers": 1,
> "artifact": {
> "id": "registry:latest",
> "type": "DOCKER"
> },
> "resource": {
> "cpus": 1,
> "memory": "256"
> },
> "run_privileged_container": true,
> "configuration": {
> "env": {
> "YARN_CONTAINER_RUNTIME_DOCKER_RUN_OVERRIDE_DISABLE":"true",
>
> "YARN_CONTAINER_RUNTIME_DOCKER_MOUNTS":"/path/to/config.yml:/etc/docker/registry/config.yml"
> },
> "properties": {
> "docker.network": "host"
> }
> }
> }
> ]
> }
> {code}
> Where /path/to/config.yml maps to a Docker S3 storage option config.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
