Re: Amazon ECR Plugin not providing credentials for Docker Build & Push Plugin

[A. Best]

 

Just so that I can bring this thread to a proper conclusion, I worked out 
the issue.


*TL;DR:*

The Jenkins slave container derived from the jnlp-slave image (with the 
docker client installed) was using the *root* user instead of the *jenkins* 
user. This causes the ECR credentials to be stored in the wrong place. So, 
when the *docker-build-publish* plugin pushes to the registry, docker push 
.dkr.ecr.us-east-1.amazonaws.com/:latest, there is no docker 
config file with the proper credentials. This causes the no basic auth 
credentials error.


*Recap:*

I am using a Jenkins master to trigger builds in an ECS cloud. The ECS 
slave task template uses an image derived from the jnlp-slave image with 
the docker client added. Additionally, /var/run/docker.sock is mounted from 
the source to the container to give the slave container access to the 
docker host’s docker server.

The goal of this configuration is to provide a simple way for jnlp-worker 
containers to build & push docker images to a registry. The physical 
configuration is as follows:





*So, what was the source of the problem?*

Well, initially I was having problems with the *jenkins* user accessing 
/var/run/docker.sock. The socket belongs to the *docker* group on the host 
and is assigned an random? GID. The *docker* group, however, was 1) not a 
group in the container and 2) the *jenkins* user was not a member of the 
group. So, I copped out and had the container run as *root*–laziness 
invites issues.

The jenkins worker will hum along properly until it’s time to docker push 
to the registry and it cannot authenticate. The *Docker Build and Publish* 
plugin does correctly utilize the ECR plugin to retrieve a token to access 
the ECR registry. But, because this is all happening as the *root* user, 
the *Docker Commons* plugin stores the resultant login info at 
/root/.dockercfg. When docker push is invoked by the plugin, it can’t find 
credentials…booo.


*A resolution*

The resolution is simple, ensure that the jnlp worker if running as the 
*jenkins* user and ensure that the *docker* group from the host is 
replicated in the worker. A bit of searching led me to this post on the 
docker forums 
 and 
this script 

 
by Sven Dowideit. With a few modifications to use this script as the 
ENTRYPOINT of the jnlp worker image, everything now works.


--

a

On Tuesday, April 12, 2016 at 11:15:38 AM UTC-4, A. Best wrote:
>
> Nicolas,
>
> Thanks for the response.
>
> Yes, I did select amazon credentials, as those are the only credentials I 
> have setup on this instance.  In this particular case I was doing a variation 
> of the a Jenkins-Amazon build pipeline 
> .
>   
> I was using a simple prebuilt source repo to test the process. 
>
> Here's a screenshot of the *Docker Build and Publish* build step.
>
>
> 
>
> Thanks,
>
>
> On Monday, April 11, 2016 at 6:42:38 PM UTC-4, nicolas de loof wrote:
>>
>> Did you well selected amazon credentials in build and publish build step 
>> configuration ?
>>
>> 2016-04-11 22:53 GMT+02:00 A. Best :
>>
>>> *Versions used in testing:*
>>> Jenkins: 1.642.3
>>> Amazon ECR Plugin: 1.0 
>>> 
>>> Cloudbees Docker Build & Publish: 1.2.1 
>>> 
>>>
>>> *ECS Container Instance*
>>> Docker Version: 1.9.1
>>> API Version: 1.21
>>>
>>>
>>> I have a couple of quick question about the amazon-ecr-plugin 
>>> .
>>>
>>> In the About 
>>>  
>>> section 
>>> of the plugin, the Cloudbees Docker Build and Publish 
>>> 
>>>  is 
>>> referenced as an example of how the ECR plugin can be used.
>>>
>>> For my specific use case, I have the Jenkins master connecting to a 
>>> Jenkins JNLP slave running in an ECS cluster. I’m using a container based 
>>> on the jenkinsci/jnlp-slave 
>>>  to perform the build.  
>>> With the right permissions and mounted volumes, I am able to use the docker 
>>> host(which is the ECS container instance) to build docker images.
>>>
>>>
>>> 

Re: Amazon ECR Plugin not providing credentials for Docker Build & Push Plugin

[A. Best]

Nicolas,

Thanks for the response.

Yes, I did select amazon credentials, as those are the only credentials I 
have setup on this instance.  In this particular case I was doing a variation 
of the a Jenkins-Amazon build pipeline 
.
  
I was using a simple prebuilt source repo to test the process. 

Here's a screenshot of the *Docker Build and Publish* build step.



Thanks,


On Monday, April 11, 2016 at 6:42:38 PM UTC-4, nicolas de loof wrote:
>
> Did you well selected amazon credentials in build and publish build step 
> configuration ?
>
> 2016-04-11 22:53 GMT+02:00 A. Best :
>
>> *Versions used in testing:*
>> Jenkins: 1.642.3
>> Amazon ECR Plugin: 1.0 
>> 
>> Cloudbees Docker Build & Publish: 1.2.1 
>> 
>>
>> *ECS Container Instance*
>> Docker Version: 1.9.1
>> API Version: 1.21
>>
>>
>> I have a couple of quick question about the amazon-ecr-plugin 
>> .
>>
>> In the About 
>>  
>> section 
>> of the plugin, the Cloudbees Docker Build and Publish 
>> 
>>  is 
>> referenced as an example of how the ECR plugin can be used.
>>
>> For my specific use case, I have the Jenkins master connecting to a 
>> Jenkins JNLP slave running in an ECS cluster. I’m using a container based 
>> on the jenkinsci/jnlp-slave 
>>  to perform the build.  
>> With the right permissions and mounted volumes, I am able to use the docker 
>> host(which is the ECS container instance) to build docker images.
>>
>>
>> 
>>
>>
>> The *Docker Build and Publish* plugin does use the dockerfile at the 
>> root of the project and build it as expected. However, I’m running into an 
>> issue when the plugin attempts to push the image to ECR.
>>
>> The push refers to a repository 
>> [.dkr.ecr.us-east-1.amazonaws.com/ 
>> test-repository] (len: 1)7a8e1872c5e2: 
>> Preparing
>> Post 
>> https://.dkr.ecr.us-east-1.amazonaws.com/v2/test-repository/blobs/uploads/
>>  : no basic auth 
>> credentials
>> Build step 'Docker Build and Publish' marked build as failure
>> Finished: FAILURE
>>
>> It seems that the build is attempting to push to the registry with no 
>> credentials. I was assuming that the ECR plugin would provide docker with 
>> the correct AWS credentials to login to the registry so that the newly 
>> built image could be pushed.  
>>
>> Do I need any additional packages installed on the Jenkins slave to get 
>> this to work?
>>
>> Am I missing something?
>>
>> Are my expectations for the plugin wrong?
>>
>>
>> Thanks, 
>>
>> Adam
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Jenkins Users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to jenkinsci-use...@googlegroups.com .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/jenkinsci-users/eaf6f71a-9758-4838-bb05-fd4fa43ee021%40googlegroups.com
>>  
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/8abb2389-eb6b-4386-9912-3ce2d817d168%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Amazon ECR Plugin not providing credentials for Docker Build & Push Plugin

[nicolas de loof]

Did you well selected amazon credentials in build and publish build step
configuration ?

2016-04-11 22:53 GMT+02:00 A. Best :

> *Versions used in testing:*
> Jenkins: 1.642.3
> Amazon ECR Plugin: 1.0
> 
> Cloudbees Docker Build & Publish: 1.2.1
> 
>
> *ECS Container Instance*
> Docker Version: 1.9.1
> API Version: 1.21
>
>
> I have a couple of quick question about the amazon-ecr-plugin
> .
>
> In the About
>  
> section
> of the plugin, the Cloudbees Docker Build and Publish
> 
>  is
> referenced as an example of how the ECR plugin can be used.
>
> For my specific use case, I have the Jenkins master connecting to a
> Jenkins JNLP slave running in an ECS cluster. I’m using a container based
> on the jenkinsci/jnlp-slave
>  to perform the build.
> With the right permissions and mounted volumes, I am able to use the docker
> host(which is the ECS container instance) to build docker images.
>
>
> 
>
>
> The *Docker Build and Publish* plugin does use the dockerfile at the root
> of the project and build it as expected. However, I’m running into an issue
> when the plugin attempts to push the image to ECR.
>
> The push refers to a repository 
> [.dkr.ecr.us-east-1.amazonaws.com/ 
> test-repository] (len: 1)7a8e1872c5e2: 
> Preparing
> Post 
> https://.dkr.ecr.us-east-1.amazonaws.com/v2/test-repository/blobs/uploads/
>  : no basic auth 
> credentials
> Build step 'Docker Build and Publish' marked build as failure
> Finished: FAILURE
>
> It seems that the build is attempting to push to the registry with no
> credentials. I was assuming that the ECR plugin would provide docker with
> the correct AWS credentials to login to the registry so that the newly
> built image could be pushed.
>
> Do I need any additional packages installed on the Jenkins slave to get
> this to work?
>
> Am I missing something?
>
> Are my expectations for the plugin wrong?
>
>
> Thanks,
>
> Adam
>
> --
> You received this message because you are subscribed to the Google Groups
> "Jenkins Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jenkinsci-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jenkinsci-users/eaf6f71a-9758-4838-bb05-fd4fa43ee021%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/CANMVJzkbcHPGFmQD%3DWknEaKQf3%2BpM7KLmnk9FPVs0Qy_pNzPSA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Amazon ECR Plugin not providing credentials for Docker Build & Push Plugin

[A. Best]

*Versions used in testing:*
Jenkins: 1.642.3
Amazon ECR Plugin: 1.0 

Cloudbees Docker Build & Publish: 1.2.1 


*ECS Container Instance*
Docker Version: 1.9.1
API Version: 1.21


I have a couple of quick question about the amazon-ecr-plugin 
.

In the About 
 
section 
of the plugin, the Cloudbees Docker Build and Publish 

 is 
referenced as an example of how the ECR plugin can be used.

For my specific use case, I have the Jenkins master connecting to a Jenkins 
JNLP slave running in an ECS cluster. I’m using a container based on the 
jenkinsci/jnlp-slave  to 
perform the build.  With the right permissions and mounted volumes, I am 
able to use the docker host(which is the ECS container instance) to build 
docker images.




The *Docker Build and Publish* plugin does use the dockerfile at the root 
of the project and build it as expected. However, I’m running into an issue 
when the plugin attempts to push the image to ECR.

The push refers to a repository [.dkr.ecr.us-east-1.amazonaws.com/ 
test-repository] (len: 1)7a8e1872c5e2: 
Preparing
Post 
https://.dkr.ecr.us-east-1.amazonaws.com/v2/test-repository/blobs/uploads/
 : no basic auth 
credentials
Build step 'Docker Build and Publish' marked build as failure
Finished: FAILURE

It seems that the build is attempting to push to the registry with no 
credentials. I was assuming that the ECR plugin would provide docker with 
the correct AWS credentials to login to the registry so that the newly 
built image could be pushed.  

Do I need any additional packages installed on the Jenkins slave to get 
this to work?

Am I missing something?

Are my expectations for the plugin wrong?


Thanks, 

Adam

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/eaf6f71a-9758-4838-bb05-fd4fa43ee021%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.