Re: Docker Slaves plugin - how to specify volumes

['Archimedes Trajano' via Jenkins Users]

>
>
> We also considered to offer an option to cache some path, comparable to 
> travis-ci approach (based on tar.gz), so typically let end-user configure 
> maven local repository as "cached". Backed could rely on additional volumes 
> for this purpose, but definitively not on bind-mounts :P. This would let us 
> investigate scalability challenge to distribute such a cache, maybe just by 
> relying on docker volume-plugin features (but then relying on proprietary 
> API), or maybe by using jenkins as a volume replication facility, *à la* 
> TravisCI.
>
>
Is there an example of how to do caching of some folders with 
docker-slaves-plugin?

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/78b8fb05-015f-4b49-954a-5666ad88e949%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Docker Slaves plugin - how to specify volumes

[nicolas de loof]

The whole build agent root directory is managed as a docker volume, and 
retrieved from previous build, so you can easily benefit such caching if 
you configure maven to use adequate path for local repository (which should 
be the case by default, as /home/jenkins is slave root).

We also considered to offer an option to cache some path, comparable to 
travis-ci approach (based on tar.gz), so typically let end-user configure 
maven local repository as "cached". Backed could rely on additional volumes 
for this purpose, but definitively not on bind-mounts :P. This would let us 
investigate scalability challenge to distribute such a cache, maybe just by 
relying on docker volume-plugin features (but then relying on proprietary 
API), or maybe by using jenkins as a volume replication facility, *à la* 
TravisCI.

Le dimanche 16 octobre 2016 00:44:12 UTC+2, Qiang a écrit :
>
> Hi, Nicolas,
>
> One example I can think of:  for maven build, we want to cache the 
> downloaded dependencies so we don't have to download them every time.
>
> So in that case, I was thinking to cache on the host, or a data volume. 
> Any suggestions?
>
> Thanks,
> Qiang
>
> On Saturday, October 15, 2016 at 11:59:08 AM UTC-5, Qiang wrote:
>>
>>
>> Thank you for posting the response ! I do need to rethink my strategy 
>> now. Even though I need to solve legacy problems in a short term , I agree 
>> that it does not become a requirement :)
>>
>> So for moving data between containers , what options can I explore ? And 
>> is there instruction to use side containers ? I was not able to find much 
>> documentation.
>>
>> Regards,
>> Qiang
>>
>>
>>
>> Hi,
>>
>> There's no technical limitation to use this option (we actually already 
>> support it to allow docker.sock bind mount on a dedicated side container), 
>> but our experience is most user use this for bind-mount, which result in 
>> permission issues and in most case are just short terms workarounds to 
>> introduce bad practices. So we decided *not* to expose this option to 
>> end-user.
>>
>> IIUC your use case (from some private email) some legacy jobs you run 
>> rely on a NFS server to host project dependencies. Relying on a bind mount 
>> would make your build fragile and non-reproducible. You better should 
>> create a project specific docker image, to include those dependencies 
>> (assuming you can't update your build script to a modern dependency 
>> resolution approach).  
>> For sure this require some effort to migrate your builds, but I can't 
>> consider "*legacy bad practice*" as a valid use-case :P
>>
>>
>> Le vendredi 14 octobre 2016 15:55:01 UTC+2, Qiang a écrit :
>>>
>>> hi,
>>>
>>> How can I set container options to the Docker slave such as volume 
>>> settings? I need to mount a host directory to the container.
>>>
>>> So far, the only option I can see is to define the slave container 
>>> image, and side container image as following:
>>>
>>>   dockerNode(image: "maven:3.3.3-jdk-8", sideContainers: 
>>> ["selenium/standalone-firefox"]) 
>>>
>>> How do I pass "-v" or "--volumes-from" to the slave container?
>>>
>>> Regards,
>>> Qiang
>>>
>>>
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/f788cd3e-fd30-44aa-b2e8-34e20582827a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Docker Slaves plugin - how to specify volumes

[Qiang]

Hi, Nicolas,

One example I can think of:  for maven build, we want to cache the 
downloaded dependencies so we don't have to download them every time.

So in that case, I was thinking to cache on the host, or a data volume. Any 
suggestions?

Thanks,
Qiang

On Saturday, October 15, 2016 at 11:59:08 AM UTC-5, Qiang wrote:
>
>
> Thank you for posting the response ! I do need to rethink my strategy now. 
> Even though I need to solve legacy problems in a short term , I agree that 
> it does not become a requirement :)
>
> So for moving data between containers , what options can I explore ? And 
> is there instruction to use side containers ? I was not able to find much 
> documentation.
>
> Regards,
> Qiang
>
>
>
> Hi,
>
> There's no technical limitation to use this option (we actually already 
> support it to allow docker.sock bind mount on a dedicated side container), 
> but our experience is most user use this for bind-mount, which result in 
> permission issues and in most case are just short terms workarounds to 
> introduce bad practices. So we decided *not* to expose this option to 
> end-user.
>
> IIUC your use case (from some private email) some legacy jobs you run rely 
> on a NFS server to host project dependencies. Relying on a bind mount would 
> make your build fragile and non-reproducible. You better should create a 
> project specific docker image, to include those dependencies (assuming you 
> can't update your build script to a modern dependency resolution approach). 
>  
> For sure this require some effort to migrate your builds, but I can't 
> consider "*legacy bad practice*" as a valid use-case :P
>
>
> Le vendredi 14 octobre 2016 15:55:01 UTC+2, Qiang a écrit :
>>
>> hi,
>>
>> How can I set container options to the Docker slave such as volume 
>> settings? I need to mount a host directory to the container.
>>
>> So far, the only option I can see is to define the slave container image, 
>> and side container image as following:
>>
>>   dockerNode(image: "maven:3.3.3-jdk-8", sideContainers: 
>> ["selenium/standalone-firefox"]) 
>>
>> How do I pass "-v" or "--volumes-from" to the slave container?
>>
>> Regards,
>> Qiang
>>
>>
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/4b18770a-001b-46ce-a6d5-79ac8c9da698%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Docker Slaves plugin - how to specify volumes

[Qiang]

Thank you for posting the response ! I do need to rethink my strategy now. Even 
though I need to solve legacy problems in a short term , I agree that it does 
not become a requirement :)

So for moving data between containers , what options can I explore ? And is 
there instruction to use side containers ? I was not able to find much 
documentation.

Regards,
Qiang

> On Oct 15, 2016, at 11:18, nicolas de loof  wrote:
> 
> Hi,
> 
> There's no technical limitation to use this option (we actually already 
> support it to allow docker.sock bind mount on a dedicated side container), 
> but our experience is most user use this for bind-mount, which result in 
> permission issues and in most case are just short terms workarounds to 
> introduce bad practices. So we decided *not* to expose this option to 
> end-user.
> 
> IIUC your use case (from some private email) some legacy jobs you run rely on 
> a NFS server to host project dependencies. Relying on a bind mount would make 
> your build fragile and non-reproducible. You better should create a project 
> specific docker image, to include those dependencies (assuming you can't 
> update your build script to a modern dependency resolution approach).  
> For sure this require some effort to migrate your builds, but I can't 
> consider "legacy bad practice" as a valid use-case :P
> 
> 
> Le vendredi 14 octobre 2016 15:55:01 UTC+2, Qiang a écrit :
>> 
>> hi,
>> 
>> How can I set container options to the Docker slave such as volume settings? 
>> I need to mount a host directory to the container.
>> 
>> So far, the only option I can see is to define the slave container image, 
>> and side container image as following:
>> 
>>   dockerNode(image: "maven:3.3.3-jdk-8", sideContainers: 
>> ["selenium/standalone-firefox"]) 
>> 
>> How do I pass "-v" or "--volumes-from" to the slave container?
>> 
>> Regards,
>> Qiang
>> 
> 
> -- 
> You received this message because you are subscribed to a topic in the Google 
> Groups "Jenkins Users" group.
> To unsubscribe from this topic, visit 
> https://groups.google.com/d/topic/jenkinsci-users/voLK2zsKHDA/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to 
> jenkinsci-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/jenkinsci-users/06362a69-ead3-440f-a636-283707f541e2%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/99796C20-49F8-475B-866B-33C30763543D%40gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Docker Slaves plugin - how to specify volumes

[nicolas de loof]

Hi,

There's no technical limitation to use this option (we actually already 
support it to allow docker.sock bind mount on a dedicated side container), 
but our experience is most user use this for bind-mount, which result in 
permission issues and in most case are just short terms workarounds to 
introduce bad practices. So we decided *not* to expose this option to 
end-user.

IIUC your use case (from some private email) some legacy jobs you run rely 
on a NFS server to host project dependencies. Relying on a bind mount would 
make your build fragile and non-reproducible. You better should create a 
project specific docker image, to include those dependencies (assuming you 
can't update your build script to a modern dependency resolution approach). 
 
For sure this require some effort to migrate your builds, but I can't 
consider "*legacy bad practice*" as a valid use-case :P


Le vendredi 14 octobre 2016 15:55:01 UTC+2, Qiang a écrit :
>
> hi,
>
> How can I set container options to the Docker slave such as volume 
> settings? I need to mount a host directory to the container.
>
> So far, the only option I can see is to define the slave container image, 
> and side container image as following:
>
>   dockerNode(image: "maven:3.3.3-jdk-8", sideContainers: 
> ["selenium/standalone-firefox"]) 
>
> How do I pass "-v" or "--volumes-from" to the slave container?
>
> Regards,
> Qiang
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/06362a69-ead3-440f-a636-283707f541e2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.