Re: One-Shot Executors

[nicolas de loof]

we want to distinguish error due to job misconfiguration (bad docker image
ID for sample) and infrastructure issue (no available resources)

2016-03-10 19:58 GMT+01:00 Suckow, Thomas J :

>
>
> On 3/9/16, 12:40 PM, "jenkinsci-dev@googlegroups.com on behalf of Jesse
> Glick" 
> wrote:
> >
> >Or just fail the build and the next one should work.
>
> I just don't want to have to kick builds when a normal slave would "just
> wait". Our cloud is also far from 99.9% uptime.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Jenkins Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jenkinsci-dev+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jenkinsci-dev/D30701C3.2858A%25thomas.suckow%40pnnl.gov
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CANMVJzk%2BaX0-1te7s8hg5_xnDE5o%3D0Gtmvkcqy40v7Q%2Bofkj1w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

OT - was Re: One-Shot Executors

[James Nord]

>
> Could also be interesting if you are shutting down Jenkins and don't wan't 
> to wait for a build to complete

 
use pipeline and then your builds will then mostly survive Jenkins restarts.

Or there is a proprietary restart aborted builds 
<https://www.cloudbees.com/products/cloudbees-jenkins-platform/enterprise-edition/features/restart-aborted-builds-plugin>
 
plugin that will restart builds that where aborted on Jenkins startup.

On Thursday, March 10, 2016 at 7:09:41 PM UTC, Thomas Suckow wrote:
>
> It would be an interesting feature if a build could be returned to the 
> queue. I would actually like that in the case the slave disappeared during 
> the build, I've had that happen a couple times over the last few years. 
> Could also be interesting if you are shutting down jenkins and don't wan't 
> to wait for a build to complete you could abort it and return it to the 
> queue.
>
> I think in the race condition case, as long as only a handful make it into 
> the run state you won't be pounding the server saying "give me a slice." We 
> have some large (to us) matrix builds and it probably isn't good to have 
> 20-30 jobs all trying to get a cut of the pie.
>
> From: <jenkin...@googlegroups.com > on behalf of nicolas de 
> loof <nicolas...@gmail.com >
> Reply-To: "jenkin...@googlegroups.com " <
> jenkin...@googlegroups.com >
> Date: Thursday, March 10, 2016 at 1:46 AM
> To: "jenkin...@googlegroups.com " <jenkin...@googlegroups.com 
> >
> Subject: Re: One-Shot Executors
>
> please see 
> https://github.com/jenkinsci/one-shot-executor-plugin/commit/86c632091391e3204432a8c7d1a18bb48b3a66e3
>  
>
> basically, this on let the provisionner keep a task in the Queue when it 
> can determine it hasn't enough resources to run extra slaves.
> this can be used to prevent overload, or allocate some extra physical 
> nodes.
>
> 2016-03-09 23:32 GMT+01:00 nicolas de loof <nicolas...@gmail.com 
> >:
>
>> " 
>> I do agree that certain issues should fail immediately (image not found). 
>> Certain other issues should perform exponential backoff (Cloud 
>> infrastructure down). Provisioning limits could be annoying though, would 
>> be interesting if they could be left in the queue until Jenkins side 
>> provisioning limits are not violated. I am not sure how to handle an 
>> environment like Kubernetes though where other entities may be utilizing 
>> resources and you have to "share". 
>> "
>>
>> This is something we have in mind. Provisioner could wait for available 
>> resources before it creates a Slave, leaving the task in the queue with a 
>> LabelAssignment waiting for matching executor. Would anyway need to let the 
>> Run start *then* create the slave, which means some race condition could 
>> appear and the required resources aren't available for this Slave to start 
>> even they were, few ms before - or we need some way to reserve resources on 
>> the infra, which then would significantly limit the available 
>> implementations. Maybe then we could cancel the Run, as we run early in 
>> it's lifecycle, and re-schedule it as a task in the Queue, claiming the Run 
>> never existed ? 
>>
>> 2016-03-09 21:40 GMT+01:00 Jesse Glick <jgl...@cloudbees.com 
>> >:
>>
>>> On Wed, Mar 9, 2016 at 11:52 AM, Suckow, Thomas J
>>> <thomas...@pnnl.gov > wrote:
>>> > Certain other issues should perform exponential backoff (Cloud
>>> > infrastructure down).
>>>
>>> Or just fail the build and the next one should work.
>>>
>>> > It could also handle the logic of some users wanting to configure 
>>> slaves on
>>> > a per job basis.
>>>
>>> If you mean “configure Docker images on a per-job basis”, this is
>>> addressed by both the Docker Custom Build Environment and Docker
>>> Pipeline plugins, both of which ought to move toward using this new
>>> infrastructure.
>>>
>>> --
>>> You received this message because you are subscribed to the Google 
>>> Groups "Jenkins Developers" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to jenkinsci-de...@googlegroups.com .
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr3ycEVavhfm2Kuj5pOB5fkPF6vaOJfb6Zu-89y%2BQRm95g%40mail.gmail.com
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>
> -- 
> You received this message because you are subscri

Re: One-Shot Executors

[Suckow, Thomas J]

It would be an interesting feature if a build could be returned to the queue. I 
would actually like that in the case the slave disappeared during the build, 
I've had that happen a couple times over the last few years. Could also be 
interesting if you are shutting down jenkins and don't wan't to wait for a 
build to complete you could abort it and return it to the queue.

I think in the race condition case, as long as only a handful make it into the 
run state you won't be pounding the server saying "give me a slice." We have 
some large (to us) matrix builds and it probably isn't good to have 20-30 jobs 
all trying to get a cut of the pie.

From: <jenkinsci-dev@googlegroups.com<mailto:jenkinsci-dev@googlegroups.com>> 
on behalf of nicolas de loof 
<nicolas.del...@gmail.com<mailto:nicolas.del...@gmail.com>>
Reply-To: 
"jenkinsci-dev@googlegroups.com<mailto:jenkinsci-dev@googlegroups.com>" 
<jenkinsci-dev@googlegroups.com<mailto:jenkinsci-dev@googlegroups.com>>
Date: Thursday, March 10, 2016 at 1:46 AM
To: "jenkinsci-dev@googlegroups.com<mailto:jenkinsci-dev@googlegroups.com>" 
<jenkinsci-dev@googlegroups.com<mailto:jenkinsci-dev@googlegroups.com>>
Subject: Re: One-Shot Executors

please see 
https://github.com/jenkinsci/one-shot-executor-plugin/commit/86c632091391e3204432a8c7d1a18bb48b3a66e3

basically, this on let the provisionner keep a task in the Queue when it can 
determine it hasn't enough resources to run extra slaves.
this can be used to prevent overload, or allocate some extra physical nodes.

2016-03-09 23:32 GMT+01:00 nicolas de loof 
<nicolas.del...@gmail.com<mailto:nicolas.del...@gmail.com>>:
"
I do agree that certain issues should fail immediately (image not found). 
Certain other issues should perform exponential backoff (Cloud infrastructure 
down). Provisioning limits could be annoying though, would be interesting if 
they could be left in the queue until Jenkins side provisioning limits are not 
violated. I am not sure how to handle an environment like Kubernetes though 
where other entities may be utilizing resources and you have to "share".
"

This is something we have in mind. Provisioner could wait for available 
resources before it creates a Slave, leaving the task in the queue with a 
LabelAssignment waiting for matching executor. Would anyway need to let the Run 
start then create the slave, which means some race condition could appear and 
the required resources aren't available for this Slave to start even they were, 
few ms before - or we need some way to reserve resources on the infra, which 
then would significantly limit the available implementations. Maybe then we 
could cancel the Run, as we run early in it's lifecycle, and re-schedule it as 
a task in the Queue, claiming the Run never existed ?

2016-03-09 21:40 GMT+01:00 Jesse Glick 
<jgl...@cloudbees.com<mailto:jgl...@cloudbees.com>>:
On Wed, Mar 9, 2016 at 11:52 AM, Suckow, Thomas J
<thomas.suc...@pnnl.gov<mailto:thomas.suc...@pnnl.gov>> wrote:
> Certain other issues should perform exponential backoff (Cloud
> infrastructure down).

Or just fail the build and the next one should work.

> It could also handle the logic of some users wanting to configure slaves on
> a per job basis.

If you mean “configure Docker images on a per-job basis”, this is
addressed by both the Docker Custom Build Environment and Docker
Pipeline plugins, both of which ought to move toward using this new
infrastructure.

--
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to 
jenkinsci-dev+unsubscr...@googlegroups.com<mailto:jenkinsci-dev%2bunsubscr...@googlegroups.com>.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr3ycEVavhfm2Kuj5pOB5fkPF6vaOJfb6Zu-89y%2BQRm95g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.



--
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to 
jenkinsci-dev+unsubscr...@googlegroups.com<mailto:jenkinsci-dev+unsubscr...@googlegroups.com>.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CANMVJz%3DQNuogSJvuUMWR9RbjczW1fAd8pAJD9aa0Lk0H5aFwAQ%40mail.gmail.com<https://groups.google.com/d/msgid/jenkinsci-dev/CANMVJz%3DQNuogSJvuUMWR9RbjczW1fAd8pAJD9aa0Lk0H5aFwAQ%40mail.gmail.com?utm_medium=email_source=footer>.
For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from i

Re: One-Shot Executors

[Suckow, Thomas J]

On 3/9/16, 12:40 PM, "jenkinsci-dev@googlegroups.com on behalf of Jesse
Glick" 
wrote:
>
>Or just fail the build and the next one should work.

I just don't want to have to kick builds when a normal slave would "just
wait". Our cloud is also far from 99.9% uptime.

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/D30701C3.2858A%25thomas.suckow%40pnnl.gov.
For more options, visit https://groups.google.com/d/optout.

Re: One-Shot Executors

[nicolas de loof]

please see
https://github.com/jenkinsci/one-shot-executor-plugin/commit/86c632091391e3204432a8c7d1a18bb48b3a66e3

basically, this on let the provisionner keep a task in the Queue when it
can determine it hasn't enough resources to run extra slaves.
this can be used to prevent overload, or allocate some extra physical nodes.

2016-03-09 23:32 GMT+01:00 nicolas de loof :

> "
> I do agree that certain issues should fail immediately (image not found).
> Certain other issues should perform exponential backoff (Cloud
> infrastructure down). Provisioning limits could be annoying though, would
> be interesting if they could be left in the queue until Jenkins side
> provisioning limits are not violated. I am not sure how to handle an
> environment like Kubernetes though where other entities may be utilizing
> resources and you have to "share".
> "
>
> This is something we have in mind. Provisioner could wait for available
> resources before it creates a Slave, leaving the task in the queue with a
> LabelAssignment waiting for matching executor. Would anyway need to let the
> Run start *then* create the slave, which means some race condition could
> appear and the required resources aren't available for this Slave to start
> even they were, few ms before - or we need some way to reserve resources on
> the infra, which then would significantly limit the available
> implementations. Maybe then we could cancel the Run, as we run early in
> it's lifecycle, and re-schedule it as a task in the Queue, claiming the Run
> never existed ?
>
> 2016-03-09 21:40 GMT+01:00 Jesse Glick :
>
>> On Wed, Mar 9, 2016 at 11:52 AM, Suckow, Thomas J
>>  wrote:
>> > Certain other issues should perform exponential backoff (Cloud
>> > infrastructure down).
>>
>> Or just fail the build and the next one should work.
>>
>> > It could also handle the logic of some users wanting to configure
>> slaves on
>> > a per job basis.
>>
>> If you mean “configure Docker images on a per-job basis”, this is
>> addressed by both the Docker Custom Build Environment and Docker
>> Pipeline plugins, both of which ought to move toward using this new
>> infrastructure.
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Jenkins Developers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to jenkinsci-dev+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr3ycEVavhfm2Kuj5pOB5fkPF6vaOJfb6Zu-89y%2BQRm95g%40mail.gmail.com
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CANMVJz%3DQNuogSJvuUMWR9RbjczW1fAd8pAJD9aa0Lk0H5aFwAQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: One-Shot Executors

[nicolas de loof]

"
I do agree that certain issues should fail immediately (image not found).
Certain other issues should perform exponential backoff (Cloud
infrastructure down). Provisioning limits could be annoying though, would
be interesting if they could be left in the queue until Jenkins side
provisioning limits are not violated. I am not sure how to handle an
environment like Kubernetes though where other entities may be utilizing
resources and you have to "share".
"

This is something we have in mind. Provisioner could wait for available
resources before it creates a Slave, leaving the task in the queue with a
LabelAssignment waiting for matching executor. Would anyway need to let the
Run start *then* create the slave, which means some race condition could
appear and the required resources aren't available for this Slave to start
even they were, few ms before - or we need some way to reserve resources on
the infra, which then would significantly limit the available
implementations. Maybe then we could cancel the Run, as we run early in
it's lifecycle, and re-schedule it as a task in the Queue, claiming the Run
never existed ?

2016-03-09 21:40 GMT+01:00 Jesse Glick :

> On Wed, Mar 9, 2016 at 11:52 AM, Suckow, Thomas J
>  wrote:
> > Certain other issues should perform exponential backoff (Cloud
> > infrastructure down).
>
> Or just fail the build and the next one should work.
>
> > It could also handle the logic of some users wanting to configure slaves
> on
> > a per job basis.
>
> If you mean “configure Docker images on a per-job basis”, this is
> addressed by both the Docker Custom Build Environment and Docker
> Pipeline plugins, both of which ought to move toward using this new
> infrastructure.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Jenkins Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jenkinsci-dev+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr3ycEVavhfm2Kuj5pOB5fkPF6vaOJfb6Zu-89y%2BQRm95g%40mail.gmail.com
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CANMVJzke%3D_zx5%2BupiXp7ZBwnzwCLmvcnnUr-Hms%2BvGJ9ii13tg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: One-Shot Executors

[Jesse Glick]

On Wed, Mar 9, 2016 at 11:52 AM, Suckow, Thomas J
 wrote:
> Certain other issues should perform exponential backoff (Cloud
> infrastructure down).

Or just fail the build and the next one should work.

> It could also handle the logic of some users wanting to configure slaves on
> a per job basis.

If you mean “configure Docker images on a per-job basis”, this is
addressed by both the Docker Custom Build Environment and Docker
Pipeline plugins, both of which ought to move toward using this new
infrastructure.

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr3ycEVavhfm2Kuj5pOB5fkPF6vaOJfb6Zu-89y%2BQRm95g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: One-Shot Executors

[Kanstantsin Shautsou]

> On Mar 9, 2016, at 19:52, Suckow, Thomas J <thomas.suc...@pnnl.gov> wrote:
> 
> I like that this moves the provisioning to the build log.
> 
> I do agree that certain issues should fail immediately (image not found). 
> Certain other issues should perform exponential backoff (Cloud infrastructure 
> down). Provisioning limits could be annoying though, would be interesting if 
> they could be left in the queue until Jenkins side provisioning limits are 
> not violated. I am not sure how to handle an environment like Kubernetes 
> though where other entities may be utilizing resources and you have to 
> "share".
> 
> You mention using labels to pick the slave. I'm wondering if it would be 
> feasible and worthwhile to make such a plugin generic to be the middle layer 
> for the Jenkins hooks to the cloud specific implementation (Docker, 
> Kubernetes, AWS).
> 
> It could also handle the logic of some users wanting to configure slaves on a 
> per job basis. Would be interesting if could also be integrated into 
> cloudbees folder level. If the later could work then I wouldn't need to run 
> my own Jenkins install at work for using containers and instead could use the 
> company cloudbees Jenkins.
I guess folder has some FolderProperty that you cat get from job.getParent(), 
that’s all.
> 
> -
> Thomas
> 
> From: <jenkinsci-dev@googlegroups.com 
> <mailto:jenkinsci-dev@googlegroups.com>> on behalf of nicolas de loof 
> <nicolas.del...@gmail.com <mailto:nicolas.del...@gmail.com>>
> Reply-To: "jenkinsci-dev@googlegroups.com 
> <mailto:jenkinsci-dev@googlegroups.com>" <jenkinsci-dev@googlegroups.com 
> <mailto:jenkinsci-dev@googlegroups.com>>
> Date: Friday, March 4, 2016 at 3:08 AM
> To: "jenkinsci-dev@googlegroups.com <mailto:jenkinsci-dev@googlegroups.com>" 
> <jenkinsci-dev@googlegroups.com <mailto:jenkinsci-dev@googlegroups.com>>
> Subject: One-Shot Executors
> 
> Hi folks,
> 
> Yoann and I have extracted from our docker-slaves hack-ish pet project a 
> stable sub-component so a plugin can manage slaves that are dedicated to a 
> Build, not relying on Cloud API, and get them tied to a Build. i.e Slave and 
> Build share a common lifecycle. If Slave fails to start, Build will fail as 
> well. If slave is slow to start (maybe pulling a huge docker image), build 
> log will report the progress.
> etc.
> 
> design and use-case : 
> https://wiki.jenkins-ci.org/display/JENKINS/One-Shot+Executor 
> <https://wiki.jenkins-ci.org/display/JENKINS/One-Shot+Executor>
> code : https://github.com/jenkinsci/one-shot-executor-plugin 
> <https://github.com/jenkinsci/one-shot-executor-plugin>
> 
> the current code base do rely on hacks, our goal is to demonstrate this use 
> case (can be tested reusing sample 
> <https://github.com/jenkinsci/one-shot-executor-plugin/blob/master/src/sample/java/org/jenkinsci/plugins/dockerprovisioner/DemoOneShotProvisioner.java>)
>  so we can get the adequate hooks introduced in jenkins-core and later 
> re-implement same API on a cleaner basis.
> 
> 
> 
> 
> --
> You received this message because you are subscribed to the Google Groups 
> "Jenkins Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to jenkinsci-dev+unsubscr...@googlegroups.com 
> <mailto:jenkinsci-dev+unsubscr...@googlegroups.com>.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/jenkinsci-dev/CANMVJznmnydODNX%2BYXmh0ujnJXsoZWcPT%2BpJCWwdR0_wUpfDTQ%40mail.gmail.com
>  
> <https://groups.google.com/d/msgid/jenkinsci-dev/CANMVJznmnydODNX%2BYXmh0ujnJXsoZWcPT%2BpJCWwdR0_wUpfDTQ%40mail.gmail.com?utm_medium=email_source=footer>.
> For more options, visit https://groups.google.com/d/optout 
> <https://groups.google.com/d/optout>.
> 
> --
> You received this message because you are subscribed to a topic in the Google 
> Groups "Jenkins Developers" group.
> To unsubscribe from this topic, visit 
> https://groups.google.com/d/topic/jenkinsci-dev/WV6sPmIh-tk/unsubscribe 
> <https://groups.google.com/d/topic/jenkinsci-dev/WV6sPmIh-tk/unsubscribe>.
> To unsubscribe from this group and all its topics, send an email to 
> jenkinsci-dev+unsubscr...@googlegroups.com 
> <mailto:jenkinsci-dev+unsubscr...@googlegroups.com>.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/jenkinsci-dev/D305926F.28292%25thomas.suckow%40pnnl.gov
>  
> <https://groups.google.com/d/msgid/jenkinsci-dev/D305926F.28292%25thomas.suckow%40pnnl.gov?utm_medium=email_source=footer>.
> For more options, visit https://group

Re: One-Shot Executors

[Suckow, Thomas J]

I like that this moves the provisioning to the build log.

I do agree that certain issues should fail immediately (image not found). 
Certain other issues should perform exponential backoff (Cloud infrastructure 
down). Provisioning limits could be annoying though, would be interesting if 
they could be left in the queue until Jenkins side provisioning limits are not 
violated. I am not sure how to handle an environment like Kubernetes though 
where other entities may be utilizing resources and you have to "share".

You mention using labels to pick the slave. I'm wondering if it would be 
feasible and worthwhile to make such a plugin generic to be the middle layer 
for the Jenkins hooks to the cloud specific implementation (Docker, Kubernetes, 
AWS).

It could also handle the logic of some users wanting to configure slaves on a 
per job basis. Would be interesting if could also be integrated into cloudbees 
folder level. If the later could work then I wouldn't need to run my own 
Jenkins install at work for using containers and instead could use the company 
cloudbees Jenkins.

-
Thomas

From: <jenkinsci-dev@googlegroups.com<mailto:jenkinsci-dev@googlegroups.com>> 
on behalf of nicolas de loof 
<nicolas.del...@gmail.com<mailto:nicolas.del...@gmail.com>>
Reply-To: 
"jenkinsci-dev@googlegroups.com<mailto:jenkinsci-dev@googlegroups.com>" 
<jenkinsci-dev@googlegroups.com<mailto:jenkinsci-dev@googlegroups.com>>
Date: Friday, March 4, 2016 at 3:08 AM
To: "jenkinsci-dev@googlegroups.com<mailto:jenkinsci-dev@googlegroups.com>" 
<jenkinsci-dev@googlegroups.com<mailto:jenkinsci-dev@googlegroups.com>>
Subject: One-Shot Executors

Hi folks,

Yoann and I have extracted from our docker-slaves hack-ish pet project a stable 
sub-component so a plugin can manage slaves that are dedicated to a Build, not 
relying on Cloud API, and get them tied to a Build. i.e Slave and Build share a 
common lifecycle. If Slave fails to start, Build will fail as well. If slave is 
slow to start (maybe pulling a huge docker image), build log will report the 
progress.
etc.

design and use-case : 
https://wiki.jenkins-ci.org/display/JENKINS/One-Shot+Executor
code : https://github.com/jenkinsci/one-shot-executor-plugin

the current code base do rely on hacks, our goal is to demonstrate this use 
case (can be tested reusing 
sample<https://github.com/jenkinsci/one-shot-executor-plugin/blob/master/src/sample/java/org/jenkinsci/plugins/dockerprovisioner/DemoOneShotProvisioner.java>)
 so we can get the adequate hooks introduced in jenkins-core and later 
re-implement same API on a cleaner basis.




--
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to 
jenkinsci-dev+unsubscr...@googlegroups.com<mailto:jenkinsci-dev+unsubscr...@googlegroups.com>.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CANMVJznmnydODNX%2BYXmh0ujnJXsoZWcPT%2BpJCWwdR0_wUpfDTQ%40mail.gmail.com<https://groups.google.com/d/msgid/jenkinsci-dev/CANMVJznmnydODNX%2BYXmh0ujnJXsoZWcPT%2BpJCWwdR0_wUpfDTQ%40mail.gmail.com?utm_medium=email_source=footer>.
For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/D305926F.28292%25thomas.suckow%40pnnl.gov.
For more options, visit https://groups.google.com/d/optout.

Re: One-Shot Executors

[nicolas de loof]

Yes, was me.
I never will get used with your "style". You claim you don't work on a
plugin, you even create another one, and now complain you've been removed
from the plugin admin team ?
Feel free to report this "abuse" to board if this is such a disturbance for
you, I'm sure someone will find the IRC bot command to give you back access
to this repo.


2016-03-05 16:42 GMT+01:00 Kanstantsin Shautsou :

> Was it you? I still answering in PRs and issues. @magnayan is also not
> actively maintaining it, would you also remove it? If it was you (the
> person that never worked on this plugin) then i would be glad to see the
> list of plugins what you maintain and ensure that you also removed from all
> unrelated teams. Would it be fair? Either i would request descriptions to
> the Jenkins Board.
>
> On Mar 5, 2016, at 18:33, nicolas de loof 
> wrote:
>
> *you* just said you aren't working anymore on this plugin. So why complain
> ?
>
> 2016-03-05 16:31 GMT+01:00 Kanstantsin Shautsou  >:
>
>> `You've been removed from the Jenkins team docker-plugin Developers`
>> Thanks to anonymous person, removing person from team would be a great
>> benefit to this plugin! ;)
>> That a bit offtop, please continue with the initial topic.
>>
>>
>> On Saturday, March 5, 2016 at 11:47:09 AM UTC+3, Kanstantsin Shautsou
>> wrote:
>>>
>>>
>>> On Mar 5, 2016, at 11:15, nicolas de loof 
>>> wrote:
>>>
>>>
>>>
>>> 2016-03-05 7:36 GMT+01:00 Kanstantsin Shautsou <
>>> kanstantsin@gmail.com>:
>>>
 I made my plugin after @stephenc comments and seems that his work
 somebody finally tried to expose
 https://github.com/jenkinsci/mansion-cloud-plugin/pull/9 According to
 stephenc comments such logic should be generic and would make sense for all
 cloud plugins, that’s why i made separate plugin.

 @batmat if you are using docker-plugin then it may have synchronisation
 or logic issues.

>>>
>>> @KostyaSha is this something you're trying to address with you
>>> "yet-another-docker-plugin" ? Are you working on both plugins ?
>>>
>>> I’m not maintaining docker-plugin anymore (reasons described in
>>> readme). I had good experience with migrating static ssh slaves to
>>> docker-plugin, but it had not enough stability.  In yad-plugin i just
>>> continue the work and want get stable and faster provisioning for standard
>>> Cloud slaves (for integration tests invented remote run and configuration
>>> of master instance and started working on jmeter plugin for doing
>>> performance tests). Delays mostly not a cloud plugin issue, so experimental
>>> “faster” i placed in separate plugin. In yad-plugin and docker-java library
>>> i almost resolved all related issues and hope it will work good, stable and
>>> covered with ITs.
>>> For future I have some experimental ideas for DSLs, but they may
>>> duplicate existing features with the difference that i will use
>>> docker-java. Even if nobody will use it, i would have reference
>>> implementation for docker-java where i’m co-maintainer. For example it
>>> should be possible to expose globally configured docker connection into
>>> JobProperty configuration (that of course will use QTD for hacks), but that
>>> would raise questions about limits and security. Everything depends on free
>>> time and mood :)
>>>
>>> yad-plugin -  is released into UC (it my experiment to host it under
>>> personal account).
>>> faster-node-provision-plugin
>>>  - isn’t
>>> released as i had no final conclusion on it’s stability (maybe it would
>>> require to substitute the whole provisioning strategy). If stephenc will
>>> finally review and bless then it would make sense to fork and release it.
>>>
>>>
>>>
 On Mar 5, 2016, at 01:50, Baptiste Mathus  wrote:


 2016-03-04 21:14 GMT+01:00 Kanstantsin Shautsou <
 kanstantsin@gmail.com>:

> Cloud delays in theory should be possible to avoid by:
> 1) kicking suggestReview. I wrote plugin but had no ability to test on
> huge installations
> https://github.com/KostyaSha/faster-node-provision-plugin
> 2) increasing one of coefficients for provisioner(or statistics
> collector, forgot classname) (jglick is using it in demo's and i'm using 
> in
> my installations).
>

 NodeProvisionner?

 Btw, that page might be related IIUC what you talk about. There're many
 seemingly (didn't test yet) useful infos about those coeffs and what they
 do:

 https://cloudbees.zendesk.com/hc/en-us/articles/204690520-Why-do-slaves-show-as-suspended-while-jobs-wait-in-the-queue-

 Btw, IIUC, this page might explain why I'm seeing my slave/agent docker
 containers being provisionned in between 5 seconds and roughly 6 minutes...


> AFAIK the main jenkins pain is the Queue 

Re: One-Shot Executors

[Kanstantsin Shautsou]

Ok, offtopic should die, i sent email to board, i will get explanations and 
description from board.

For cloud faster provisioning stephenc replied that will look at possible 
ways on Monday.

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/6b5e0b9b-76d2-42ec-af52-3fd746af98d5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: One-Shot Executors

[Kanstantsin Shautsou]

Was it you? I still answering in PRs and issues. @magnayan is also not actively 
maintaining it, would you also remove it? If it was you (the person that never 
worked on this plugin) then i would be glad to see the list of plugins what you 
maintain and ensure that you also removed from all unrelated teams. Would it be 
fair? Either i would request descriptions to the Jenkins Board.

> On Mar 5, 2016, at 18:33, nicolas de loof  wrote:
> 
> *you* just said you aren't working anymore on this plugin. So why complain ?
> 
> 2016-03-05 16:31 GMT+01:00 Kanstantsin Shautsou  >:
> `You've been removed from the Jenkins team docker-plugin Developers`
> Thanks to anonymous person, removing person from team would be a great 
> benefit to this plugin! ;)
> That a bit offtop, please continue with the initial topic.
> 
> 
> On Saturday, March 5, 2016 at 11:47:09 AM UTC+3, Kanstantsin Shautsou wrote:
> 
>> On Mar 5, 2016, at 11:15, nicolas de loof > > wrote:
>> 
>> 
>> 
>> 2016-03-05 7:36 GMT+01:00 Kanstantsin Shautsou > >:
>> I made my plugin after @stephenc comments and seems that his work somebody 
>> finally tried to expose 
>> https://github.com/jenkinsci/mansion-cloud-plugin/pull/9 
>>  According to 
>> stephenc comments such logic should be generic and would make sense for all 
>> cloud plugins, that’s why i made separate plugin.
>> 
>> @batmat if you are using docker-plugin then it may have synchronisation or 
>> logic issues.
>> 
>> @KostyaSha is this something you're trying to address with you 
>> "yet-another-docker-plugin" ? Are you working on both plugins ?
> I’m not maintaining docker-plugin anymore (reasons described in readme). I 
> had good experience with migrating static ssh slaves to docker-plugin, but it 
> had not enough stability.  In yad-plugin i just continue the work and want 
> get stable and faster provisioning for standard Cloud slaves (for integration 
> tests invented remote run and configuration of master instance and started 
> working on jmeter plugin for doing performance tests). Delays mostly not a 
> cloud plugin issue, so experimental “faster” i placed in separate plugin. In 
> yad-plugin and docker-java library i almost resolved all related issues and 
> hope it will work good, stable and covered with ITs.
> For future I have some experimental ideas for DSLs, but they may duplicate 
> existing features with the difference that i will use docker-java. Even if 
> nobody will use it, i would have reference implementation for docker-java 
> where i’m co-maintainer. For example it should be possible to expose globally 
> configured docker connection into JobProperty configuration (that of course 
> will use QTD for hacks), but that would raise questions about limits and 
> security. Everything depends on free time and mood :)
> 
> yad-plugin -  is released into UC (it my experiment to host it under personal 
> account).
> faster-node-provision-plugin 
>  - isn’t released 
> as i had no final conclusion on it’s stability (maybe it would require to 
> substitute the whole provisioning strategy). If stephenc will finally review 
> and bless then it would make sense to fork and release it.
> 
>> 
>>> On Mar 5, 2016, at 01:50, Baptiste Mathus >> > wrote:
>>> 
>>> 
>>> 2016-03-04 21:14 GMT+01:00 Kanstantsin Shautsou >> >:
>>> Cloud delays in theory should be possible to avoid by:
>>> 1) kicking suggestReview. I wrote plugin but had no ability to test on huge 
>>> installations https://github.com/KostyaSha/faster-node-provision-plugin 
>>> 
>>> 2) increasing one of coefficients for provisioner(or statistics collector, 
>>> forgot classname) (jglick is using it in demo's and i'm using in my 
>>> installations).
>>> 
>>> NodeProvisionner?
>>> 
>>> Btw, that page might be related IIUC what you talk about. There're many 
>>> seemingly (didn't test yet) useful infos about those coeffs and what they 
>>> do:
>>> https://cloudbees.zendesk.com/hc/en-us/articles/204690520-Why-do-slaves-show-as-suspended-while-jobs-wait-in-the-queue-
>>>  
>>> 
>>> 
>>> Btw, IIUC, this page might explain why I'm seeing my slave/agent docker 
>>> containers being provisionned in between 5 seconds and roughly 6 minutes...
>>> 
>>> 
>>> AFAIK the main jenkins pain is the Queue locks. How this plugin will deal 
>>> with locks?
>>> 
>>> 
>>> 2016-03-04 12:08 GMT+01:00 nicolas de loof >:
>>> Hi 

Re: One-Shot Executors

[nicolas de loof]

*you* just said you aren't working anymore on this plugin. So why complain ?

2016-03-05 16:31 GMT+01:00 Kanstantsin Shautsou :

> `You've been removed from the Jenkins team docker-plugin Developers`
> Thanks to anonymous person, removing person from team would be a great
> benefit to this plugin! ;)
> That a bit offtop, please continue with the initial topic.
>
>
> On Saturday, March 5, 2016 at 11:47:09 AM UTC+3, Kanstantsin Shautsou
> wrote:
>>
>>
>> On Mar 5, 2016, at 11:15, nicolas de loof 
>> wrote:
>>
>>
>>
>> 2016-03-05 7:36 GMT+01:00 Kanstantsin Shautsou > >:
>>
>>> I made my plugin after @stephenc comments and seems that his work
>>> somebody finally tried to expose
>>> https://github.com/jenkinsci/mansion-cloud-plugin/pull/9 According to
>>> stephenc comments such logic should be generic and would make sense for all
>>> cloud plugins, that’s why i made separate plugin.
>>>
>>> @batmat if you are using docker-plugin then it may have synchronisation
>>> or logic issues.
>>>
>>
>> @KostyaSha is this something you're trying to address with you
>> "yet-another-docker-plugin" ? Are you working on both plugins ?
>>
>> I’m not maintaining docker-plugin anymore (reasons described in
>> readme). I had good experience with migrating static ssh slaves to
>> docker-plugin, but it had not enough stability.  In yad-plugin i just
>> continue the work and want get stable and faster provisioning for standard
>> Cloud slaves (for integration tests invented remote run and configuration
>> of master instance and started working on jmeter plugin for doing
>> performance tests). Delays mostly not a cloud plugin issue, so experimental
>> “faster” i placed in separate plugin. In yad-plugin and docker-java library
>> i almost resolved all related issues and hope it will work good, stable and
>> covered with ITs.
>> For future I have some experimental ideas for DSLs, but they may
>> duplicate existing features with the difference that i will use
>> docker-java. Even if nobody will use it, i would have reference
>> implementation for docker-java where i’m co-maintainer. For example it
>> should be possible to expose globally configured docker connection into
>> JobProperty configuration (that of course will use QTD for hacks), but that
>> would raise questions about limits and security. Everything depends on free
>> time and mood :)
>>
>> yad-plugin -  is released into UC (it my experiment to host it under
>> personal account).
>> faster-node-provision-plugin
>>  - isn’t
>> released as i had no final conclusion on it’s stability (maybe it would
>> require to substitute the whole provisioning strategy). If stephenc will
>> finally review and bless then it would make sense to fork and release it.
>>
>>
>>
>>> On Mar 5, 2016, at 01:50, Baptiste Mathus  wrote:
>>>
>>>
>>> 2016-03-04 21:14 GMT+01:00 Kanstantsin Shautsou <
>>> kanstantsin@gmail.com>:
>>>
 Cloud delays in theory should be possible to avoid by:
 1) kicking suggestReview. I wrote plugin but had no ability to test on
 huge installations
 https://github.com/KostyaSha/faster-node-provision-plugin
 2) increasing one of coefficients for provisioner(or statistics
 collector, forgot classname) (jglick is using it in demo's and i'm using in
 my installations).

>>>
>>> NodeProvisionner?
>>>
>>> Btw, that page might be related IIUC what you talk about. There're many
>>> seemingly (didn't test yet) useful infos about those coeffs and what they
>>> do:
>>>
>>> https://cloudbees.zendesk.com/hc/en-us/articles/204690520-Why-do-slaves-show-as-suspended-while-jobs-wait-in-the-queue-
>>>
>>> Btw, IIUC, this page might explain why I'm seeing my slave/agent docker
>>> containers being provisionned in between 5 seconds and roughly 6 minutes...
>>>
>>>
 AFAIK the main jenkins pain is the Queue locks. How this plugin will
 deal with locks?


>>> 2016-03-04 12:08 GMT+01:00 nicolas de loof :
>>>
 Hi folks,

 Yoann and I have extracted from our docker-slaves hack-ish pet
 project a stable sub-component so a plugin can manage slaves that are
 dedicated to a Build, not relying on Cloud API, and get them tied to a
 Build. i.e Slave and Build share a common lifecycle. If Slave fails to
 start, Build will fail as well. If slave is slow to start (maybe 
 pulling a
 huge docker image), build log will report the progress.
 etc.

 design and use-case :
 https://wiki.jenkins-ci.org/display/JENKINS/One-Shot+Executor
 code : https://github.com/jenkinsci/one-shot-executor-plugin

 the current code base do rely on hacks, our goal is to demonstrate
 this use case (can be tested reusing sample
 

Re: One-Shot Executors

[Kanstantsin Shautsou]

`You've been removed from the Jenkins team docker-plugin Developers`
Thanks to anonymous person, removing person from team would be a great 
benefit to this plugin! ;)
That a bit offtop, please continue with the initial topic. 

On Saturday, March 5, 2016 at 11:47:09 AM UTC+3, Kanstantsin Shautsou wrote:
>
>
> On Mar 5, 2016, at 11:15, nicolas de loof  
> wrote:
>
>
>
> 2016-03-05 7:36 GMT+01:00 Kanstantsin Shautsou 
> :
>
>> I made my plugin after @stephenc comments and seems that his work 
>> somebody finally tried to expose 
>> https://github.com/jenkinsci/mansion-cloud-plugin/pull/9 According to 
>> stephenc comments such logic should be generic and would make sense for all 
>> cloud plugins, that’s why i made separate plugin.
>>
>> @batmat if you are using docker-plugin then it may have synchronisation 
>> or logic issues. 
>>
>
> @KostyaSha is this something you're trying to address with you 
> "yet-another-docker-plugin" ? Are you working on both plugins ?
>
> I’m not maintaining docker-plugin anymore (reasons described in readme). I 
> had good experience with migrating static ssh slaves to docker-plugin, but 
> it had not enough stability.  In yad-plugin i just continue the work and 
> want get stable and faster provisioning for standard Cloud slaves (for 
> integration tests invented remote run and configuration of master instance 
> and started working on jmeter plugin for doing performance tests). Delays 
> mostly not a cloud plugin issue, so experimental “faster” i placed in 
> separate plugin. In yad-plugin and docker-java library i almost resolved 
> all related issues and hope it will work good, stable and covered with ITs.
> For future I have some experimental ideas for DSLs, but they may duplicate 
> existing features with the difference that i will use docker-java. Even if 
> nobody will use it, i would have reference implementation for docker-java 
> where i’m co-maintainer. For example it should be possible to expose 
> globally configured docker connection into JobProperty configuration (that 
> of course will use QTD for hacks), but that would raise questions about 
> limits and security. Everything depends on free time and mood :)
>
> yad-plugin -  is released into UC (it my experiment to host it under 
> personal account).
> faster-node-provision-plugin 
>  - isn’t 
> released as i had no final conclusion on it’s stability (maybe it would 
> require to substitute the whole provisioning strategy). If stephenc will 
> finally review and bless then it would make sense to fork and release it.
>
>  
>
>> On Mar 5, 2016, at 01:50, Baptiste Mathus  wrote:
>>
>>
>> 2016-03-04 21:14 GMT+01:00 Kanstantsin Shautsou <
>> kanstantsin@gmail.com>:
>>
>>> Cloud delays in theory should be possible to avoid by:
>>> 1) kicking suggestReview. I wrote plugin but had no ability to test on 
>>> huge installations 
>>> https://github.com/KostyaSha/faster-node-provision-plugin
>>> 2) increasing one of coefficients for provisioner(or statistics 
>>> collector, forgot classname) (jglick is using it in demo's and i'm using in 
>>> my installations).
>>>
>>
>> NodeProvisionner?
>>
>> Btw, that page might be related IIUC what you talk about. There're many 
>> seemingly (didn't test yet) useful infos about those coeffs and what they 
>> do:
>>
>> https://cloudbees.zendesk.com/hc/en-us/articles/204690520-Why-do-slaves-show-as-suspended-while-jobs-wait-in-the-queue-
>>
>> Btw, IIUC, this page might explain why I'm seeing my slave/agent docker 
>> containers being provisionned in between 5 seconds and roughly 6 minutes...
>>
>>
>>> AFAIK the main jenkins pain is the Queue locks. How this plugin will 
>>> deal with locks? 
>>>
>>>
>> 2016-03-04 12:08 GMT+01:00 nicolas de loof :
>>
>>> Hi folks,
>>>
>>> Yoann and I have extracted from our docker-slaves hack-ish pet 
>>> project a stable sub-component so a plugin can manage slaves that are 
>>> dedicated to a Build, not relying on Cloud API, and get them tied to a 
>>> Build. i.e Slave and Build share a common lifecycle. If Slave fails to 
>>> start, Build will fail as well. If slave is slow to start (maybe 
>>> pulling a 
>>> huge docker image), build log will report the progress.
>>> etc.
>>>
>>> design and use-case : 
>>> https://wiki.jenkins-ci.org/display/JENKINS/One-Shot+Executor
>>> code : https://github.com/jenkinsci/one-shot-executor-plugin
>>>
>>> the current code base do rely on hacks, our goal is to demonstrate 
>>> this use case (can be tested reusing sample 
>>> )
>>>  
>>> so we can get the adequate hooks introduced in jenkins-core and later 
>>> re-implement 

Re: One-Shot Executors

[Kanstantsin Shautsou]

> On Mar 5, 2016, at 11:15, nicolas de loof  wrote:
> 
> 
> 
> 2016-03-05 7:36 GMT+01:00 Kanstantsin Shautsou  >:
> I made my plugin after @stephenc comments and seems that his work somebody 
> finally tried to expose 
> https://github.com/jenkinsci/mansion-cloud-plugin/pull/9 
>  According to 
> stephenc comments such logic should be generic and would make sense for all 
> cloud plugins, that’s why i made separate plugin.
> 
> @batmat if you are using docker-plugin then it may have synchronisation or 
> logic issues.
> 
> @KostyaSha is this something you're trying to address with you 
> "yet-another-docker-plugin" ? Are you working on both plugins ?
I’m not maintaining docker-plugin anymore (reasons described in readme). I had 
good experience with migrating static ssh slaves to docker-plugin, but it had 
not enough stability.  In yad-plugin i just continue the work and want get 
stable and faster provisioning for standard Cloud slaves (for integration tests 
invented remote run and configuration of master instance and started working on 
jmeter plugin for doing performance tests). Delays mostly not a cloud plugin 
issue, so experimental “faster” i placed in separate plugin. In yad-plugin and 
docker-java library i almost resolved all related issues and hope it will work 
good, stable and covered with ITs.
For future I have some experimental ideas for DSLs, but they may duplicate 
existing features with the difference that i will use docker-java. Even if 
nobody will use it, i would have reference implementation for docker-java where 
i’m co-maintainer. For example it should be possible to expose globally 
configured docker connection into JobProperty configuration (that of course 
will use QTD for hacks), but that would raise questions about limits and 
security. Everything depends on free time and mood :)

yad-plugin -  is released into UC (it my experiment to host it under personal 
account).
faster-node-provision-plugin 
 - isn’t released as 
i had no final conclusion on it’s stability (maybe it would require to 
substitute the whole provisioning strategy). If stephenc will finally review 
and bless then it would make sense to fork and release it.

> 
>> On Mar 5, 2016, at 01:50, Baptiste Mathus > > wrote:
>> 
>> 
>> 2016-03-04 21:14 GMT+01:00 Kanstantsin Shautsou > >:
>> Cloud delays in theory should be possible to avoid by:
>> 1) kicking suggestReview. I wrote plugin but had no ability to test on huge 
>> installations https://github.com/KostyaSha/faster-node-provision-plugin 
>> 
>> 2) increasing one of coefficients for provisioner(or statistics collector, 
>> forgot classname) (jglick is using it in demo's and i'm using in my 
>> installations).
>> 
>> NodeProvisionner?
>> 
>> Btw, that page might be related IIUC what you talk about. There're many 
>> seemingly (didn't test yet) useful infos about those coeffs and what they do:
>> https://cloudbees.zendesk.com/hc/en-us/articles/204690520-Why-do-slaves-show-as-suspended-while-jobs-wait-in-the-queue-
>>  
>> 
>> 
>> Btw, IIUC, this page might explain why I'm seeing my slave/agent docker 
>> containers being provisionned in between 5 seconds and roughly 6 minutes...
>> 
>> 
>> AFAIK the main jenkins pain is the Queue locks. How this plugin will deal 
>> with locks?
>> 
>> 
>> 2016-03-04 12:08 GMT+01:00 nicolas de loof >:
>> Hi folks,
>> 
>> Yoann and I have extracted from our docker-slaves hack-ish pet project a 
>> stable sub-component so a plugin can manage slaves that are dedicated to a 
>> Build, not relying on Cloud API, and get them tied to a Build. i.e Slave and 
>> Build share a common lifecycle. If Slave fails to start, Build will fail as 
>> well. If slave is slow to start (maybe pulling a huge docker image), build 
>> log will report the progress.
>> etc.
>> 
>> design and use-case : 
>> https://wiki.jenkins-ci.org/display/JENKINS/One-Shot+Executor 
>> 
>> code : https://github.com/jenkinsci/one-shot-executor-plugin 
>> 
>> 
>> the current code base do rely on hacks, our goal is to demonstrate this use 
>> case (can be tested reusing sample 
>> )
>>  so we can get the adequate hooks introduced in jenkins-core and later 
>> re-implement same API on a cleaner basis.
>> 

Re: One-Shot Executors

[nicolas de loof]

2016-03-05 7:36 GMT+01:00 Kanstantsin Shautsou :

> I made my plugin after @stephenc comments and seems that his work somebody
> finally tried to expose
> https://github.com/jenkinsci/mansion-cloud-plugin/pull/9 According to
> stephenc comments such logic should be generic and would make sense for all
> cloud plugins, that’s why i made separate plugin.
>
> @batmat if you are using docker-plugin then it may have synchronisation or
> logic issues.
>

@KostyaSha is this something you're trying to address with you
"yet-another-docker-plugin" ? Are you working on both plugins ?


> On Mar 5, 2016, at 01:50, Baptiste Mathus  wrote:
>
>
> 2016-03-04 21:14 GMT+01:00 Kanstantsin Shautsou  >:
>
>> Cloud delays in theory should be possible to avoid by:
>> 1) kicking suggestReview. I wrote plugin but had no ability to test on
>> huge installations
>> https://github.com/KostyaSha/faster-node-provision-plugin
>> 2) increasing one of coefficients for provisioner(or statistics
>> collector, forgot classname) (jglick is using it in demo's and i'm using in
>> my installations).
>>
>
> NodeProvisionner?
>
> Btw, that page might be related IIUC what you talk about. There're many
> seemingly (didn't test yet) useful infos about those coeffs and what they
> do:
>
> https://cloudbees.zendesk.com/hc/en-us/articles/204690520-Why-do-slaves-show-as-suspended-while-jobs-wait-in-the-queue-
>
> Btw, IIUC, this page might explain why I'm seeing my slave/agent docker
> containers being provisionned in between 5 seconds and roughly 6 minutes...
>
>
>> AFAIK the main jenkins pain is the Queue locks. How this plugin will deal
>> with locks?
>>
>>
> 2016-03-04 12:08 GMT+01:00 nicolas de loof :
>
>> Hi folks,
>>
>> Yoann and I have extracted from our docker-slaves hack-ish pet
>> project a stable sub-component so a plugin can manage slaves that are
>> dedicated to a Build, not relying on Cloud API, and get them tied to a
>> Build. i.e Slave and Build share a common lifecycle. If Slave fails to
>> start, Build will fail as well. If slave is slow to start (maybe pulling 
>> a
>> huge docker image), build log will report the progress.
>> etc.
>>
>> design and use-case :
>> https://wiki.jenkins-ci.org/display/JENKINS/One-Shot+Executor
>> code : https://github.com/jenkinsci/one-shot-executor-plugin
>>
>> the current code base do rely on hacks, our goal is to demonstrate
>> this use case (can be tested reusing sample
>> )
>> so we can get the adequate hooks introduced in jenkins-core and later
>> re-implement same API on a cleaner basis.
>>
>>
>>
>>
>
 --
 You received this message because you are subscribed to the Google
 Groups "Jenkins Developers" group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to jenkinsci-de...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/jenkinsci-dev/f5935828-61d5-4afa-afab-9090deacb19e%40googlegroups.com
 
 .
 For more options, visit https://groups.google.com/d/optout.

>>>
>>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Jenkins Developers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to jenkinsci-dev+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/jenkinsci-dev/d65c0d1f-e298-4e08-b5f9-15f883b55f12%40googlegroups.com
>> 
>> .
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "Jenkins Developers" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/jenkinsci-dev/WV6sPmIh-tk/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> jenkinsci-dev+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jenkinsci-dev/CANWgJS7kb2RkXzgQPrv7SS%2BCJVsV79q17sHsVE6pzYXrCh3QHw%40mail.gmail.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Jenkins Developers" 

Re: One-Shot Executors

[Kanstantsin Shautsou]

I made my plugin after @stephenc comments and seems that his work somebody 
finally tried to expose 
https://github.com/jenkinsci/mansion-cloud-plugin/pull/9 
 According to 
stephenc comments such logic should be generic and would make sense for all 
cloud plugins, that’s why i made separate plugin.

@batmat if you are using docker-plugin then it may have synchronisation or 
logic issues.
> On Mar 5, 2016, at 01:50, Baptiste Mathus  wrote:
> 
> 
> 2016-03-04 21:14 GMT+01:00 Kanstantsin Shautsou  >:
> Cloud delays in theory should be possible to avoid by:
> 1) kicking suggestReview. I wrote plugin but had no ability to test on huge 
> installations https://github.com/KostyaSha/faster-node-provision-plugin 
> 
> 2) increasing one of coefficients for provisioner(or statistics collector, 
> forgot classname) (jglick is using it in demo's and i'm using in my 
> installations).
> 
> NodeProvisionner?
> 
> Btw, that page might be related IIUC what you talk about. There're many 
> seemingly (didn't test yet) useful infos about those coeffs and what they do:
> https://cloudbees.zendesk.com/hc/en-us/articles/204690520-Why-do-slaves-show-as-suspended-while-jobs-wait-in-the-queue-
>  
> 
> 
> Btw, IIUC, this page might explain why I'm seeing my slave/agent docker 
> containers being provisionned in between 5 seconds and roughly 6 minutes...
> 
> 
> AFAIK the main jenkins pain is the Queue locks. How this plugin will deal 
> with locks?
> 
> 
> 2016-03-04 12:08 GMT+01:00 nicolas de loof >:
> Hi folks,
> 
> Yoann and I have extracted from our docker-slaves hack-ish pet project a 
> stable sub-component so a plugin can manage slaves that are dedicated to a 
> Build, not relying on Cloud API, and get them tied to a Build. i.e Slave and 
> Build share a common lifecycle. If Slave fails to start, Build will fail as 
> well. If slave is slow to start (maybe pulling a huge docker image), build 
> log will report the progress.
> etc.
> 
> design and use-case : 
> https://wiki.jenkins-ci.org/display/JENKINS/One-Shot+Executor 
> 
> code : https://github.com/jenkinsci/one-shot-executor-plugin 
> 
> 
> the current code base do rely on hacks, our goal is to demonstrate this use 
> case (can be tested reusing sample 
> )
>  so we can get the adequate hooks introduced in jenkins-core and later 
> re-implement same API on a cleaner basis.
> 
> 
> 
> 
> 
> --
> You received this message because you are subscribed to the Google Groups 
> "Jenkins Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to jenkinsci-de...@googlegroups.com <>.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/jenkinsci-dev/f5935828-61d5-4afa-afab-9090deacb19e%40googlegroups.com
>  
> .
> For more options, visit https://groups.google.com/d/optout 
> .
> 
> 
> --
> You received this message because you are subscribed to the Google Groups 
> "Jenkins Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to jenkinsci-dev+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/jenkinsci-dev/d65c0d1f-e298-4e08-b5f9-15f883b55f12%40googlegroups.com
>  
> .
> 
> For more options, visit https://groups.google.com/d/optout 
> .
> 
> 
> --
> You received this message because you are subscribed to a topic in the Google 
> Groups "Jenkins Developers" group.
> To unsubscribe from this topic, visit 
> https://groups.google.com/d/topic/jenkinsci-dev/WV6sPmIh-tk/unsubscribe 
> .
> To unsubscribe from this group and all its topics, send an email to 
> jenkinsci-dev+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/jenkinsci-dev/CANWgJS7kb2RkXzgQPrv7SS%2BCJVsV79q17sHsVE6pzYXrCh3QHw%40mail.gmail.com
>  
> 

Re: One-Shot Executors

[Baptiste Mathus]

2016-03-04 21:14 GMT+01:00 Kanstantsin Shautsou :

> Cloud delays in theory should be possible to avoid by:
> 1) kicking suggestReview. I wrote plugin but had no ability to test on
> huge installations
> https://github.com/KostyaSha/faster-node-provision-plugin
> 2) increasing one of coefficients for provisioner(or statistics collector,
> forgot classname) (jglick is using it in demo's and i'm using in my
> installations).
>

NodeProvisionner?

Btw, that page might be related IIUC what you talk about. There're many
seemingly (didn't test yet) useful infos about those coeffs and what they
do:
https://cloudbees.zendesk.com/hc/en-us/articles/204690520-Why-do-slaves-show-as-suspended-while-jobs-wait-in-the-queue-

Btw, IIUC, this page might explain why I'm seeing my slave/agent docker
containers being provisionned in between 5 seconds and roughly 6 minutes...


> AFAIK the main jenkins pain is the Queue locks. How this plugin will deal
> with locks?
>
>
 2016-03-04 12:08 GMT+01:00 nicolas de loof :

> Hi folks,
>
> Yoann and I have extracted from our docker-slaves hack-ish pet project
> a stable sub-component so a plugin can manage slaves that are dedicated to
> a Build, not relying on Cloud API, and get them tied to a Build. i.e Slave
> and Build share a common lifecycle. If Slave fails to start, Build will
> fail as well. If slave is slow to start (maybe pulling a huge docker
> image), build log will report the progress.
> etc.
>
> design and use-case :
> https://wiki.jenkins-ci.org/display/JENKINS/One-Shot+Executor
> code : https://github.com/jenkinsci/one-shot-executor-plugin
>
> the current code base do rely on hacks, our goal is to demonstrate
> this use case (can be tested reusing sample
> )
> so we can get the adequate hooks introduced in jenkins-core and later
> re-implement same API on a cleaner basis.
>
>
>
>
 --
>>> You received this message because you are subscribed to the Google
>>> Groups "Jenkins Developers" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to jenkinsci-de...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/jenkinsci-dev/f5935828-61d5-4afa-afab-9090deacb19e%40googlegroups.com
>>> 
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>> --
> You received this message because you are subscribed to the Google Groups
> "Jenkins Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jenkinsci-dev+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jenkinsci-dev/d65c0d1f-e298-4e08-b5f9-15f883b55f12%40googlegroups.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CANWgJS7kb2RkXzgQPrv7SS%2BCJVsV79q17sHsVE6pzYXrCh3QHw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: One-Shot Executors

[Kanstantsin Shautsou]

Cloud delays in theory should be possible to avoid by:
1) kicking suggestReview. I wrote plugin but had no ability to test on huge 
installations https://github.com/KostyaSha/faster-node-provision-plugin
2) increasing one of coefficients for provisioner(or statistics collector, 
forgot classname) (jglick is using it in demo's and i'm using in my 
installations).

AFAIK the main jenkins pain is the Queue locks. How this plugin will deal 
with locks? 


>>> 2016-03-04 12:08 GMT+01:00 nicolas de loof :
>>>
 Hi folks,

 Yoann and I have extracted from our docker-slaves hack-ish pet project 
 a stable sub-component so a plugin can manage slaves that are dedicated to 
 a Build, not relying on Cloud API, and get them tied to a Build. i.e Slave 
 and Build share a common lifecycle. If Slave fails to start, Build will 
 fail as well. If slave is slow to start (maybe pulling a huge docker 
 image), build log will report the progress.
 etc.

 design and use-case : 
 https://wiki.jenkins-ci.org/display/JENKINS/One-Shot+Executor
 code : https://github.com/jenkinsci/one-shot-executor-plugin

 the current code base do rely on hacks, our goal is to demonstrate this 
 use case (can be tested reusing sample 
 )
  
 so we can get the adequate hooks introduced in jenkins-core and later 
 re-implement same API on a cleaner basis.




>>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Jenkins Developers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to jenkinsci-de...@googlegroups.com .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/jenkinsci-dev/f5935828-61d5-4afa-afab-9090deacb19e%40googlegroups.com
>>  
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/d65c0d1f-e298-4e08-b5f9-15f883b55f12%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: One-Shot Executors

[nicolas de loof]

There was actually an extra + sign on URL that broke the link

2016-03-04 17:25 GMT+01:00 Kanstantsin Shautsou :

> This is called cached. Everybody who uses jenkins jira hits this issue
> always.
>
> On Friday, March 4, 2016 at 3:41:47 PM UTC+3, nicolas de loof wrote:
>>
>> Dominik edited the wiki page to fix a typo and now it is not available as
>> https://wiki.jenkins-ci.org/display/JENKINS/One-Shot+Executor anymore,
>> but still there
>> https://wiki.jenkins-ci.org/pages/viewpage.action?pageId=96075993
>>
>> Can a Confluence expert can explain this ?
>>
>> 2016-03-04 12:08 GMT+01:00 nicolas de loof :
>>
>>> Hi folks,
>>>
>>> Yoann and I have extracted from our docker-slaves hack-ish pet project a
>>> stable sub-component so a plugin can manage slaves that are dedicated to a
>>> Build, not relying on Cloud API, and get them tied to a Build. i.e Slave
>>> and Build share a common lifecycle. If Slave fails to start, Build will
>>> fail as well. If slave is slow to start (maybe pulling a huge docker
>>> image), build log will report the progress.
>>> etc.
>>>
>>> design and use-case :
>>> https://wiki.jenkins-ci.org/display/JENKINS/One-Shot+Executor
>>> code : https://github.com/jenkinsci/one-shot-executor-plugin
>>>
>>> the current code base do rely on hacks, our goal is to demonstrate this
>>> use case (can be tested reusing sample
>>> )
>>> so we can get the adequate hooks introduced in jenkins-core and later
>>> re-implement same API on a cleaner basis.
>>>
>>>
>>>
>>>
>> --
> You received this message because you are subscribed to the Google Groups
> "Jenkins Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jenkinsci-dev+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jenkinsci-dev/f5935828-61d5-4afa-afab-9090deacb19e%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CANMVJzmzgHDaGJ93pAvbjidcon1zgeTQ5Hi-wa6Mws84Cbuk3Q%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: One-Shot Executors

[Kanstantsin Shautsou]

This is called cached. Everybody who uses jenkins jira hits this issue 
always.

On Friday, March 4, 2016 at 3:41:47 PM UTC+3, nicolas de loof wrote:
>
> Dominik edited the wiki page to fix a typo and now it is not available as 
> https://wiki.jenkins-ci.org/display/JENKINS/One-Shot+Executor anymore, 
> but still there 
> https://wiki.jenkins-ci.org/pages/viewpage.action?pageId=96075993
>
> Can a Confluence expert can explain this ?
>
> 2016-03-04 12:08 GMT+01:00 nicolas de loof  >:
>
>> Hi folks,
>>
>> Yoann and I have extracted from our docker-slaves hack-ish pet project a 
>> stable sub-component so a plugin can manage slaves that are dedicated to a 
>> Build, not relying on Cloud API, and get them tied to a Build. i.e Slave 
>> and Build share a common lifecycle. If Slave fails to start, Build will 
>> fail as well. If slave is slow to start (maybe pulling a huge docker 
>> image), build log will report the progress.
>> etc.
>>
>> design and use-case : 
>> https://wiki.jenkins-ci.org/display/JENKINS/One-Shot+Executor
>> code : https://github.com/jenkinsci/one-shot-executor-plugin
>>
>> the current code base do rely on hacks, our goal is to demonstrate this 
>> use case (can be tested reusing sample 
>> )
>>  
>> so we can get the adequate hooks introduced in jenkins-core and later 
>> re-implement same API on a cleaner basis.
>>
>>
>>
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/f5935828-61d5-4afa-afab-9090deacb19e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: One-Shot Executors

[nicolas de loof]

Dominik edited the wiki page to fix a typo and now it is not available as
https://wiki.jenkins-ci.org/display/JENKINS/One-Shot+Executor anymore, but
still there
https://wiki.jenkins-ci.org/pages/viewpage.action?pageId=96075993

Can a Confluence expert can explain this ?

2016-03-04 12:08 GMT+01:00 nicolas de loof :

> Hi folks,
>
> Yoann and I have extracted from our docker-slaves hack-ish pet project a
> stable sub-component so a plugin can manage slaves that are dedicated to a
> Build, not relying on Cloud API, and get them tied to a Build. i.e Slave
> and Build share a common lifecycle. If Slave fails to start, Build will
> fail as well. If slave is slow to start (maybe pulling a huge docker
> image), build log will report the progress.
> etc.
>
> design and use-case :
> https://wiki.jenkins-ci.org/display/JENKINS/One-Shot+Executor
> code : https://github.com/jenkinsci/one-shot-executor-plugin
>
> the current code base do rely on hacks, our goal is to demonstrate this
> use case (can be tested reusing sample
> )
> so we can get the adequate hooks introduced in jenkins-core and later
> re-implement same API on a cleaner basis.
>
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CANMVJzmdBquxVk9%3DtwVCOi6%2Bk-oKukvhFL%3DKOY9kFcpV-gbeEQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

One-Shot Executors

[nicolas de loof]

Hi folks,

Yoann and I have extracted from our docker-slaves hack-ish pet project a
stable sub-component so a plugin can manage slaves that are dedicated to a
Build, not relying on Cloud API, and get them tied to a Build. i.e Slave
and Build share a common lifecycle. If Slave fails to start, Build will
fail as well. If slave is slow to start (maybe pulling a huge docker
image), build log will report the progress.
etc.

design and use-case :
https://wiki.jenkins-ci.org/display/JENKINS/One-Shot+Executor
code : https://github.com/jenkinsci/one-shot-executor-plugin

the current code base do rely on hacks, our goal is to demonstrate this use
case (can be tested reusing sample
)
so we can get the adequate hooks introduced in jenkins-core and later
re-implement same API on a cleaner basis.

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CANMVJznmnydODNX%2BYXmh0ujnJXsoZWcPT%2BpJCWwdR0_wUpfDTQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.