Re: Starting an asynchronous step

[Jesse Glick]

On Sun, Apr 8, 2018 at 12:06 PM, Jeff Wilson  wrote:
> I have a long running build action
> that involves sending commands to a REST API, getting the results, parsing
> them and writing out artifacts that could be used by a later build step.

Most similar functions use `SynchronousNonBlockingStepExecution` and
abort on Jenkins restart under the guess that the time window here is
pretty small and a restart in the middle is unlikely. Perhaps you can
do better if you want to put in the effort, but I am not sure how—if
you are making a REST call from the Jenkins JVM, then a restart will
of course break that TCP connection.

Simpler options:

· Use `SynchronousNonBlockingStepExecution` but have users wrap the
step in `retry` if they care enough about a single build passing. (I
have not personally tested the combination.)
· Do not bother with a step at all, or a Jenkins plugin for that
matter. Write a little Python program or whatever that does the same
thing and have people call it via `sh`/`bat`. Kills a whole flock of
birds with one little stone.

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr2L82Yi4SZ2zQ_mE%2BJ05ZMocgX7gERA9MEYRyQL1_b6Qg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Starting an asynchronous step

[Jeff Wilson]

And it would have to survive a Jenkins restart, I would think.

On Sunday, April 8, 2018 at 12:06:37 PM UTC-4, Jeff Wilson wrote:
>
> Probably I'm using the wrong terminology. I have a long running build 
> action that involves sending commands to a REST API, getting the results, 
> parsing them and writing out artifacts that could be used by a later build 
> step. This step can also break the build based on the output of the REST 
> calls.  Obviously the Jenkinsfile would have to include things like the URI 
> to the REST API and timeouts, etc. I'm simply looking for an example in 
> case I'm missing out on built in functionality of the pipeline APIs, rather 
> than doing the whole thing myself including the multithreading. The Jenkins 
> pipeline APIs are fairly dense and under-documented (IMO), so I'm having a 
> bit of difficulty getting started.
>
> On Friday, April 6, 2018 at 4:56:35 PM UTC-4, Jesse Glick wrote:
>>
>> On Fri, Apr 6, 2018 at 4:48 PM, Jeff Wilson  wrote: 
>> > So, we basically handle our own multi-threading in the start() command, 
>> as 
>> > in this example? 
>>
>> Yes, it is an asynchronous API—the Pipeline runtime tells your step 
>> when to start, and you tell it when it is finished. What you do in the 
>> meantime is your business. Perhaps nothing is happening in Jenkins—you 
>> may simply be passively awaiting a webhook or something. 
>>
>> > Does that change if, within the StepExecution, I want to trigger a 
>> BuildStep 
>> > asynchronously? 
>>
>> I do not think you want to do that. `BuildStep.perform` is synchronous 
>> (blocking). 
>>
>> If the only thing you are doing is calling some synchronous Jenkins 
>> APIs, and you cannot get around that easily, then you probably just 
>> want to extend `SynchronousNonBlockingStepExecution`. The step will 
>> then not survive a Jenkins restart—since the underlying API it is 
>> calling could not either. 
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/c176acfe-b78c-41d8-814c-6cd88e5ae807%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Starting an asynchronous step

[Jeff Wilson]

Probably I'm using the wrong terminology. I have a long running build 
action that involves sending commands to a REST API, getting the results, 
parsing them and writing out artifacts that could be used by a later build 
step. This step can also break the build based on the output of the REST 
calls.  Obviously the Jenkinsfile would have to include things like the URI 
to the REST API and timeouts, etc. I'm simply looking for an example in 
case I'm missing out on built in functionality of the pipeline APIs, rather 
than doing the whole thing myself including the multithreading. The Jenkins 
pipeline APIs are fairly dense and under-documented (IMO), so I'm having a 
bit of difficulty getting started.

On Friday, April 6, 2018 at 4:56:35 PM UTC-4, Jesse Glick wrote:
>
> On Fri, Apr 6, 2018 at 4:48 PM, Jeff Wilson  > wrote: 
> > So, we basically handle our own multi-threading in the start() command, 
> as 
> > in this example? 
>
> Yes, it is an asynchronous API—the Pipeline runtime tells your step 
> when to start, and you tell it when it is finished. What you do in the 
> meantime is your business. Perhaps nothing is happening in Jenkins—you 
> may simply be passively awaiting a webhook or something. 
>
> > Does that change if, within the StepExecution, I want to trigger a 
> BuildStep 
> > asynchronously? 
>
> I do not think you want to do that. `BuildStep.perform` is synchronous 
> (blocking). 
>
> If the only thing you are doing is calling some synchronous Jenkins 
> APIs, and you cannot get around that easily, then you probably just 
> want to extend `SynchronousNonBlockingStepExecution`. The step will 
> then not survive a Jenkins restart—since the underlying API it is 
> calling could not either. 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/903db347-bc9e-4c8c-acf7-4ea85bd2010e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Starting an asynchronous step

[Jesse Glick]

On Fri, Apr 6, 2018 at 4:48 PM, Jeff Wilson  wrote:
> So, we basically handle our own multi-threading in the start() command, as
> in this example?

Yes, it is an asynchronous API—the Pipeline runtime tells your step
when to start, and you tell it when it is finished. What you do in the
meantime is your business. Perhaps nothing is happening in Jenkins—you
may simply be passively awaiting a webhook or something.

> Does that change if, within the StepExecution, I want to trigger a BuildStep
> asynchronously?

I do not think you want to do that. `BuildStep.perform` is synchronous
(blocking).

If the only thing you are doing is calling some synchronous Jenkins
APIs, and you cannot get around that easily, then you probably just
want to extend `SynchronousNonBlockingStepExecution`. The step will
then not survive a Jenkins restart—since the underlying API it is
calling could not either.

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr33seZi2FMY3t06mUDV8sPFoUze5BHQSao8uQZuPD50dw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Starting an asynchronous step

[Jeff Wilson]

In this example, the important parts are in the setUpTimer() method, as are 
as follows:

task = Timer.get().schedule(new Runnable() {
   @Override public void run() {
  getContext().onSuccess(null);
   }
}, end - now, TimeUnit.MILLISECONDS);

So, we basically handle our own multi-threading in the start() command, as 
in this example?

Does that change if, within the StepExecution, I want to trigger a 
BuildStep asynchronously? 



On Friday, April 6, 2018 at 2:57:04 PM UTC-4, Jesse Glick wrote:
>
> On Fri, Apr 6, 2018 at 10:30 AM, Jeff Wilson  > wrote: 
> > I can't find a (canonical) example -- does anyone 
> > have a pointer to such? 
>
> The `sleep` step is probably the simplest to understand. The most relevant 
> bits: 
>
>
> https://github.com/jenkinsci/workflow-basic-steps-plugin/blob/7d42af656e09eda765befc8d2267064e1a46a824/src/main/java/org/jenkinsci/plugins/workflow/steps/SleepStep.java#L74-L128
>  
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/a1097dbe-8800-4aeb-aa0f-589bc8f5dfee%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Starting an asynchronous step

[Jesse Glick]

On Fri, Apr 6, 2018 at 10:30 AM, Jeff Wilson  wrote:
> I can't find a (canonical) example -- does anyone
> have a pointer to such?

The `sleep` step is probably the simplest to understand. The most relevant bits:

https://github.com/jenkinsci/workflow-basic-steps-plugin/blob/7d42af656e09eda765befc8d2267064e1a46a824/src/main/java/org/jenkinsci/plugins/workflow/steps/SleepStep.java#L74-L128

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr255s5zW%3DSOWG2Yn8RUmiAoy8ST_1LgKkNZE-%3DcKHTE5w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Starting an asynchronous step

[Jeff Wilson]

Indeed. It simply says to implement the start() method as needed. There are 
no examples of what an implementation might look like, nor is there a 
description/documentation of what to do.

On Friday, April 6, 2018 at 12:12:47 PM UTC-4, Robert Sandell wrote:
>
> Perhaps you've already read this guide? 
> https://github.com/jenkinsci/workflow-step-api-plugin/blob/master/README.md#creating-an-asynchronous-step
>
> /B
>
> 2018-04-06 16:30 GMT+02:00 Jeff Wilson 
> :
>
>> I'm trying to write a new pipeline plugin for a build step that is 
>> long-running and prone to network delays and timeouts, so it would appear 
>> that an asynchronous step is the best way to implement this. However, I 
>> have questions about how to implement the StepExecution.start() method to 
>> start this asynchronous step. I can't find a (canonical) example -- does 
>> anyone have a pointer to such? Alternatively, a good development guide that 
>> shows how to implement this type of step?
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Jenkins Developers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to jenkinsci-de...@googlegroups.com .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/jenkinsci-dev/1652f66e-46bc-4d94-8e30-dac8993e817c%40googlegroups.com
>>  
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> -- 
> *Robert Sandell*
> Software Engineer
> CloudBees, Inc.
> [image: CloudBees-Logo.png] 
> E: rsan...@cloudbees.com 
> Twitter: robert_sandell
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/9702d1d6-e186-4fb3-955b-2a04360c46c4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Starting an asynchronous step

[Robert Sandell]

Perhaps you've already read this guide?
https://github.com/jenkinsci/workflow-step-api-plugin/blob/master/README.md#creating-an-asynchronous-step

/B

2018-04-06 16:30 GMT+02:00 Jeff Wilson :

> I'm trying to write a new pipeline plugin for a build step that is
> long-running and prone to network delays and timeouts, so it would appear
> that an asynchronous step is the best way to implement this. However, I
> have questions about how to implement the StepExecution.start() method to
> start this asynchronous step. I can't find a (canonical) example -- does
> anyone have a pointer to such? Alternatively, a good development guide that
> shows how to implement this type of step?
>
> --
> You received this message because you are subscribed to the Google Groups
> "Jenkins Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jenkinsci-dev+unsubscr...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/jenkinsci-dev/1652f66e-46bc-4d94-8e30-dac8993e817c%
> 40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>



-- 
*Robert Sandell*
Software Engineer
CloudBees, Inc.
[image: CloudBees-Logo.png] 
E: rsand...@cloudbees.com
Twitter: robert_sandell

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CALzHZS0Sg_DEXZGfNV2%2BiVMZBjLRVHwezaGSXPaWNUtCFANm1g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Starting an asynchronous step

[Jeff Wilson]

I'm trying to write a new pipeline plugin for a build step that is 
long-running and prone to network delays and timeouts, so it would appear 
that an asynchronous step is the best way to implement this. However, I 
have questions about how to implement the StepExecution.start() method to 
start this asynchronous step. I can't find a (canonical) example -- does 
anyone have a pointer to such? Alternatively, a good development guide that 
shows how to implement this type of step?

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/1652f66e-46bc-4d94-8e30-dac8993e817c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.