Exposing a step as global variable (or calling it implicitly)

[thomas . weissschuh]

Hi all,

I have written a custom step (let's call it "foo") that allows pipeline 
authors
to configure aspects of their build. The step can be used with our without a
body:

# applies the config to the executed block
foo(config: ...) {
}



# applies the config to the current block
stage("s") {
def config = foo()
config.setting = 1

# or
foo().setting = 1
}



This works very nicely.
The configuration is expected to be called many times in a pipeline 
definition
and for this reason I would like to increase the ergonomics of the syntax
(even more).

# desired syntax
stage("s") {
foo.setting = 1
}



The logic in the "foo" step needs access to the current StepContext, so I 
can't
implement it completely as a global variable, as those don't have access to 
the
context.

Ideas:
a) implement it with a global variable, that directly call the step via the 
CpsScript.
   However the CPS engine always resolves symbol look-ups by looking at 
global
   variables, so my global variable calls itself.
   (I have a PoC that allows GlobalVariables to be exempt from method 
look-ups)
b) Allow steps to declare that they also want to be exposed as a global
   variable, which when accessed will call the Step without arguments?


Do you think this functionality would be in scope of Jenkins?

Thanks,
Thomas

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/ab64b91e-262d-4b25-9408-9e90493bd271%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Modifying Environment variables for each workflow step from a plugin

[thomas . weissschuh]

Hi Jesse,

thanks for the quick response!

On Tuesday, May 1, 2018 at 3:14:48 PM UTC+2, Jesse Glick wrote:
On Mon, Apr 30, 2018 at 10:09 AM,   wrote:

>> Is there a way to modify the value of an environment variable for a 
single
>> step
>> or block in Pipeline from a GraphListener?
>> The value would have to change for each node.

> I do not think so. As of JENKINS-42499, an `EnvironmentContributor`
> could change a value dynamically. This is not scoped to a step; it
> just means that if and when a step requests access to environment
> variables, your plugin will be asked anew for contributions.

Thanks for the hint I will take a look at it.


>> before
>> each build in AbstractBuild.getEnvironment().
>
> `Run.getEnvironment` I guess you mean. Pipeline builds are not 
`AbstractBuild`s.

Yes.

>> I also tried EnvActionImpl.forRun((Run) 
flowNode.getExection().getOwner().getExecutable())

> I am not sure what you are trying to accomplish here, but stop. Do not
> declare a (`compile`-scoped) dependency on `workflow-cps` at all. You
> may use `workflow-step-api` and, if necessary, `workflow-api` as
> dependencies.

Thanks for the hint. The dependency on `workflow-api` was enough for all 
other
ways forward I tried.

>> Now the issue is to propagate the tracing context from within Jenkins to 
the
>> executed steps. in this case Maven for which a similar plugin was 
created.
>> The probably best way of propagating this information is via environment
>> variables.
>> Obviously the value of this environment variable will have to change 
often
>> during the course of a single build.
>
> Just define a step which would provide the current tracing context
> (whatever that is) as its return value. That will work fine in
> Scripted. If you require Declarative compatibility, your best bet is
> to define a block-scoped step which would define some environment
> variable(s) for its nested steps. That is my best recommendation based
> on my very limited grasp of what it is you are trying to do.

This worked best so far, only a few issues remain.
There seems to be a slight API-assumption mismatch between Jenkins and the
component I am trying to integrate it with, which I have to deal with.

The fact that this requires explicit adaptions to the pipeline definition 
is a
bit unfortunate, though.

The goal I am trying to achieve is a deep insight into the CI/build/etc
pipeline.
So if for example a user of the platform complains about slow builds, we can
see exactly where time is spent.
Waiting for executors, running a certain shell script, a single maven step 
and
so on.
And all of this would be stored in a uniform way for queries and analysis.

>> it seems the usage of env vars with a dash ("-") in their keys is
>> broken.
>> Even when set directly via the Java APIs, they don't show up in the "sh"
>> pipeline step.

> https://unix.stackexchange.com/a/23714/26736 

As everything goes through a shell fair enough.
I somehow thought `sh` would be like plain `execve` which was quite far
fetched.

Thanks again.

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/2bed4fdc-63c6-43c7-9e11-944d3e50b162%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Modifying Environment variables for each workflow step from a plugin

[thomas . weissschuh]

Is there a way to modify the value of an environment variable for a single 
step
or block in Pipeline from a GraphListener?
The value would have to change for each node. (For the background read on)

So far I have tried to add a custom EnvironmentContributingAction in
GraphListener.onNewHead(). But those actions seem to be only evaluated 
before
each build in AbstractBuild.getEnvironment().

I also tried EnvActionImpl.forRun((Run) 
flowNode.getExection().getOwner().getExecutable())
in GraphListener.onNewHead() which only changes it for the *next* step, as
apparently the StepContext has already been set up.
(It is probably also not great in case of parallel jobs)

EnvironmentContributor is also documented to affect the whole build.

The best way forward seems to somehow get hold of the StepContext from
GraphListener.onNewHead() but so far I could not find a way to do so.


I would be grateful for any pointers.


Background:

I am trying to integrate an OpenTracing [0] tracer into Jenkins.
It will record executions for all Pipeline runs / queueing processes etc...
The tracing inside Jenkins works (mostly) fine so far.
Now the issue is to propagate the tracing context from within Jenkins to the
executed steps. in this case Maven for which a similar plugin was created.
The probably best way of propagating this information is via environment
variables.
Obviously the value of this environment variable will have to change often
during the course of a single build.
It would also be nice if the plugin could work without any changes to the
Pipeline definition itself.
(It is planned to add optional custom steps/methods)

PS: it seems the usage of env vars with a dash ("-") in their keys is 
broken.
Even when set directly via the Java APIs, they don't show up in the "sh"
pipeline step.

Thanks,
Thomas

[0] http://opentracing.io/

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/0da82c0f-c885-4093-9469-6abb9bb134c1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.