Questions about masking and managing credentials

[nikobelia]

Hi Jenkins users,

Me and a colleague are trying to make our Jenkins infrastructure nice and 
config-as-code-y and reproducible, and it works nicely except that we 
haven't yet found a good way to store and obfuscate credentials.

Here's what we're currently doing:

1.  Store all initial config using the Configuration-as-Code plugin 
(https://github.com/jenkinsci/configuration-as-code-plugin)
2. Store credentials in AWS SSM Parameter Store using the CasC SSM plugin, 
loading them using an instance profile 
(https://github.com/jenkinsci/configuration-as-code-secret-ssm-plugin) 
3. Load all jobs from seed job DSL on first boot

That way, it's easy to replace the Jenkins host for patching, and doing 
that on a regular basis makes sure that all CI jobs and config go through a 
pull request process by cleaning house on a regular basis :)

The problem with this is that AWS Parameter Store only supports storing 
string type parameters, and I can't find a smart way to turn SSM strings 
into files or other credential objects in the configuration as code plugin. 
In theory storing SSH keys/Ansible vault passphrases and similar as strings 
should be OK, as you could just write the secure string into a file in a 
shell step in a job. But in reality, while the logic that redacts 
credentials works nicely for short strings, it seems to be happy to print 
the value of the key out when I do `echo $SSH_KEY > file.txt`.

Does anyone know how that redaction gets evaluated? It says in the log "Masking 
only exact matches of $SECRET or $SSH_KEY". My guess is that line 
breaks/text wrapping is throwing it off. Which is fair enough, since 
there's a perfectly good SSH credential type!

If anyone has ideas about  

- a way to make sure the long secret text value gets redacted
- a way to manage to load AWS SSM Parameter Store SecureStrings as 
something other than text type Jenkins credentials
- or another credential that integrates with the Config-as-Code plugin

I'd appreciate it.

Thanks for reading - have a good weekend!

Nikki


-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/2ba67d0e-b217-4196-bde9-900b3df33d11%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Jenkins - link a java package to pipeline

[Marogo Ytcutc]

The main problem is that Jenkins does not recognize the TestLinkAPIClient 
class, probably because he did not find it in the given location (testlink.
api.java.client.TestLinkAPIClient.*).
How to solve this problem?


W dniu piątek, 12 kwietnia 2019 18:41:23 UTC+2 użytkownik Ivan Fernandez 
Calvo napisał:
>
> Hi,
>
> There are many reasons to not include those classes in your pipeline code 
> I will give you a couple, one it is that you will have to allow to execute 
> a bunch of packages related to that library this would make your Jenkins 
> less secure see 
> https://wiki.jenkins.io/display/JENKINS/Script+Security+Plugin, the 
> second it is related to how we should put into pipeline, the pipeline 
> script language it is designed to orchestrate your pipeline launching your 
> build, deploy, test, ... scripts or whatever, it is not recommended to 
> introduce business logic into your pipeline code, my mate Jesse Glick 
> explains it much better than me in the following keynote "Jenkins World 
> 2017: How to Use Jenkins Less" https://www.youtube.com/watch?v=Zeqc6--0eQw
>
> so my recommendation is to put that code in a script and execute this 
> groovy script from the pipeline, you do not need to include that code in 
> the pipeline.
>
>
>
> El viernes, 12 de abril de 2019, 16:55:08 (UTC+2), Marogo Ytcutc escribió:
>>
>> I would like to use the TestLinkAPIClient class (java) in my pipeline 
>> Jenkins, so I need link these modules 
>> 
>>  to 
>> the pipeline, and then run the code below in groovy (found on the net, so 
>> it is unknown if it can work at all):
>>
>> #!/usr/bin/env groovy
>>  import testlink.api.java.client.TestLinkAPIResults.*
>>  import testlink.api.java.client.TestLinkAPIClient.*
>>
>>  def DEVKEY = "1f123453b123bd8dd811a7f824c194d0"
>>  def URL = "http://PC4/lib/api/xmlrpc/v1/xmlrpc.php;
>>  def api = new TestLinkAPIClient(DEVKEY, URL)
>>  TestLinkAPIResults projects = api.getProjects()
>>  api.createTestProject(...)
>>
>>
>> Ideally, it would be possible to link these modules to the pipeline after 
>> downloading them from the local SVN (where my project is built in the job) 
>> to the job workspace. My pipeline need some reference to the location of 
>> these modules in the %workspace%\testlink\api\java\client folder.
>>
>> Marogo
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/e8a45718-9bfd-407b-9283-b3b047d95be5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: is there any plugins that just sets up a Google cloud environment?

[Filippo Morelli]

I have not and as awesome as it may look I don't think we're in a position
to switch to it right now, thanks tho.

On Fri, Apr 12, 2019 at 7:28 PM Jenn Briden  wrote:

> Have you looked at using Jenkins X?
>
> On Fri, Apr 12, 2019 at 8:56 AM SpikeX 
> wrote:
>
>> Hello,
>>
>> I'm looking for a plugin that sets up the environment before a job is ran
>> so that the job can use gsuti/gcloud utilities and spin up instances/do
>> stuff on Google Cloud.
>>
>> Looking at the plugins on the registry none of them seems to do that
>> and/or work. The closest is the Oauth plugin, but that seems more oriented
>> to getting the service account set up than prepping an environment and I've
>> been unable to get it to work (no secret can even be selected from a
>> binding). Also this plugin doesn't work with pipelines and the
>> recommendation is to just upload the json key as a secret file. This is
>> what I'm doing, however it basically leaves to each build job to duplicate
>> the code necessary to set up the environment so that the google sdk works
>> with out project. The Google SDK plugin is outdated and when I tried I
>> could not get it to work either. All the other plugins seem to work with
>> specific subcomponents like storage or the GCE one is oriented to spinning
>> up slaves on GCE, not use GCE as part of the build process.
>>
>> any pointers?
>>
>> thanks,
>>
>> Spike
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Jenkins Users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to jenkinsci-users+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/jenkinsci-users/951ae074-811d-42a1-acbc-e2076857f268%40googlegroups.com
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Jenkins Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jenkinsci-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jenkinsci-users/CAOC%3D136Vg7DqsT6%2Bua5O4aoo%2BwCqY8s3ao40hdnBVS8SkdfB2Q%40mail.gmail.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>


-- 
Filippo "Spike" Morelli
Xcalar Technical Staff
(415)-727-7453

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/CACqu10PGmK-_oNn-Wsxyjo3y_PqoNDceKcmvUGsn-SH4zokdxw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: is there any plugins that just sets up a Google cloud environment?

[Jenn Briden]

Have you looked at using Jenkins X?

On Fri, Apr 12, 2019 at 8:56 AM SpikeX 
wrote:

> Hello,
>
> I'm looking for a plugin that sets up the environment before a job is ran
> so that the job can use gsuti/gcloud utilities and spin up instances/do
> stuff on Google Cloud.
>
> Looking at the plugins on the registry none of them seems to do that
> and/or work. The closest is the Oauth plugin, but that seems more oriented
> to getting the service account set up than prepping an environment and I've
> been unable to get it to work (no secret can even be selected from a
> binding). Also this plugin doesn't work with pipelines and the
> recommendation is to just upload the json key as a secret file. This is
> what I'm doing, however it basically leaves to each build job to duplicate
> the code necessary to set up the environment so that the google sdk works
> with out project. The Google SDK plugin is outdated and when I tried I
> could not get it to work either. All the other plugins seem to work with
> specific subcomponents like storage or the GCE one is oriented to spinning
> up slaves on GCE, not use GCE as part of the build process.
>
> any pointers?
>
> thanks,
>
> Spike
>
> --
> You received this message because you are subscribed to the Google Groups
> "Jenkins Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jenkinsci-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jenkinsci-users/951ae074-811d-42a1-acbc-e2076857f268%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/CAOC%3D136Vg7DqsT6%2Bua5O4aoo%2BwCqY8s3ao40hdnBVS8SkdfB2Q%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Jenkins - link a java package to pipeline

[Ivan Fernandez Calvo]

Hi,

There are many reasons to not include those classes in your pipeline code I 
will give you a couple, one it is that you will have to allow to execute a 
bunch of packages related to that library this would make your Jenkins less 
secure see https://wiki.jenkins.io/display/JENKINS/Script+Security+Plugin, 
the second it is related to how we should put into pipeline, the pipeline 
script language it is designed to orchestrate your pipeline launching your 
build, deploy, test, ... scripts or whatever, it is not recommended to 
introduce business logic into your pipeline code, my mate Jesse Glick 
explains it much better than me in the following keynote "Jenkins World 
2017: How to Use Jenkins Less" https://www.youtube.com/watch?v=Zeqc6--0eQw

so my recommendation is to put that code in a script and execute this 
groovy script from the pipeline, you do not need to include that code in 
the pipeline.



El viernes, 12 de abril de 2019, 16:55:08 (UTC+2), Marogo Ytcutc escribió:
>
> I would like to use the TestLinkAPIClient class (java) in my pipeline 
> Jenkins, so I need link these modules 
> 
>  to 
> the pipeline, and then run the code below in groovy (found on the net, so 
> it is unknown if it can work at all):
>
> #!/usr/bin/env groovy
>  import testlink.api.java.client.TestLinkAPIResults.*
>  import testlink.api.java.client.TestLinkAPIClient.*
>
>  def DEVKEY = "1f123453b123bd8dd811a7f824c194d0"
>  def URL = "http://PC4/lib/api/xmlrpc/v1/xmlrpc.php;
>  def api = new TestLinkAPIClient(DEVKEY, URL)
>  TestLinkAPIResults projects = api.getProjects()
>  api.createTestProject(...)
>
>
> Ideally, it would be possible to link these modules to the pipeline after 
> downloading them from the local SVN (where my project is built in the job) 
> to the job workspace. My pipeline need some reference to the location of 
> these modules in the %workspace%\testlink\api\java\client folder.
>
> Marogo
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/0adea877-652d-48f7-9e57-38f77b488d9c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Why is node not installing in jenkins?

[adrian lee]

Hi All I am running a docker image of jenkins.

I have installed my plugin node js and named the tool node.

I tried writing a pipeline like below but it keeps telling me :

Does anyone know how tor esolve this?

Unpacking https://nodejs.org/dist/v11.13.0/node-v11.13.0-linux-x64.tar.gz
to /var/jenkins_home/tools/jenkins.plugins.nodejs.tools.NodeJSInstallation/node
on Jenkins[Pipeline] sh
+ npm --version
env: ‘node’: No such file or directory[Pipeline] }[Pipeline] //
node[Pipeline] End of Pipeline



node {
env.NODEJS_HOME = "${tool 'node'}"
env.PATH="${env.NODEJS_HOME}/bin:${env.PATH}"
sh 'npm --version'
}

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/CAJi3E4T_JmLqRKTNCc08hAXqjMhiW-O0DP2WM5nqByt3mCjFag%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Jenkinslibrary stepEnvironmentVariables aren't encoded properly

[Ivan Fernandez Calvo]

I'd guess the step `stepEnvironmentVariables` is in your Pipeline Shared 
Library `@Library('jenkinslib@jira-SBS-8863')_`  because it is not in this 
page https://jenkins.io/doc/pipeline/steps/, so it is not a pipeline step 
provided by some Pipeline plugin, so it is an in house implementation you 
will have to ask the people that have made it

El viernes, 12 de abril de 2019, 9:27:05 (UTC+2), Sarfroz Basha escribió:
>
> Hi All,
>
> I need some help from you guys.
>
> e.g. value "daivb/r/DAIVB2/config-server\ndaivb/r/DAIVB2/registry-server"
> causes build or more specific script-execution to fail with:
>
>
> Error: java.io.IOException: Failed to run image '
> .com:10004/buildcontainers/linux-18.04-base'. Error: docker: invalid 
> reference format: repository name must be lowercase.
>
> I have given wrong environmentID but still build is success. I want to 
> give wrong environment ID and build needs to fail like above Error 
> message. So if I set the variable in replay button like this:-
>
> #!/usr/bin/env groovy
> @Library('jenkinslib@jira-SBS-8863')_
>
> Build(
>  
> pipeline: this,
> releaseBranchPattern: 'test/jira-SBS-8863',
> abortOnFailedTests: true,
> checkoutPostStep: [scriptName : './testScript1'],
> buildPreStep: [scriptName : './testScript2'],
>  stepEnvironmentVariables: [
>  'ENVIRONMENT_ID' : '23\\ / : ; 45d3',
>  'ENVIRONMENT_ID2' : 'JUST A Test',
>  'BUILD _NUMBER' : env.BUILD_ID], 
>  buildPeriodically: '30 08 * * 1-6'
> ); 
>
>
> If value string is encoded as base64 everything works fine. If without 
> encode the value is failing the job with below error. so I don't want to 
> encode the value.
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/5dfb8ddf-1df8-4a6a-87c3-66983a26b604%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

is there any plugins that just sets up a Google cloud environment?

[SpikeX]

Hello,

I'm looking for a plugin that sets up the environment before a job is ran 
so that the job can use gsuti/gcloud utilities and spin up instances/do 
stuff on Google Cloud.

Looking at the plugins on the registry none of them seems to do that and/or 
work. The closest is the Oauth plugin, but that seems more oriented to 
getting the service account set up than prepping an environment and I've 
been unable to get it to work (no secret can even be selected from a 
binding). Also this plugin doesn't work with pipelines and the 
recommendation is to just upload the json key as a secret file. This is 
what I'm doing, however it basically leaves to each build job to duplicate 
the code necessary to set up the environment so that the google sdk works 
with out project. The Google SDK plugin is outdated and when I tried I 
could not get it to work either. All the other plugins seem to work with 
specific subcomponents like storage or the GCE one is oriented to spinning 
up slaves on GCE, not use GCE as part of the build process.

any pointers?

thanks,

Spike

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/951ae074-811d-42a1-acbc-e2076857f268%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Jenkins - link a java package to pipeline

[Marogo Ytcutc]

I would like to use the TestLinkAPIClient class (java) in my pipeline 
Jenkins, so I need link these modules 

 to 
the pipeline, and then run the code below in groovy (found on the net, so 
it is unknown if it can work at all):

#!/usr/bin/env groovy
 import testlink.api.java.client.TestLinkAPIResults.*
 import testlink.api.java.client.TestLinkAPIClient.*

 def DEVKEY = "1f123453b123bd8dd811a7f824c194d0"
 def URL = "http://PC4/lib/api/xmlrpc/v1/xmlrpc.php;
 def api = new TestLinkAPIClient(DEVKEY, URL)
 TestLinkAPIResults projects = api.getProjects()
 api.createTestProject(...)


Ideally, it would be possible to link these modules to the pipeline after 
downloading them from the local SVN (where my project is built in the job) 
to the job workspace. My pipeline need some reference to the location of 
these modules in the %workspace%\testlink\api\java\client folder.

Marogo

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/195bb62e-9067-428c-827a-f0d037715ee4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

issue setting build status from within evaluated groovy script run as system script

['monger_39' via Jenkins Users]

Hi,
I have a job to do some cleanup in Jenkins.
the job is defined to run a "Execute system Groovy script" as build step. The 
code contains something like:

    import hudson.FilePath

    import hudson.model.*

    final GROOVY_SCRIPT ="scripts/do_job_cleanup.groovy"

    evaluate(new FilePath(build.workspace,GROOVY_SCRIPT).read().text)

and within the script:

    doCleanup()  // code that checks and deletes stuff we want removed


    if ( nrIssues > issueThreshold ) {

   println("ERROR: too many issuesfound")

       //currentBuild.result = 'FAIL'

       manager.buildFailure()

   }
During cleanup issues may be found; if we found too many, I want to set status 
of the job to unstable or even fail, depending on some threshold.
Whatever I try, I always get an error when trying to set the status:

ERROR:Build step failed with exception

groovy.lang.MissingPropertyException:No such property: manager for class: 
Script1

   
atorg.codehaus.groovy.runtime.ScriptBytecodeAdapter.unwrap(ScriptBytecodeAdapter.java:53)

   
atorg.codehaus.groovy.runtime.callsite.PogoGetPropertySite.getProperty(PogoGetPropertySite.java:52)

   at 
org.codehaus.groovy.runtime.callsite.AbstractCallSite.callGroovyObjectGetProperty(AbstractCallSite.java:307)

   at Script1.run(Script1.groovy:501)

 I have almost identical code used in another job where the Groovy is run as a 
postbuild step; then it works fine.

Why not here ??
thx, M.




-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/794414828.795738.1555063665312%40mail.yahoo.com.
For more options, visit https://groups.google.com/d/optout.

Jenkinslibrary stepEnvironmentVariables aren't encoded properly

[Sarfroz Basha]

Hi All,

I need some help from you guys.

e.g. value "daivb/r/DAIVB2/config-server\ndaivb/r/DAIVB2/registry-server"
causes build or more specific script-execution to fail with:


Error: java.io.IOException: Failed to run image '
.com:10004/buildcontainers/linux-18.04-base'. Error: docker: invalid 
reference format: repository name must be lowercase.

I have given wrong environmentID but still build is success. I want to give 
wrong environment ID and build needs to fail like above Error message. So 
if I set the variable in replay button like this:-

#!/usr/bin/env groovy
@Library('jenkinslib@jira-SBS-8863')_

Build(
 
pipeline: this,
releaseBranchPattern: 'test/jira-SBS-8863',
abortOnFailedTests: true,
checkoutPostStep: [scriptName : './testScript1'],
buildPreStep: [scriptName : './testScript2'],
 stepEnvironmentVariables: [
 'ENVIRONMENT_ID' : '23\\ / : ; 45d3',
 'ENVIRONMENT_ID2' : 'JUST A Test',
 'BUILD _NUMBER' : env.BUILD_ID], 
 buildPeriodically: '30 08 * * 1-6'
); 


If value string is encoded as base64 everything works fine. If without 
encode the value is failing the job with below error. so I don't want to 
encode the value.


-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/be3e1182-f7d8-49bd-a363-71eb362bc6b6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.