Re: "Team-folder" multibranch pipelines for github

2019-08-06 Thread David Karlsen
https://github.com/jenkinsci/github-branch-source-plugin/pull/214 seems to
implement exactly the feature I am looking for but needs some review-love.
From
https://www.cloudbees.com/blog/ensuring-corporate-standards-pipelines-custom-marker-files
it
seems it is not simply a marker file - but I need to change the actual
Jenkinsfile to have some team-custom name - which works - but is a bit
quirky.

tir. 6. aug. 2019 kl. 14:34 skrev David Karlsen :

> Hi - yes, list repos via teams, or filter repos based on team-access level
> - but the marker-file approach should work equally well - where is this
> configured?
>
> tir. 6. aug. 2019 kl. 14:03 skrev Baptiste Mathus :
>
>> IIUC what is team-folder inside bitbucket, then yes I believe you've got
>> it right by matching it to organization folders.
>> The fact seems to me that given this is what GitHub offers, this is what
>> the plugin could propose?
>>
>> How would you propose, think this should evolve?
>> For instance starting with a GitHub team and find repos what that team is
>> admin? Other?
>>
>> On our organization, mixing many different teams, we've solved this by
>> using "marker files", which we then filter the full GitHub organization
>> with.
>> When such a file is found (say `.our-team`), then it is matched and
>> imported inside a given Jenkins org folder (using ghbs plugin, yes).
>>
>> Cheers
>>
>> Le mar. 6 août 2019 à 13:10, David Karlsen  a
>> écrit :
>>
>>> Bitbuckets offers "teamfolders" - which are multibranch pipelines around
>>> the project scope in bitbucket. One project contains several repos, each
>>> repo has branches - they have Jenkinsfile - you get jobs for several repos
>>> by doing just one manual configuration.
>>>
>>> For github I don't see a similar approach which would be "team-level".
>>>
>>> I can find a multibranch option, but then you need to go off and
>>> configure it by hand once per repo.
>>> I find https://plugins.jenkins.io/github-branch-source - but this seems
>>> to be per gh organization or user ("Scans a GitHub organization (or
>>> user account) for all repositories matching some defined markers.").
>>>
>>> I find organization too wide (as they can grow very large) - and
>>> individuals to narrow - while team-level would be perfect.
>>>
>>> Any thoughts?
>>>
>>> --
>>> --
>>> David J. M. Karlsen - http://www.linkedin.com/in/davidkarlsen
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Jenkins Users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to jenkinsci-users+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/jenkinsci-users/CAGO7Ob2Dvibh-%3Dsmynv1-Da9DR%2BKHmciMXnhBex8gJMX%2B%3D1upA%40mail.gmail.com
>>> 
>>> .
>>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Jenkins Users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to jenkinsci-users+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/jenkinsci-users/CANWgJS51%2BhvJ1wGLHemWJuwfYy0qYo%2BK31GwB1EGO3dcF8qKRw%40mail.gmail.com
>> 
>> .
>>
>
>
> --
> --
> David J. M. Karlsen - http://www.linkedin.com/in/davidkarlsen
>


-- 
--
David J. M. Karlsen - http://www.linkedin.com/in/davidkarlsen

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/CAGO7Ob23zGB_qG0hvSgi5qMeYOd-dx%2BvdhAUzkOQ5SGx8RYHvw%40mail.gmail.com.


Re: Converting to pipeline questions

2019-08-06 Thread Louis Elston
I guess that either I misread this page, or it is out of date already...
https://jenkins.io/doc/book/blueocean/creating-pipelines/ .  It says "You 
now need to specify a local 
 or 
a remote 
 
repository 
from which to build your Pipeline project".  I was trying to test 
converting our existing system to pipeline, and getting permission and 
funding for another large GitHub repository was not in the cards right now, 
so, I was attempting to do this work on a test system, with the repository 
there.  I will try a Git repository on a different\remote system.  We do 
not have Linux, and I do not have that experience (only Windows). 

It also said "Note: Under the hood, a Pipeline project created through Blue 
Ocean is actually "multibranch Pipeline". Therefore, Jenkins looks for the 
presence of at least one Jenkinsfile in any branch of your repository.  So, 
your comment "Blue Ocean is not limited to multibranch Pipelines," is news 
to me.

My version of Jenkins is actually 2.176.2 (on my test VM), as I did 
recently upgrade it.  When I mentioned 2.107, I was probably looking at the 
actual production system.   I cannot even keep the current classic Jenkins 
build system up to date (I see all kinds of red warnings when I go into 
Manage), but it is feared that any change could possible cause a problem 
that could delay a build or the release of the product.  So thus I try to 
work on a test system.  We are behind corp firewalls, so they that figure 
that the production system is safe.  

I had worked my way through the cloudbees Blue Ocean tutorial, but only 
with help from them.  I cannot recommend it.  I have found over the years, 
that people who put together online course and write books, have someone 
who is already familiar with the context, and can fill in the missing 
pieces, do the reviewing.  No one uses someone new to the subject matter 
try to work through it, to see if it really can be understood, and 
completed successfully. Unless you are a real DevOps type, it can be 
difficult to understand the documentation.  Also a lot the 
courses\tutorials\books are using containers.  Again something that I do 
not see them allowing me to play with, so that rules them out.  I had 
recently purchased the book "Beginning Jenkins Blue Ocean".  No where in 
the online information did it mention Docker, but that is what the book 
requires.

I do appreciate you responding, and I will look at the links that you 
listed.


On Tuesday, August 6, 2019 at 12:07:24 PM UTC-4, Mark Waite wrote:
>
>
>
> On Tue, Aug 6, 2019 at 9:36 AM Louis Elston  > wrote:
>
>> I believe that this is a bug.  What do I need to do to either get 
>> comments, or action on this?
>>
>>
> I believe it is not a bug.  Blue Ocean is not designed, tested, or 
> expected to work with a git repository on a local file system.  It is 
> designed, tested, and known to work with remote git servers, including 
> GitHub, Bitbucket, and plain Git.  A pull request is pending to include 
> Perforce support as well.
>
> Why doesn't Blue Ocean support git repositories on a local file system?  
> Git repositories on a local file system are only available from agents that 
> share the same file system.  Most Jenkins best practices include the 
> recommendation, "Do not run builds on the master, use an agent".  Running 
> builds on the master provides the executing job with full access to the 
> file system of the Jenkins master.
>
> Recommendation: Configure a git server and use that git server as your 
> repository.  A git server could be as simple as a Linux computer with a 
> shell account that hosts the bare repository or could include a web 
> interface with Gitea (my favorite for local installation) or Gitlab or 
> could use a remote repository (like GitHub, Bitbucket, Visual Studio, 
> Assembla, Beanstalk, Gitlab, etc.).
>
> For your multibranch question, you need a Jenkinsfile on every branch that 
> you want to run with a Pipeline from SCM.
>
> Blue Ocean is not limited to multibranch Pipelines.  You can use the Blue 
> Ocean editor to create a Pipeline in a git repository that has no 
> Jenkinsfile on any branch.
>
> The Jenkins community is a community.  Members of the community are 
> motivated by different things to decide whether they will respond or not.
>
> In this case, Jenkins 2.107.1 is 15 months old.  The Jenkins community 
> provides security updates for the current long term support release 
> (2.176.2) and current weekly release (2.187).  LTS releases every 3 
> months.  Jenkins 2.107.1 was released 16 months ago.  That is 5 LTS 
> releases ago.  Some hesitation to respond may be due to the outdated 
> version you're running.  There have been many improvements to Jenkins 
> Pipeline in the 5 LTS versions since Jenkins 2.107.1.
>
> There are many different ways that you can learn more 

Re: Converting to pipeline questions

2019-08-06 Thread Mark Waite
On Tue, Aug 6, 2019 at 9:36 AM Louis Elston  wrote:

> I believe that this is a bug.  What do I need to do to either get
> comments, or action on this?
>
>
I believe it is not a bug.  Blue Ocean is not designed, tested, or expected
to work with a git repository on a local file system.  It is designed,
tested, and known to work with remote git servers, including GitHub,
Bitbucket, and plain Git.  A pull request is pending to include Perforce
support as well.

Why doesn't Blue Ocean support git repositories on a local file system?
Git repositories on a local file system are only available from agents that
share the same file system.  Most Jenkins best practices include the
recommendation, "Do not run builds on the master, use an agent".  Running
builds on the master provides the executing job with full access to the
file system of the Jenkins master.

Recommendation: Configure a git server and use that git server as your
repository.  A git server could be as simple as a Linux computer with a
shell account that hosts the bare repository or could include a web
interface with Gitea (my favorite for local installation) or Gitlab or
could use a remote repository (like GitHub, Bitbucket, Visual Studio,
Assembla, Beanstalk, Gitlab, etc.).

For your multibranch question, you need a Jenkinsfile on every branch that
you want to run with a Pipeline from SCM.

Blue Ocean is not limited to multibranch Pipelines.  You can use the Blue
Ocean editor to create a Pipeline in a git repository that has no
Jenkinsfile on any branch.

The Jenkins community is a community.  Members of the community are
motivated by different things to decide whether they will respond or not.

In this case, Jenkins 2.107.1 is 15 months old.  The Jenkins community
provides security updates for the current long term support release
(2.176.2) and current weekly release (2.187).  LTS releases every 3
months.  Jenkins 2.107.1 was released 16 months ago.  That is 5 LTS
releases ago.  Some hesitation to respond may be due to the outdated
version you're running.  There have been many improvements to Jenkins
Pipeline in the 5 LTS versions since Jenkins 2.107.1.

There are many different ways that you can learn more about Jenkins
Pipeline.

   - Tutorials -  https://jenkins.io/doc/tutorials/
   - User Handbook -  https://jenkins.io/doc/book/pipeline/
   - Jenkins Minute videos -
   https://jenkins.io/blog/2017/08/08/introducing-jenkins-minute/
   - CloudBees' free Pipeline Fundamentals core -
   
https://standard.cbu.cloudbees.com/cloudbees-university-jenkins-pipeline-fundamentals
   - Udemy courses on Jenkins Pipeline -
   https://www.udemy.com/courses/search/?src=ukw=jenkins%20pipeline

Mark Waite


> On Thursday, August 1, 2019 at 5:05:02 PM UTC-4, Louis Elston wrote:
>>
>> Studying and playing with pipelines.  I see that you can use Declarative
>> in the Pipeline Scrip window, but it still stores it in the config.xml
>> file.  And I have played with the combination of both Declarative and non
>> Declarative in the same script.
>>
>> I am trying to understand the Blue Ocean interface, the word
>> "MultiBranch" is throwing me a little.  We do not create test branches, and
>> them merge them back into the master.  In the repository, we have branches
>> for each release of the product, and we rarely go back to previous
>> branches\versions.  So, if I am working on branchV9 right now, do I also
>> need a Jenkinsfile in the Master branch, or any other of the previous
>> version branches?
>>
>> I have been playing with Blue Ocean (which only does MultiBranch
>> pipelines).  I am on a Windows system, Jenkins 2.176.2, and have all the
>> latest Blue Ocean plugins as of today (1.18.0).  I am accessing a local Git
>> repository (not GitHub), and am running into the following...
>>
>> If I try to use use “c:\GitRepos\Pipelines1\.git”, i get "not a valid
>> name"...
>>
>> [image: 1.PNG]
>>
>>
>> [image: 2.PNG]
>>
>>
>>
>> [image: 3.PNG]
>>
>>
>> [image: 4.PNG]
>>
>>
>> Why is this happening?
>>
>>
>>
>>
>>
>>
>> On Monday, July 29, 2019 at 11:40:56 AM UTC-4, Louis Elston wrote:
>>>
>>> 07/17/19 – wrote this…
>>>
>>> We are currently using Windows \ Jenkins 2.107.1 (no pipeline), and I am
>>> researching going to pipeline. We have a nightly build job, that fetches
>>> from repositories, and submits and waits on other jobs. I see 9 jobs
>>> running on the same Master node (we only have a master), at the same time.
>>> I am not clear on if we should have one Jenkinsfile or multiple
>>> Jenkinsfiles. It will not be a multibranch pipeline, as we do not create
>>> test branches and then merge back to a master. In the repository we have
>>> product1.0 branch, product2.0 branch etc., and build only one branch (the
>>> latest one). While I do like the Blue Ocean editor, it is only for
>>> MultiBranch pipelines.
>>>
>>> Looking for directions and\or examples on how to convert existing
>>> Jenkins non-pipeline systems, to pipeline.  I did find this…
>>> 

Re: Converting to pipeline questions

2019-08-06 Thread Louis Elston
I believe that this is a bug.  What do I need to do to either get comments, 
or action on this?

On Thursday, August 1, 2019 at 5:05:02 PM UTC-4, Louis Elston wrote:
>
> Studying and playing with pipelines.  I see that you can use Declarative 
> in the Pipeline Scrip window, but it still stores it in the config.xml 
> file.  And I have played with the combination of both Declarative and non 
> Declarative in the same script.
>
> I am trying to understand the Blue Ocean interface, the word "MultiBranch" 
> is throwing me a little.  We do not create test branches, and them merge 
> them back into the master.  In the repository, we have branches for each 
> release of the product, and we rarely go back to previous 
> branches\versions.  So, if I am working on branchV9 right now, do I also 
> need a Jenkinsfile in the Master branch, or any other of the previous 
> version branches?
>
> I have been playing with Blue Ocean (which only does MultiBranch 
> pipelines).  I am on a Windows system, Jenkins 2.176.2, and have all the 
> latest Blue Ocean plugins as of today (1.18.0).  I am accessing a local Git 
> repository (not GitHub), and am running into the following...
>
> If I try to use use “c:\GitRepos\Pipelines1\.git”, i get "not a valid 
> name"...
>
> [image: 1.PNG]
>
>
> [image: 2.PNG]
>
>
>
> [image: 3.PNG]
>
>
> [image: 4.PNG]
>
>
> Why is this happening?
>
>
>
>
>
>
> On Monday, July 29, 2019 at 11:40:56 AM UTC-4, Louis Elston wrote:
>>
>> 07/17/19 – wrote this…
>>
>> We are currently using Windows \ Jenkins 2.107.1 (no pipeline), and I am 
>> researching going to pipeline. We have a nightly build job, that fetches 
>> from repositories, and submits and waits on other jobs. I see 9 jobs 
>> running on the same Master node (we only have a master), at the same time. 
>> I am not clear on if we should have one Jenkinsfile or multiple 
>> Jenkinsfiles. It will not be a multibranch pipeline, as we do not create 
>> test branches and then merge back to a master. In the repository we have 
>> product1.0 branch, product2.0 branch etc., and build only one branch (the 
>> latest one). While I do like the Blue Ocean editor, it is only for 
>> MultiBranch pipelines.
>>
>> Looking for directions and\or examples on how to convert existing Jenkins 
>> non-pipeline systems, to pipeline.  I did find this…
>> https://wiki.jenkins.io/display/JENKINS/Convert+To+Pipeline+Plugin. It 
>> does help a little in that it gives you some converted steps, but cannot 
>> convert all the steps, and will give comments in the pipeline script 
>> "//Unable to convert a build step referring to...please verify and convert 
>> manually if required." There is an option "Recursively convert downstream 
>> jobs if any" and if you select that, it appears to add all the downstream 
>> jobs to the same pipeline script, and really confuses the job parameters. 
>> There is also an option to "Commit JenkinsFile" (if doing declarative). I 
>> will play with this some more, but it is not the be all and end all of 
>> converting to pipeline, and I still am not sure of whether I should be have 
>> one or more scripts.
>>
>> Added 07/26/19 - Let’s see if I have my research to date correct…
>>
>> A Declarative pipeline (Pipeline Script from SCM), is stored in a 
>> Jenkinsfile in the repository. Every time that this Jenkins job is 
>> executed, a fetch from the repository is done (to get the latest version of 
>> the Jenkinsfile).
>>
>> A Pipeline script is stored as part of the config.xml file in the 
>> Jenkins\Jobs folder (it is not stored in the repository, or in a separate 
>> Jenkinsfile in the jobs folder). There is a fetch from the repository only 
>> if you put it in (you do not need to do a repository fetch to get the 
>> Pipeline script).
>>
>> Besides our nightly product build, we also have other jobs. I could 
>> create a separate Declarative Jenkinsfile for each of them (JenkinsfileA, 
>> JenkinsfileB, etc.) for each of the other jobs and store then in the 
>> repository also (in the same branch as the main Jenkinsfile), but that 
>> would mean that every one of those additional jobs, to get the particular 
>> Jenkinsfile for that job, would also need to do a repository fetch 
>> (basically fetching\cloning the repository branch for each job, and have 
>> multiple versions of the repository branch unnecessarily downloaded to the 
>> workspace of each job).
>>
>> That does not make sense to me (unless my understanding of things to date 
>> is incorrect). Because the main product build does require a fetch every 
>> time it is run (to get any possible developer check-ins), I do not see a 
>> problem doing Declarative Jenkinsfile for that job. For the other jobs (if 
>> we do not leave then for the time being in the classic (non-pipeline) 
>> format)), they will be Pipeline scripts.
>>
>> Is there any way of (or plans for), being able to do Declarative pipeline 
>> without having to store in the repository and doing a fetch every time 
>> 

Jenkins pipeline Dependency-Check Plugin v5 migration

2019-08-06 Thread ya
Hi all. 
We use jenkins pipelines and want start migrate to new version of OWASP 
Dependency-Check plugin, but can't understand new syntax.
On github page 
 we 
find only description of new version. 
On jenkins wiki 
 we 
find only free style project example.
On jenkins 
 we 
find new method dependencyCheck without examples. And on google we can't 
find any reference about new syntax.

Our stage in 4.* version
stage("Dependency Check") {
 dependencyCheckAnalyzer datadir: 'dependency-check-data', 
isFailOnErrorDisabled: true, hintsFile: '', includeCsvReports: false, 
includeHtmlReports: true, includeJsonReports: false, isAutoupdateDisabled: 
false, outdir: '', scanpath: '', skipOnScmChange: false, 
skipOnUpstreamChange: false, suppressionFile: '', zipExtensions: ''
 dependencyCheckPublisher canComputeNew: false, defaultEncoding: '', 
healthy: '', pattern: '', unHealthy: ''
 archiveArtifacts allowEmptyArchive: true, artifacts: 
'**/dependency-check-report.*', onlyIfSuccessful: true
  }

So in new version jenkins write: java.lang.NoSuchMethodError: No such DSL 
method 'dependencyCheckAnalyzer' , we understand this cause. 

But when we try do somethink like this in new version, we have just errors: 
 stage("Dependency Check") {
dependency-check //or dependencycheck or dependencyCheck
dependencyCheckPublisher canComputeNew: false, defaultEncoding: '', 
healthy: '', pattern: '', unHealthy: ''
archiveArtifacts allowEmptyArchive: true, artifacts: 
'**/dependency-check-report.*', onlyIfSuccessful: true
  }
ERROR:

groovy.lang.MissingPropertyException: No such property: dependency for class: 
WorkflowScript


Another try:

stage("Dependency Check") {
dependencyCheck datadir: 'dependency-check-data', isFailOnErrorDisabled: 
true, hintsFile: '', includeCsvReports: false, includeHtmlReports: true, 
includeJsonReports: false, isAutoupdateDisabled: false, outdir: '', scanpath: 
'', skipOnScmChange: false, skipOnUpstreamChange: false, suppressionFile: '', 
zipExtensions: '' 
dependencyCheckPublisher canComputeNew: false, defaultEncoding: '', 
healthy: '', pattern: '', unHealthy: ''
archiveArtifacts allowEmptyArchive: true, artifacts: 
'**/dependency-check-report.*', onlyIfSuccessful: true
  }
ERROR:

[DependencyCheck] A Dependency-Check installation was not specified. Please 
configure the build and specify a Dependency-Check installation to use.


Another try:
stage("Dependency Check") {
dependency-check datadir: 'dependency-check-data', isFailOnErrorDisabled: 
true, hintsFile: '', includeCsvReports: false, includeHtmlReports: true, 
includeJsonReports: false, isAutoupdateDisabled: false, outdir: '', scanpath: 
'', skipOnScmChange: false, skipOnUpstreamChange: false, suppressionFile: '', 
zipExtensions: '' 
dependencyCheckPublisher canComputeNew: false, defaultEncoding: '', 
healthy: '', pattern: '', unHealthy: ''
archiveArtifacts allowEmptyArchive: true, artifacts: 
'**/dependency-check-report.*', onlyIfSuccessful: true
  }
ERROR: 

org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
WorkflowScript: 15: expecting '}', found 'datadir' @ line 15, column 26.
   dependency-check datadir: 'dependency-check-data', 
isFailOnErrorDisabled: true, hintsFile: '', includeCsvReports: false, 
includeHtmlReports: true, includeJsonReports: false, isAutoupdateDisabled: 
false, outdir: '', scanpath: '', skipOnScmChange: false, skipOnUpstreamChange: 
false, suppressionFile: '', zipExtensions: '' 



And we read this github description 
, so we need to just use bash 
command in our pipeline to start a dependency check?
Maybe someone who use pipelines with dependency check can write examples how to 
use new version of dependency check with pipelines?

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/21e5e5bb-eebe-433c-984f-fe31f89288c8%40googlegroups.com.


Help me raise attention to an Atlassian feature request about pushing tags to a Bitbucket repo from build server with deployment access key.

2019-08-06 Thread Aslak


We are using access_keys to access repo in Bitbucket on our Jenkins CI 
server to build projects.

Access key for a project is read-only, but it it would have be nice to be 
able to push a build tag after successful build.

https://bitbucket.org/site/master/issues/11266/allow-deployment-keys-to-push-tags

Thanks all

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/adfee94d-63e7-4786-bb3f-4bada4e0a94f%40googlegroups.com.


How to override "Shell executable" in declarative JenkinsFile

2019-08-06 Thread Jérémie Faucher-Goulet
Hi, I find myself in a bit of a bind and would like suggestions on how to 
move forward with our Jenkins cluster.

We've recently updated from a 3 year old version to start using all the 
beautiful pipeline plugins stuff, but would like to slowly transition our 
hundreds of job over time. The first part worked without too many issues 
and our old jobs are running fine on the updated cluster (1 master + 7 
slaves). However I seem to be stuck in regards to the shell executable.

Currently, it is defined as an environment variable in our Jenkins global 
configuration. In this manner:
Shell executable: ${SHELL_EXEC}

Not far above, we have a default value which is defined globally, pointing 
to a location where we have MSYS available on most windows slaves.

Each Linux slaves override this environment variable in this manner in 
their node configuration:
SHELL_EXEC=/bin/sh


When I put a "sh" step in my new declarative file, I get a on a Linux 
node/agent:

*sh: 1: ${SHELL_EXEC}: not found*


I read somewhere that environment variables defined in the slave node 
configuration are not taken into account when using Jenkins pipeline. A pity. 
So I did the following tests

I tried using a environment block:

stage {
environment {
SHELL_EXEC = '/bin/sh'
}
steps {
sh 'echo "Something"'
}
}



But it doesn't work and I get the same error. I also tried using 'withEnv' this 
way:


stage {
steps {
withEnv(["SHELL_EXEC='/bin/sh'"]) {
sh 'echo "This is a print test"'
}
}
}


And I also get the same error.



Does anyone have any suggestion on how to override the 'Shell executable' 
currently set in the global jenkins configuration? I would like to make the 
transition to Jenkins Pipeline as easy as possible and not having to 
reconfigure 100+ jobs manually just so I can blank this global setting.

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/a9d34071-aa69-4529-99a7-76994a82faf3%40googlegroups.com.


Re: "Team-folder" multibranch pipelines for github

2019-08-06 Thread David Karlsen
Hi - yes, list repos via teams, or filter repos based on team-access level
- but the marker-file approach should work equally well - where is this
configured?

tir. 6. aug. 2019 kl. 14:03 skrev Baptiste Mathus :

> IIUC what is team-folder inside bitbucket, then yes I believe you've got
> it right by matching it to organization folders.
> The fact seems to me that given this is what GitHub offers, this is what
> the plugin could propose?
>
> How would you propose, think this should evolve?
> For instance starting with a GitHub team and find repos what that team is
> admin? Other?
>
> On our organization, mixing many different teams, we've solved this by
> using "marker files", which we then filter the full GitHub organization
> with.
> When such a file is found (say `.our-team`), then it is matched and
> imported inside a given Jenkins org folder (using ghbs plugin, yes).
>
> Cheers
>
> Le mar. 6 août 2019 à 13:10, David Karlsen  a
> écrit :
>
>> Bitbuckets offers "teamfolders" - which are multibranch pipelines around
>> the project scope in bitbucket. One project contains several repos, each
>> repo has branches - they have Jenkinsfile - you get jobs for several repos
>> by doing just one manual configuration.
>>
>> For github I don't see a similar approach which would be "team-level".
>>
>> I can find a multibranch option, but then you need to go off and
>> configure it by hand once per repo.
>> I find https://plugins.jenkins.io/github-branch-source - but this seems
>> to be per gh organization or user ("Scans a GitHub organization (or user
>> account) for all repositories matching some defined markers.").
>>
>> I find organization too wide (as they can grow very large) - and
>> individuals to narrow - while team-level would be perfect.
>>
>> Any thoughts?
>>
>> --
>> --
>> David J. M. Karlsen - http://www.linkedin.com/in/davidkarlsen
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Jenkins Users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to jenkinsci-users+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/jenkinsci-users/CAGO7Ob2Dvibh-%3Dsmynv1-Da9DR%2BKHmciMXnhBex8gJMX%2B%3D1upA%40mail.gmail.com
>> 
>> .
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Jenkins Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jenkinsci-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jenkinsci-users/CANWgJS51%2BhvJ1wGLHemWJuwfYy0qYo%2BK31GwB1EGO3dcF8qKRw%40mail.gmail.com
> 
> .
>


-- 
--
David J. M. Karlsen - http://www.linkedin.com/in/davidkarlsen

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/CAGO7Ob1FQFoutXg_%3D5MNYa9P9xebgqaFK_9xTr1LizQTa2cOxQ%40mail.gmail.com.


Re: "Team-folder" multibranch pipelines for github

2019-08-06 Thread Baptiste Mathus
IIUC what is team-folder inside bitbucket, then yes I believe you've got it
right by matching it to organization folders.
The fact seems to me that given this is what GitHub offers, this is what
the plugin could propose?

How would you propose, think this should evolve?
For instance starting with a GitHub team and find repos what that team is
admin? Other?

On our organization, mixing many different teams, we've solved this by
using "marker files", which we then filter the full GitHub organization
with.
When such a file is found (say `.our-team`), then it is matched and
imported inside a given Jenkins org folder (using ghbs plugin, yes).

Cheers

Le mar. 6 août 2019 à 13:10, David Karlsen  a
écrit :

> Bitbuckets offers "teamfolders" - which are multibranch pipelines around
> the project scope in bitbucket. One project contains several repos, each
> repo has branches - they have Jenkinsfile - you get jobs for several repos
> by doing just one manual configuration.
>
> For github I don't see a similar approach which would be "team-level".
>
> I can find a multibranch option, but then you need to go off and configure
> it by hand once per repo.
> I find https://plugins.jenkins.io/github-branch-source - but this seems
> to be per gh organization or user ("Scans a GitHub organization (or user
> account) for all repositories matching some defined markers.").
>
> I find organization too wide (as they can grow very large) - and
> individuals to narrow - while team-level would be perfect.
>
> Any thoughts?
>
> --
> --
> David J. M. Karlsen - http://www.linkedin.com/in/davidkarlsen
>
> --
> You received this message because you are subscribed to the Google Groups
> "Jenkins Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jenkinsci-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jenkinsci-users/CAGO7Ob2Dvibh-%3Dsmynv1-Da9DR%2BKHmciMXnhBex8gJMX%2B%3D1upA%40mail.gmail.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/CANWgJS51%2BhvJ1wGLHemWJuwfYy0qYo%2BK31GwB1EGO3dcF8qKRw%40mail.gmail.com.


"Team-folder" multibranch pipelines for github

2019-08-06 Thread David Karlsen
Bitbuckets offers "teamfolders" - which are multibranch pipelines around
the project scope in bitbucket. One project contains several repos, each
repo has branches - they have Jenkinsfile - you get jobs for several repos
by doing just one manual configuration.

For github I don't see a similar approach which would be "team-level".

I can find a multibranch option, but then you need to go off and configure
it by hand once per repo.
I find https://plugins.jenkins.io/github-branch-source - but this seems to
be per gh organization or user ("Scans a GitHub organization (or user
account) for all repositories matching some defined markers.").

I find organization too wide (as they can grow very large) - and
individuals to narrow - while team-level would be perfect.

Any thoughts?

-- 
--
David J. M. Karlsen - http://www.linkedin.com/in/davidkarlsen

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/CAGO7Ob2Dvibh-%3Dsmynv1-Da9DR%2BKHmciMXnhBex8gJMX%2B%3D1upA%40mail.gmail.com.


Re: Memory issues with Jenkins

2019-08-06 Thread Sverre Moe
Sadly I was mistaken. We do not use NFS for JENKINS_HOME.

We do however use NFS for the location where builds copy the RPM build 
artifacts.

mandag 5. august 2019 22.17.46 UTC+2 skrev Ivan Fernandez Calvo følgende:
>
> Hi, 
>
> Severe has another email thread open, I think it is the same Jenkins 
> instance 
> https://groups.google.com/d/msgid/jenkinsci-users/cc2d0bdb-b15f-4bec-a0a3-0562ea8c7df7%40googlegroups.com?utm_medium=email_source=footer.
>  
> I dunno what happens on your instance but probably it isn’t better that you 
> open another email thread with the description of your issue

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/81039869-a11d-47e0-9c8b-7c2b4774da36%40googlegroups.com.


Re: Jenkins Agents getting disconnected

2019-08-06 Thread Sverre Moe
We do have one NFS, for copying build artifacts to RPM repository.

tirsdag 6. august 2019 09.08.48 UTC+2 skrev Sverre Moe følgende:
>
> I was mistaken. We did not use NFS.
> The disk for JENKINS_HOME (Jenkins running on VM), is a LVM disk.
>
> mandag 29. juli 2019 18.15.20 UTC+2 skrev Ivan Fernandez Calvo følgende:
>>
>> check the Cloudbees links, I've helped to write those KB when I was on 
>> CloudBees :), I'm pretty sure that the NFS is your pain and the root cause 
>> of all your problems if you can rid of it better.
>>
>> El lunes, 29 de julio de 2019, 18:03:59 (UTC+2), slide escribió:
>>>
>>> CloudBees (not my employer) has some resources on using NFS (generally 
>>> the recommendation is to NOT use NFS for JENKINS_HOME). 
>>>
>>>
>>> https://support.cloudbees.com/hc/en-us/articles/115000486312-CJP-Performance-Best-Practices-for-Linux#nfs
>>>  
>>> and
>>> https://support.cloudbees.com/hc/en-us/articles/217479948-NFS-Guide 
>>>
>>> On Mon, Jul 29, 2019 at 8:51 AM Sverre Moe  wrote:
>>>
 Yes, we are using NFS for JENKINS_HOME.

 mandag 29. juli 2019 15.41.00 UTC+2 skrev Ivan Fernandez Calvo følgende:
>
> you have 83 threads in state:IN_NATIVE, probably stuck in IO 
> operations, those 83 threads are blocking the other 382 threads, if you 
> use 
> an NFS or similar device for you Jenkins HOME this is probably your 
> bottleneck, if not check the IO stats on the OS to see where you have the 
> bottleneck.
>
> El lunes, 29 de julio de 2019, 11:20:50 (UTC+2), Sverre Moe escribió:
>>
>> I was unable to determine something from the stack output
>> Here is the result: 
>> https://fastthread.io/my-thread-report.jsp?p=c2hhcmVkLzIwMTkvMDcvMjkvLS1qc3RhY2sudHh0LS05LTE2LTI3
>>
>> torsdag 18. juli 2019 11.28.06 UTC+2 skrev Sverre Moe følgende:
>>>
>>> There is no such reference in my jstack output.
>>> The output says no deadlock detected.
>>> I will try that site for analyzing the jstack.
>>>
>>> Even a normal running Jenkins has many BLOCKED threads. If that is 
>>> normal I don't know.
>>>
>>> We have a test Jenkins instance running on Java 11. That one does 
>>> not have any BLOCKED threads.
>>> Our production Jenkins is running Java 8u181.
>>>
>>> torsdag 18. juli 2019 11.04.16 UTC+2 skrev Ivan Fernandez Calvo 
>>> følgende:

 In that dump I can not see which thread is blocking the others, the 
 jstack output has a reference on each thread that said what thread is 
 the 
 blocker on each thread (- locked <0x> a java.lang.Object), 
 you can try to analyze those thread dump with 
 https://fastthread.io/index.jsp or other online tools to see if 
 you see something relevant, it looks like there is a deadlock.

 https://dzone.com/articles/how-to-read-a-thread-dump

>>> -- 
 You received this message because you are subscribed to the Google 
 Groups "Jenkins Users" group.
 To unsubscribe from this group and stop receiving emails from it, send 
 an email to jenkins...@googlegroups.com.
 To view this discussion on the web visit 
 https://groups.google.com/d/msgid/jenkinsci-users/cc2d0bdb-b15f-4bec-a0a3-0562ea8c7df7%40googlegroups.com
  
 
 .

>>>
>>>
>>> -- 
>>> Website: http://earl-of-code.com
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/d6699e75-4a3c-4125-8b5d-4df0fc12fb47%40googlegroups.com.


Re: Jenkins Agents getting disconnected

2019-08-06 Thread Sverre Moe
I was mistaken. We did not use NFS.
The disk for JENKINS_HOME (Jenkins running on VM), is a LVM disk.

mandag 29. juli 2019 18.15.20 UTC+2 skrev Ivan Fernandez Calvo følgende:
>
> check the Cloudbees links, I've helped to write those KB when I was on 
> CloudBees :), I'm pretty sure that the NFS is your pain and the root cause 
> of all your problems if you can rid of it better.
>
> El lunes, 29 de julio de 2019, 18:03:59 (UTC+2), slide escribió:
>>
>> CloudBees (not my employer) has some resources on using NFS (generally 
>> the recommendation is to NOT use NFS for JENKINS_HOME). 
>>
>>
>> https://support.cloudbees.com/hc/en-us/articles/115000486312-CJP-Performance-Best-Practices-for-Linux#nfs
>>  
>> and
>> https://support.cloudbees.com/hc/en-us/articles/217479948-NFS-Guide 
>>
>> On Mon, Jul 29, 2019 at 8:51 AM Sverre Moe  wrote:
>>
>>> Yes, we are using NFS for JENKINS_HOME.
>>>
>>> mandag 29. juli 2019 15.41.00 UTC+2 skrev Ivan Fernandez Calvo følgende:

 you have 83 threads in state:IN_NATIVE, probably stuck in IO 
 operations, those 83 threads are blocking the other 382 threads, if you 
 use 
 an NFS or similar device for you Jenkins HOME this is probably your 
 bottleneck, if not check the IO stats on the OS to see where you have the 
 bottleneck.

 El lunes, 29 de julio de 2019, 11:20:50 (UTC+2), Sverre Moe escribió:
>
> I was unable to determine something from the stack output
> Here is the result: 
> https://fastthread.io/my-thread-report.jsp?p=c2hhcmVkLzIwMTkvMDcvMjkvLS1qc3RhY2sudHh0LS05LTE2LTI3
>
> torsdag 18. juli 2019 11.28.06 UTC+2 skrev Sverre Moe følgende:
>>
>> There is no such reference in my jstack output.
>> The output says no deadlock detected.
>> I will try that site for analyzing the jstack.
>>
>> Even a normal running Jenkins has many BLOCKED threads. If that is 
>> normal I don't know.
>>
>> We have a test Jenkins instance running on Java 11. That one does not 
>> have any BLOCKED threads.
>> Our production Jenkins is running Java 8u181.
>>
>> torsdag 18. juli 2019 11.04.16 UTC+2 skrev Ivan Fernandez Calvo 
>> følgende:
>>>
>>> In that dump I can not see which thread is blocking the others, the 
>>> jstack output has a reference on each thread that said what thread is 
>>> the 
>>> blocker on each thread (- locked <0x> a java.lang.Object), 
>>> you can try to analyze those thread dump with 
>>> https://fastthread.io/index.jsp or other online tools to see if you 
>>> see something relevant, it looks like there is a deadlock.
>>>
>>> https://dzone.com/articles/how-to-read-a-thread-dump
>>>
>> -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "Jenkins Users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to jenkins...@googlegroups.com.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/jenkinsci-users/cc2d0bdb-b15f-4bec-a0a3-0562ea8c7df7%40googlegroups.com
>>>  
>>> 
>>> .
>>>
>>
>>
>> -- 
>> Website: http://earl-of-code.com
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/da7baa40-bdac-490b-bad6-067aba4591b4%40googlegroups.com.