[JIRA] [core] (JENKINS-23929) Cannot use 'WORKSPACE' variable when creating custom workspace to *modify* the auto-generated workspace
Phil Rutherford commented on JENKINS-23929 Cannot use WORKSPACE variable when creating custom workspace to *modify* the auto-generated workspace Our builds are volatile, with a constantly changing process and new data files (and types) added and old ones removed. If you know of a process that can guarantee the same level of effective clean with the same cost/benefit ratio (i.e. 100% guaranteed bug free cleaning and zero work in maintaining it) then I would be interested to know what that is. We do use a branch parameter in the client build spec. But unless you also use a different workspace root then any unused branches in the client spec are removed by Perforce. Have you found a way around that? That would be really useful to know. The branches we want to build change frequently. If we had to set up new jobs all the time it would be a constant maintenance headache. Using a build parameter means fewer jobs and lower maintenance / fewer mistakes. The jobs are run in parallel, as I said in my original post - and because they are effectively the same job we can use the "throttle concurrent builds" plug-in to control the resource usage. We can see which branches are running as we have overridden the default naming convention (as Jenkins allows) so you can see what the important parameters are for each job that is executing. I don't mean to be argumentative for the sake of it, but the bottom line is we now have a solution that works perfectly for our needs. If anyone else wants to do the same thing then you can use my solution, or if you know of a better way please share. Thanks. This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [core] (JENKINS-23929) Cannot use 'WORKSPACE' variable when creating custom workspace to *modify* the auto-generated workspace
Phil Rutherford commented on JENKINS-23929 Cannot use WORKSPACE variable when creating custom workspace to *modify* the auto-generated workspace Our cleaning scripts do work reasonably well, but like all code they can get broken and they do change frequently so breakages are statistically probable. They are fine for devs to rely on locally, but I wouldn't be happy to solely rely on them for important client builds and not for an overnight clean where we want to ensure everything is reset. Templates are an interesting idea, not something I have really used so far. I'll have a closer look for future use. Thanks Rob. The client isn't reused. The branch name variable is already part of the client name which Jenkins creates so client workspaces are unique per branch. I agree, it would be bad practice to do otherwise. We are happy with the solution we have at the moment. It is very flexible with branches, reliable and very little work to maintain. The only issue I encountered with setting it up was the problem with custom workspaces which I addressed in my changes. This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [core] (JENKINS-23929) Cannot use 'WORKSPACE' variable when creating custom workspace to *modify* the auto-generated workspace
Phil Rutherford commented on JENKINS-23929 Cannot use WORKSPACE variable when creating custom workspace to *modify* the auto-generated workspace @Rob: We run the P4QUICKCLEANWORKSPACE in overnight clean jobs to ensure the workspaces are clean during the day and there are no leftover unused files that make their way into client deliveries. The incremental CI builds that occur throughout the day do not clean the workspace, that would take far too long for each build. Our builds involve over 80Gb per branch consisting of tens of thousands of files and many many build scripts. Ensure these are always perfect and never leave any unused files behind would be impossible. The short answer is that we need to use branches and we need P4QUICKCLEANWORKSPACE to work. @Morne: Yes, we also don't use the '@' separator as it causes trouble with platform specific tools. You can replace it when you launch Jenkins, e.g.: %JAVA% -Dhudson.slaves.WorkspaceList=_ -jar jenkins.war The changes I made do adhere to this convention. This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [core] (JENKINS-23929) Cannot use 'WORKSPACE' variable when creating custom workspace to *modify* the auto-generated workspace
Phil Rutherford
commented on JENKINS-23929
Cannot use WORKSPACE variable when creating custom workspace to *modify* the auto-generated workspace
Hi Daniel,
I'm afraid that doesn't work. The way the Jenkins core code works any custom workspace is treated as an absolute path so in the case of concurrent builds they would all end up overwriting each other. If you use EXECUTOR_NUMBER in the custom workspace then you run into the problem in my original description above.
I've fixed this myself and created a pull request: https://github.com/jenkinsci/jenkins/pull/1347
My fix means that you can now specify "${WORKSPACE}_${Branch}" as in my original description above. If there are any concurrent builds then they will be named "${WORKSPACE}_${Branch}@N" where '@' is the separator and N is the concurrent build index - not this is not the same as using the EXECUTOR_NUMBER, i.e. the second concurrent build (i.e. N=2) could be running on executor number 4 (if you have 4 or more executors). If the number of concurrent builds is restricted to say 2 then only 2 workspaces will be created, whereas if using EXECUTOR_NUMBER then up to the number of executors worth of workspaces could be created. As I said in the description our workspaces are very large and SSD space is limited so we need to restrict the number of these, particularly as we have multiple branches as well.
If this goes into the master build I'll resolve this issue.
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators.
For more information on JIRA, see: http://www.atlassian.com/software/jira
--
You received this message because you are subscribed to the Google Groups Jenkins Issues group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[JIRA] [core] (JENKINS-23929) Cannot use 'WORKSPACE' variable when creating custom workspace to *modify* the auto-generated workspace
Phil Rutherford
created JENKINS-23929
Cannot use WORKSPACE variable when creating custom workspace to *modify* the auto-generated workspace
Issue Type:
Improvement
Affects Versions:
current
Assignee:
Unassigned
Components:
core
Created:
22/Jul/14 4:19 PM
Description:
In short: We need to have "${WORKSPACE}_${Branch}" as the custom workspace where this simply adds the branch name (a parameter) to the existing auto-created workspace. This doesn't work as the auto-generated WORKSPACE variable has not been created yet as far as I can tell. Please can it be created before the attempt to resolve the workspace and if the "${WORKSPACE}" variable is detected in the declaration then the code follows the same path as for auto-generated workspaces?
This is not the same as using say EXECUTOR_NUMBER in the workspace name in the custom workspace declaration as the auto-generated workspace uses the currently running 'instance' number of a job (where the node has multiple executors) so if you have restricted the number of instances of a job (e.g. "Throttle Concurrent Builds" plugin) then you only get up to the number of allowed instances of workspaces to be created. This is important as our workspaces are very big and consume lots of disk space - SSDs for speed, so space is relatively costly - and we have multiple branches which multiplies up this number even further!
The reason for needing to modify the workspace comes from the Perforce plugin which has useful utility parameters 'P4CLEANWORKSPACE' and 'P4QUICKCLEANWORKSPACE'. However our branches exist within the same workspace directory so if these options are used then they completely blow away any other branch than the one specified. The obvious solution is to modify the workspace (as in the first paragraph above) but without the ability to modify the existing workspace we run into space issues.
Hope I have made a somewhat complex issue clear . I have tried various workarounds (which feel more like hacks), none of which worked, but the best solution really is as described in this report I think. And this would be a useful addition to Jenkins anyway.
Unless anyone can think of a way of doing this?
Project:
Jenkins
Priority:
Major
Reporter:
Phil Rutherford
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators.
For more information on JIRA, see: http://www.atlassian.com/software/jira
--
You received this message because you are subscribed to the Google Groups Jenkins Issues group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[JIRA] [job-parameter-summary] (JENKINS-22746) Job Parameter Summary Plugin not reporing Choice Parameters
Phil Rutherford created JENKINS-22746 Job Parameter Summary Plugin not reporing Choice Parameters Issue Type: Bug Affects Versions: current Assignee: Unassigned Components: job-parameter-summary Created: 24/Apr/14 1:33 PM Description: The "Jon Parameter Summary Plugin" is not reporting the string for choice parameters, e.g. Choice Parameter Action Which action to perform on the solution String Parameter Branch = "Main" The branch to build Where the Action here should be "build" but instead is blank. Other parameter types are reported correctly. This happens for all jobs. Project: Jenkins Priority: Major Reporter: Phil Rutherford This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [job-parameter-summary] (JENKINS-22746) Job Parameter Summary Plugin not reporing Choice Parameters
Phil Rutherford updated JENKINS-22746 Job Parameter Summary Plugin not reporing Choice Parameters Change By: Phil Rutherford (24/Apr/14 1:37 PM) Description: The Jon Job ParameterSummaryPluginisnotreportingthestringforchoiceparameters,e.g.ChoiceParameterActionWhichactiontoperformonthesolutionStringParameterBranch=MainThebranchtobuildWheretheActionhereshouldbebuildbutinsteadisblank.Otherparametertypesarereportedcorrectly.Thishappensforalljobs. This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [hudsontrayapp] (JENKINS-22584) Tray app appears to not update properly under latest JRE
Phil Rutherford created JENKINS-22584 Tray app appears to not update properly under latest JRE Issue Type: Bug Affects Versions: current Assignee: Unassigned Components: hudsontrayapp Created: 11/Apr/14 3:28 PM Description: On JRE 7 and latest Jenkins it appears that this app stops updating from the server after a short while, until launched again. Project: Jenkins Priority: Major Reporter: Phil Rutherford This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [matrix] (JENKINS-22562) Cannot restrict matrix *children* to a label
Phil Rutherford created JENKINS-22562 Cannot restrict matrix *children* to a label Issue Type: Improvement Affects Versions: current Assignee: Unassigned Components: matrix Created: 10/Apr/14 9:57 AM Description: It is possible to restrict the matrix parent to a label (see https://issues.jenkins-ci.org/browse/JENKINS-7825), but it is not possible to restrict the matrix child jobs to a label. This would be very useful. Environment: Windows Project: Jenkins Priority: Major Reporter: Phil Rutherford This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [dashboard-view] (JENKINS-22563) Option to show status of individual matrix children
Phil Rutherford created JENKINS-22563 Option to show status of individual matrix children Issue Type: Improvement Assignee: Peter Hayes Components: dashboard-view Created: 10/Apr/14 10:17 AM Description: It would be very useful to have an option to show the status of the children of a matrix configuration build on the dashboard itself - just as you get if you go to the status page of a matrix configuration build. Project: Jenkins Priority: Minor Reporter: Phil Rutherford This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [build-monitor] (JENKINS-22556) Multi-configuration projects to show child matrix in the parent job's view
Phil Rutherford created JENKINS-22556 Multi-configuration projects to show child matrix in the parent jobs view Issue Type: Improvement Assignee: Unassigned Components: build-monitor Created: 09/Apr/14 11:44 PM Description: I really like the layout of the info in this view but there's one thing missing from this plugin for me: It would be great if it could show the status of each of the child jobs in a matrix config inside the 'square' for each parent job, just like you get on the status page of a matrix job in the Jenkins default status view for it. Project: Jenkins Priority: Major Reporter: Phil Rutherford This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
