[JIRA] [core] (JENKINS-23929) Cannot use 'WORKSPACE' variable when creating custom workspace to *modify* the auto-generated workspace
paul ecc commented on JENKINS-23929 Cannot use WORKSPACE variable when creating custom workspace to *modify* the auto-generated workspace For future reference, I was able to get around this bug simply by clearing the "Directory" field, then unchecking "Use custom workspace" then saving. This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [core] (JENKINS-23929) Cannot use 'WORKSPACE' variable when creating custom workspace to *modify* the auto-generated workspace
Phil Rutherford commented on JENKINS-23929 Cannot use WORKSPACE variable when creating custom workspace to *modify* the auto-generated workspace Our builds are volatile, with a constantly changing process and new data files (and types) added and old ones removed. If you know of a process that can guarantee the same level of effective clean with the same cost/benefit ratio (i.e. 100% guaranteed bug free cleaning and zero work in maintaining it) then I would be interested to know what that is. We do use a branch parameter in the client build spec. But unless you also use a different workspace root then any unused branches in the client spec are removed by Perforce. Have you found a way around that? That would be really useful to know. The branches we want to build change frequently. If we had to set up new jobs all the time it would be a constant maintenance headache. Using a build parameter means fewer jobs and lower maintenance / fewer mistakes. The jobs are run in parallel, as I said in my original post - and because they are effectively the same job we can use the "throttle concurrent builds" plug-in to control the resource usage. We can see which branches are running as we have overridden the default naming convention (as Jenkins allows) so you can see what the important parameters are for each job that is executing. I don't mean to be argumentative for the sake of it, but the bottom line is we now have a solution that works perfectly for our needs. If anyone else wants to do the same thing then you can use my solution, or if you know of a better way please share. Thanks. This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [core] (JENKINS-23929) Cannot use 'WORKSPACE' variable when creating custom workspace to *modify* the auto-generated workspace
Rob Petti commented on JENKINS-23929 Cannot use WORKSPACE variable when creating custom workspace to *modify* the auto-generated workspace "100% guaranteed bug free cleaning and zero work in maintaining it" - There is no free lunch. If these are your requirements, then yes, you are stuck using a full clean, but I still maintain that it's bad practice to rely on that. Your scripts/project files should all be cleaning up things properly, volatile or not. If you don't want the old branches removed, then just put them all in your client spec and only build the branch you want. If you are worried about the additional sync time, then put the branch parameter in the View Mask, and select Use View Mask for Syncing. "If we had to set up new jobs all the time it would be a constant maintenance headache." - I'd recommend using templates. I've currently got a system which reads a list of branches from source control, and automatically creates/updates the necessary jobs from a template. It's a heck of a lot more organized than cramming multiple branches into a single job. The only issue I see with your solution is if you are using the same client for all branches. This is extremely bad practice, and frequently causes issues. So if you aren't already, I'd recommend adding your branch parameter to your client workspace name to avoid these issues. This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [core] (JENKINS-23929) Cannot use 'WORKSPACE' variable when creating custom workspace to *modify* the auto-generated workspace
Phil Rutherford commented on JENKINS-23929 Cannot use WORKSPACE variable when creating custom workspace to *modify* the auto-generated workspace Our cleaning scripts do work reasonably well, but like all code they can get broken and they do change frequently so breakages are statistically probable. They are fine for devs to rely on locally, but I wouldn't be happy to solely rely on them for important client builds and not for an overnight clean where we want to ensure everything is reset. Templates are an interesting idea, not something I have really used so far. I'll have a closer look for future use. Thanks Rob. The client isn't reused. The branch name variable is already part of the client name which Jenkins creates so client workspaces are unique per branch. I agree, it would be bad practice to do otherwise. We are happy with the solution we have at the moment. It is very flexible with branches, reliable and very little work to maintain. The only issue I encountered with setting it up was the problem with custom workspaces which I addressed in my changes. This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [core] (JENKINS-23929) Cannot use 'WORKSPACE' variable when creating custom workspace to *modify* the auto-generated workspace
Phil Rutherford commented on JENKINS-23929 Cannot use WORKSPACE variable when creating custom workspace to *modify* the auto-generated workspace @Rob: We run the P4QUICKCLEANWORKSPACE in overnight clean jobs to ensure the workspaces are clean during the day and there are no leftover unused files that make their way into client deliveries. The incremental CI builds that occur throughout the day do not clean the workspace, that would take far too long for each build. Our builds involve over 80Gb per branch consisting of tens of thousands of files and many many build scripts. Ensure these are always perfect and never leave any unused files behind would be impossible. The short answer is that we need to use branches and we need P4QUICKCLEANWORKSPACE to work. @Morne: Yes, we also don't use the '@' separator as it causes trouble with platform specific tools. You can replace it when you launch Jenkins, e.g.: %JAVA% -Dhudson.slaves.WorkspaceList=_ -jar jenkins.war The changes I made do adhere to this convention. This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [core] (JENKINS-23929) Cannot use 'WORKSPACE' variable when creating custom workspace to *modify* the auto-generated workspace
Rob Petti commented on JENKINS-23929 Cannot use WORKSPACE variable when creating custom workspace to *modify* the auto-generated workspace I have similarly large builds, and don't have a problem with our clean scripts leaving files behind, so it's certainly not 'impossible'. In any event, if you really want a workspace for each branch, and don't want to use branch switching as I suggested earlier, then why not have one job per branch? That'll let you run builds in parallel as well, and more easily identify what code is actually being built. The way you are trying to do it seems like it would only confuse you and your users. This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [core] (JENKINS-23929) Cannot use 'WORKSPACE' variable when creating custom workspace to *modify* the auto-generated workspace
Rob Petti commented on JENKINS-23929 Cannot use WORKSPACE variable when creating custom workspace to *modify* the auto-generated workspace I fail to see the need for checking out each branch into it's own workspace. I perform similar branch selection at build time without any issues simply by using a build parameter in the client view spec. I suggest you do the same. Also, if the clean functionality doesn't work for your use-case, then simply don't use it. Fix your build scripts so they clean up after themselves and you won't have to wipe the workspace at all. This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups Jenkins Issues group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [core] (JENKINS-23929) Cannot use 'WORKSPACE' variable when creating custom workspace to *modify* the auto-generated workspace
Morne Joubert
commented on JENKINS-23929
Cannot use WORKSPACE variable when creating custom workspace to *modify* the auto-generated workspace
I need something very similar since we use Perforce as well.
I can't use the @ separator but i would need a different character e.g. _
I would want to do ${CUSTOM_WORKSPACE}_N where N is the concurrent build index and ${CUSTOM_WORKSPACE} is a parameter i pass in.
The default workspace is too long for us on windows so i want to be able to pass a very short (tiny url) like path.
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators.
For more information on JIRA, see: http://www.atlassian.com/software/jira
--
You received this message because you are subscribed to the Google Groups Jenkins Issues group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[JIRA] [core] (JENKINS-23929) Cannot use 'WORKSPACE' variable when creating custom workspace to *modify* the auto-generated workspace
Phil Rutherford
commented on JENKINS-23929
Cannot use WORKSPACE variable when creating custom workspace to *modify* the auto-generated workspace
Hi Daniel,
I'm afraid that doesn't work. The way the Jenkins core code works any custom workspace is treated as an absolute path so in the case of concurrent builds they would all end up overwriting each other. If you use EXECUTOR_NUMBER in the custom workspace then you run into the problem in my original description above.
I've fixed this myself and created a pull request: https://github.com/jenkinsci/jenkins/pull/1347
My fix means that you can now specify "${WORKSPACE}_${Branch}" as in my original description above. If there are any concurrent builds then they will be named "${WORKSPACE}_${Branch}@N" where '@' is the separator and N is the concurrent build index - not this is not the same as using the EXECUTOR_NUMBER, i.e. the second concurrent build (i.e. N=2) could be running on executor number 4 (if you have 4 or more executors). If the number of concurrent builds is restricted to say 2 then only 2 workspaces will be created, whereas if using EXECUTOR_NUMBER then up to the number of executors worth of workspaces could be created. As I said in the description our workspaces are very large and SSD space is limited so we need to restrict the number of these, particularly as we have multiple branches as well.
If this goes into the master build I'll resolve this issue.
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators.
For more information on JIRA, see: http://www.atlassian.com/software/jira
--
You received this message because you are subscribed to the Google Groups Jenkins Issues group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[JIRA] [core] (JENKINS-23929) Cannot use 'WORKSPACE' variable when creating custom workspace to *modify* the auto-generated workspace
Phil Rutherford
created JENKINS-23929
Cannot use WORKSPACE variable when creating custom workspace to *modify* the auto-generated workspace
Issue Type:
Improvement
Affects Versions:
current
Assignee:
Unassigned
Components:
core
Created:
22/Jul/14 4:19 PM
Description:
In short: We need to have "${WORKSPACE}_${Branch}" as the custom workspace where this simply adds the branch name (a parameter) to the existing auto-created workspace. This doesn't work as the auto-generated WORKSPACE variable has not been created yet as far as I can tell. Please can it be created before the attempt to resolve the workspace and if the "${WORKSPACE}" variable is detected in the declaration then the code follows the same path as for auto-generated workspaces?
This is not the same as using say EXECUTOR_NUMBER in the workspace name in the custom workspace declaration as the auto-generated workspace uses the currently running 'instance' number of a job (where the node has multiple executors) so if you have restricted the number of instances of a job (e.g. "Throttle Concurrent Builds" plugin) then you only get up to the number of allowed instances of workspaces to be created. This is important as our workspaces are very big and consume lots of disk space - SSDs for speed, so space is relatively costly - and we have multiple branches which multiplies up this number even further!
The reason for needing to modify the workspace comes from the Perforce plugin which has useful utility parameters 'P4CLEANWORKSPACE' and 'P4QUICKCLEANWORKSPACE'. However our branches exist within the same workspace directory so if these options are used then they completely blow away any other branch than the one specified. The obvious solution is to modify the workspace (as in the first paragraph above) but without the ability to modify the existing workspace we run into space issues.
Hope I have made a somewhat complex issue clear . I have tried various workarounds (which feel more like hacks), none of which worked, but the best solution really is as described in this report I think. And this would be a useful addition to Jenkins anyway.
Unless anyone can think of a way of doing this?
Project:
Jenkins
Priority:
Major
Reporter:
Phil Rutherford
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators.
For more information on JIRA, see: http://www.atlassian.com/software/jira
--
You received this message because you are subscribed to the Google Groups Jenkins Issues group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[JIRA] [core] (JENKINS-23929) Cannot use 'WORKSPACE' variable when creating custom workspace to *modify* the auto-generated workspace
Daniel Beck
commented on JENKINS-23929
Cannot use WORKSPACE variable when creating custom workspace to *modify* the auto-generated workspace
The workspace is specified relative to the node root directory, by default equivalent to
workspace/${JOB_NAME}
(depending on the value of the Java system property hudson.model.Slave.workspaceRoot). Did you try
workspace/${JOB_NAME}_${Branch}
?
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators.
For more information on JIRA, see: http://www.atlassian.com/software/jira
--
You received this message because you are subscribed to the Google Groups Jenkins Issues group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
