Try http://ingorichter.blogspot.com/2012/02/jenkins-change-workspaces-and-build.html as a blog posting which shows you how to do that.
Mark Waite On Thu, Jul 24, 2014 at 5:44 AM, bandi pavankumar reddy < [email protected]> wrote: > Hi mark 20 GB repository cloning is not posible in my jenkins but i cloned > that repository in local D drive in "Workspace" folder through bash prompt > and in jenkins slave Remote FS root i mentioned this directory name > "D:/workspace" and next i created one job under this slave "git repository > name and job name i mentioned same and i mentioned 20 Gb repository git url > after then i did build this way fetching details easily.... so easily > jenkins job fetching details and build will be sucess > this way is ok are it will create any issue's.... > > > On Thu, Jul 24, 2014 at 12:40 AM, Mark Waite <[email protected]> > wrote: > >> Much appreciated for the lateral thinking! Thanks for the suggestions. >> I've embedded comments into the items. >> >> >> On Wed, Jul 23, 2014 at 12:54 PM, Rob Mandeville < >> [email protected]> wrote: >> >>> Okay, thinking laterally here. >>> >>> >>> >>> Let’s assume that the Git plugin timeout is hardcoded to 10 minutes. >>> Now, all you have to do is bring the checkout under 10 minutes. This may >>> well be solvable. >>> >>> >>> >> >> The git plugin updates the working directory in a two step process, with >> different interactions and different timeouts applicable to the different >> steps. >> >> >> 1. Fetch remote changes to local repository ("git fetch") - timeout >> can be adjusted by user from the job definition page - "Additional >> Behaviours", "Advanced clone behaviors" - this is network intensive and >> requires work from the central git server, the network, and the local >> server >> 2. Checkout working directory from local repository ("git checkout") >> - timeout cannot be adjust by user without passing a property argument to >> the Java virtual machine - this is disc intensive and only requires work >> from the local server and the local file system. No network operations >> are >> involved in checkout. >> >> >> >>> First, if you haven’t done it already, use shallow checkouts. The Git >>> plugin has had that since 1.1.23 (September 2012). If you’re building, you >>> don’t need the histories, just the current version. If that doesn’t help… >>> >>> >>> >>> Get a sysadmin and profile the pulls. Is your Git server maxing out on >>> CPU or (more likely) disk I/O? Is your client? >>> >>> >>> >> >> I don't think that's their issue, since the fetch phase is not the source >> of the timeout. >> >> >>> If your server is maxxing out, you need to either beef up your server >>> or reduce the load. Increasing disk speed is between you and your >>> sysadmins, assuming that you own the server. Reducing load? Try one or >>> more of these: >>> >>> >>> >>> · Stop polling. If you can use GitLab, there’s a plugin to >>> have GitLab *push* to Jenkins. If you have a dozen polling projects, >>> this will reduce load big-time. There may be other Git push solutions for >>> other Git servers; I don’t know. >>> >> >> That's good advice, but I don't think central server load is their issue, >> since the timeout happens in the "checkout" phase, not the "fetch" phase >> >> >>> · The last time I had checkouts take over 10 minutes (on a >>> proprietary system, not Git), the problem was that nightly builds kicked >>> off all at once and tried to pull 60 branches of the code at once. >>> Solution? Use the >>> https://wiki.jenkins-ci.org/display/JENKINS/Throttle+Concurrent+Builds+Plugin, >>> make pulling from source control into its own step, and only allow 3-5 >>> simultaneous pulls. >>> >>> · If you have to poll, set up your polling schedule with ‘H’ >>> notation (see the help for the polling schedule on your job) to spread the >>> polling around. >>> >>> · Compress the binaries you have in Git. That can’t all be >>> source, can it? >>> >> >> That's excellent advice. If their "checkout" phase is copying large >> binaries from the local git repository to the local git working directory, >> and if they can reduce or eliminate those copies, it may increase the speed >> of the checkout. >> >> >>> · Better yet, put the binaries into something like Artifactory >>> and have the build job pull them down after getting the actual source. >>> >> >> Also excellent advice, and may substantially reduce checkout time by >> shifting the "get the large binaries" activity from a source master >> checkout fetch and checkout to a build time copy from a Artifactory or a >> local cached copy. >> >> >>> If your network is maxxing out, try one or more of these (some assume >>> that you own your Git server hardware and network—if you’re running off of >>> GitHub, some of these won’t work). >>> >>> >>> >>> · Put your build machines (and thus your Jenkins slaves) on the >>> same subnet as the Git server, whether or not the Jenkins server is there >>> as well. If that’s impossible, at least get it to the same site (so it’s >>> all LAN, no WAN). >>> >>> · Replicate the Git server on the subnet your build hosts are >>> on. Git is built to be distributed. >>> >>> · If you can’t put your build farm near your source farm, at >>> least get a Jenkins slave over on the same network as the Git server. Give >>> it a job that polls Git. Rather than actually performing the build, have >>> it compress the sources into a giant Zip file, archives that, then kick off >>> a downstream job (that runs on your local build farm) that unzips the >>> artifact and does the build and test run. You may need plugins to do this >>> right. The upstream job will still be able to tell you the changes made to >>> the source, and point you to the downstream job with the actual results. >>> >>> >>> >>> If the server and network are fine, but your build box is maxxed out on >>> I/O writes, you’re going to have to beef up your hardware (or run fewer >>> builds at once, if you run multiple builds on one host). Get faster drives >>> and/or get a RAID controller for your builds and put it into some sort of >>> striping mode for faster writes. If you just keep your sources and builds >>> on the RAID (having more permanent things like the OS and your compilers on >>> another drive/RAID), you probably don’t have to have that RAID actually be >>> redundant. If a drive blows, you lose your current build, swap out another >>> drive, and try again. >>> >>> >>> >> >> >> >> >>> --Rob >>> >> >> Mark Waite >> >> >>> >>> >>> *From:* [email protected] [mailto: >>> [email protected]] *On Behalf Of *Mark Waite >>> *Sent:* Wednesday, July 23, 2014 1:03 PM >>> *To:* [email protected] >>> *Subject:* Re: Hi friends is it possible clone 18 GB repository in >>> Jenkins.... >>> >>> >>> >>> Please, read my earlier reply >>> <https://groups.google.com/d/msg/jenkinsci-users/xMYzhcYnC0s/vs7VGo7gETIJ> >>> on this same thread. The answer is the same to this question, since they >>> are the same failure. >>> >>> >>> >>> The problem is in the git client plugin or in the combination of the git >>> client plugin, your very large repository, your slow file system, and/or >>> your unwillingness to use "sparse checkout" to reduce the size of the >>> content to be checked out to the working directory. >>> >>> >>> >>> Mark Waite >>> >>> >>> On Wednesday, July 23, 2014 9:24:44 AM UTC-6, bandi pavankumar reddy >>> wrote: >>> >>> Hi mark please suggest me any thing still i am getting this error even i >>> mentioned time out also.. >>> >>> problem is with Jenkins or git.... >>> >>> >>> >>> i configured correctly and i configured time out 60 minutes and while >>> starting build OK after 13 to 14 minutes it was giving error time out..some >>> files fetching and cloning also....but i having total 16 files in that 7 >>> files cloning in 13 minute ....but still 6 file's pending with time out >>> error...i was configured git repository with advanced clone option in that >>> i mentioned time out.....i am sending Error... i tryed with shallow clon >>> also >>> >>> Fetching upstream changes from [email protected]:Uuuu_P_SOURCE.git >>> > C:\Program Files (x86)\Git\bin\git.exe fetch --tags --progress >>> [email protected]:Uuuu_P_SOURCE.git +refs/heads/*:refs/remotes/origin/* >>> > C:\Program Files (x86)\Git\bin\git.exe rev-parse >>> "origin/master^{commit}" >>> Checking out Revision a9f4bc55deac7ceb3ef93799ca15a83cea955242 >>> (origin/master) >>> > C:\Program Files (x86)\Git\bin\git.exe config core.sparsecheckout >>> > C:\Program Files (x86)\Git\bin\git.exe checkout -f >>> a9f4bc55deac7ceb3ef93799ca15a83cea955242 >>> ERROR: Timeout after 10 minutes >>> FATAL: Could not checkout null with start point >>> a9f4bc55deac7ceb3ef93799ca15a83cea955242 >>> hudson.plugins.git.GitException: Could not checkout null with start >>> point a9f4bc55deac7ceb3ef93799ca15a83cea955242 >>> >>> >>> >>> On Sun, Jul 20, 2014 at 12:37 AM, Mark Waite <[email protected]> >>> wrote: >>> >>> The error "Timeout after 10 minutes" in your output shows that you're >>> seeing https://issues.jenkins-ci.org/browse/JENKINS-23476 . The plugin >>> assumes (incorrectly in your case) that a checkout operation (which is >>> entirely local disc I/O) will not need more than 10 minutes. Your >>> repository is so large, or your disc I/O is so slow that you can't checkout >>> within the 10 minute timeout period. >>> >>> >>> >>> You can either use sparse checkout within the git plugin to reduce the >>> time required to perform a checkout, or you can switch from using the git >>> plugin to perform your checkout yourself as a build step. Performing the >>> checkout yourself as a build step is more complicated and more error prone, >>> but does not have the timeout limit. Unfortunately, it also makes Jenkins >>> less useful, because you can no longer see source code changes as part of >>> the job, and you can no longer poll for changes. >>> >>> >>> >>> Mark Waite >>> >>> >>> >>> On Sat, Jul 19, 2014 at 12:46 PM, bandi pavankumar reddy < >>> [email protected]> wrote: >>> >>> i configured correctly and i configured time out 60 minutes and while >>> starting build OK after 13 to 14 minutes it was giving error time out..some >>> files fetching and cloning also....but i having total 16 files in that 7 >>> files cloning in 13 minute ....but still 6 file's pending with time out >>> error...i was configured git repository with advanced clone option in that >>> i mentioned time out.....i am sending Error... i tryed with shallow clon >>> also >>> >>> Fetching upstream changes from [email protected]:Uuuu_P_SOURCE.git >>> > C:\Program Files (x86)\Git\bin\git.exe fetch --tags --progress >>> [email protected]:Uuuu_P_SOURCE.git +refs/heads/*:refs/remotes/origin/* >>> > C:\Program Files (x86)\Git\bin\git.exe rev-parse >>> "origin/master^{commit}" >>> Checking out Revision a9f4bc55deac7ceb3ef93799ca15a83cea955242 >>> (origin/master) >>> > C:\Program Files (x86)\Git\bin\git.exe config core.sparsecheckout >>> > C:\Program Files (x86)\Git\bin\git.exe checkout -f >>> a9f4bc55deac7ceb3ef93799ca15a83cea955242 >>> ERROR: Timeout after 10 minutes >>> FATAL: Could not checkout null with start point >>> a9f4bc55deac7ceb3ef93799ca15a83cea955242 >>> hudson.plugins.git.GitException: Could not checkout null with start >>> point a9f4bc55deac7ceb3ef93799ca15a83cea955242 >>> >>> -- >>> >>> You received this message because you are subscribed to the Google >>> Groups "Jenkins Users" group. >>> >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> >>> >>> For more options, visit https://groups.google.com/d/optout. >>> >>> >>> >>> >>> >>> -- >>> >>> Thanks! >>> >>> Mark Waite >>> >>> -- >>> You received this message because you are subscribed to a topic in the >>> Google Groups "Jenkins Users" group. >>> To unsubscribe from this topic, visit >>> https://groups.google.com/d/topic/jenkinsci-users/xMYzhcYnC0s/unsubscribe >>> . >>> To unsubscribe from this group and all its topics, send an email to >>> [email protected]. >>> For more options, visit https://groups.google.com/d/optout. >>> >>> >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Jenkins Users" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> For more options, visit https://groups.google.com/d/optout. >>> >>> Click here >>> <https://www.mailcontrol.com/sr/J4yitjr+6XbGX2PQPOmvUqW!RI4xIDLrB5NLXDo1k838uaW3K8945cWo+l6VxGMTqv1nt4WDbc336NGKki70Ug==> >>> to report this email as spam. >>> >>> ------------------------------ >>> This e-mail and the information, including any attachments it contains, >>> are intended to be a confidential communication only to the person or >>> entity to whom it is addressed and may contain information that is >>> privileged. If the reader of this message is not the intended recipient, >>> you are hereby notified that any dissemination, distribution or copying of >>> this communication is strictly prohibited. If you have received this >>> communication in error, please immediately notify the sender and destroy >>> the original message. >>> >>> Thank you. >>> >>> Please consider the environment before printing this email. >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Jenkins Users" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> >> >> -- >> Thanks! >> Mark Waite >> >> -- >> You received this message because you are subscribed to a topic in the >> Google Groups "Jenkins Users" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/jenkinsci-users/xMYzhcYnC0s/unsubscribe >> . >> To unsubscribe from this group and all its topics, send an email to >> [email protected]. >> >> For more options, visit https://groups.google.com/d/optout. >> > > -- > You received this message because you are subscribed to the Google Groups > "Jenkins Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout. > -- Thanks! Mark Waite -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
