This sounds very useful - can you share it with the community? On Tuesday, 22 November 2016 19:19:41 UTC, Jay wrote: > > We recently developed an internal script to clean and report (html email) > any pipeline instances that were cleaned, and how much space was recovered. > The gocd-janitor utility was a nice reference for what we wanted. We ended > up creating a python script that only removes "failed" builds, and keeps > the console.log files around. For us, the logs are still valuable to look > at and do not take up a considerable amount of space. We also had a more > specific definition of what a "failed" build was. There was also one very > obvious space saver that we noticed. Removing "orphaned" pipeline artifact > folders. This might not occur very frequently, but if you do remove a > pipeline config at any point, you will also want to remove the pipeline > folder on disk, otherwise it just sits around with not references. > > On Tuesday, November 22, 2016 at 10:20:56 AM UTC-8, Ashwanth Kumar wrote: >> >> Leo, I took a quick glance at the repo, it's simple and seems to do it's >> job. Thanks for the contribution. >> >> One feedback though, the approach of deleting anything older than 180 / >> 100 days would work fine if you're doing just CI builds. Cases when you're >> taking artifacts to various pipelines using Go's Artifact dependency - I >> would recommend to give gocd-janitor >> <https://www.google.com/url?q=https%3A%2F%2Fgithub.com%2Fashwanthkumar%2Fgocd-janitor&sa=D&sntz=1&usg=AFQjCNF-RVxIp5H1ZwNRfyQAOU6twPyGxA> >> >> a shot. It looks at all the dependences of each pipeline version and >> maintains the respective versions accordingly. >> >> I've personally been part of various solutions to this problem over the >> past 4+ years and gocd-janitor seems to be the best that works out for us >> so far. >> >> >> On Tue, Nov 22, 2016 at 10:38 PM, Leo Keuken <[email protected]> wrote: >> >>> We were facing pretty much facing the same issue everyone has come >>> across by now I think: constant alerting on full disk on the vm where the >>> go-server lives. Though there is a lot to say for project management of >>> stuff that is being run inside the pipelines and what it saves onto the >>> server in a typical build (e.g. massive test reports from certain suites), >>> all in all it will fill up slowly as no build is ever discarded. With many >>> projects a real headache! >>> >>> Before looking on groups or the www extensively I just went to work and >>> re-invent the wheel (of course) and I wrote a small shell script to get rid >>> of the old builds that usually are obsolete in a fast progressing >>> environment. I've put it on github: >>> https://github.com/LeoK80/go-server-cleanup . Feel free to fork it and >>> make fit for your own purposes or just use it out-of-the-box. Readme will >>> tell you all you need to know about its use. >>> >>> Defaults: >>> - go-server artifacts directory '/var/lib/go-server/artifacts/pipelines' >>> - keep anything younger than 180 days >>> - never delete from a pipeline when there is 15 or less builds present >>> There is parameter input provided for retention & minimum of builds to >>> keep >>> >>> >>> On Wednesday, April 29, 2015 at 10:20:20 PM UTC+1, Jason D wrote: >>>> >>>> Due to problems the default artifact cleanup algorithm of GO can >>>> potentially cause, we have decided to turn it off and write our own. >>>> >>>> Our simple script currently just cycles through the artifacts and >>>> deletes all but the latest xx (configurable) for every pipeline. Very >>>> simple place to start and it works well. >>>> >>>> We'd now like to add a bit more sophistication which brings me to the >>>> questions I have for you all. >>>> >>>> Is there a programatic way (via the GO api, I assume) to tell if a >>>> particular artifact set has been successful or not, given only the info >>>> available from the file system for artifacts? In the next iteration we'd >>>> like to tighten it up a bit and save, for instance, the last xx artifact >>>> sets that were the result of a pipeline that completed successfully, or >>>> similar. >>>> >>>> If others have ideas on how you have managed this issue, I'd be >>>> grateful for your feedback. >>>> >>>> Thanks. >>>> jason >>>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "go-cd" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> >> >> -- >> >> Ashwanth Kumar / ashwanthkumar.in >> >>
-- You received this message because you are subscribed to the Google Groups "go-cd" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
