For future reference, this is the command I'm going to use:
find /var/lib/go-agent*/pipelines -mindepth 1 -maxdepth 1 -type d -mtime
+30 -exec rm -rvf {} \; &> /var/log/go-agent/go-agent_cleanup_cron.log
Intended to remove all pipeline folders that are older than 30 days. I
have it running daily, though probably a bit overkill.
Thanks!
On Wed, Nov 14, 2018 at 9:22 AM Jeff <[email protected]> wrote:
> Thanks all for the feedback! We'll proceed w/ a simple cron job/cleanup
> script for now. I haven't delved into elastic agents yet but may give it a
> look.
>
> Thanks again!
>
> On Wed, Nov 14, 2018 at 8:40 AM Aravind SV <[email protected]>
> wrote:
>
>> Hello!
>>
>> Yes, it is ok to clear out /var/lib/go-agent/pipelines/** any time when a
>> job is not running. Even when it is running, you can clear out directories
>> for other pipelines. Similar to what Jason mentioned about preventing
>> accidental deletion.
>>
>> If you're using elastic agents, then it's lesser of a problem, since
>> everything is removed.
>>
>> Another approach I've seen is to run a cleanup job on a timer. Usually
>> this is run with "Run on all agents". However, it can cause some problems
>> if an agent has gone missing. The pipeline will wait for it. There's an
>> issue open about adding a job starvation timeout in these cases.
>>
>> It's also possible to do it before a job starts, but it will have to be
>> repeated across pipeline configurations. There's no concept of a pre or
>> post job hook for cleanup at the agent level or pipeline level.
>>
>> Cheers,
>> Aravind
>>
>> On Wed, Nov 14, 2018 at 07:26:04 -0800, Jason Smyth wrote:
>> > My understanding is that everything on the agent is disposable once the
>> > agent has completed running its job. The way I have been handling it on
>> my
>> > servers is to manually sort the contents of the pipelines directory by
>> date
>> > and delete a bunch of the oldest ones to free up space whenever an
>> agent
>> > runs too low.
>> >
>> > I have been considering implementing a Scheduled Task (Windows agents)
>> or
>> > cron job (Linux agents) to automatically delete anything older than a
>> day
>> > or two. The time filter would prevent accidentally deleting the working
>> > folder of a pipeline that the agent is currently running. I haven't
>> tried
>> > it yet but I don't see why it wouldn't work.
>> >
>> > Beyond that I, too, wonder what the recommended way of handling this is.
>> >
>> > Hope this helps,
>> > Jason
>> >
>> >
>> > On Tuesday, 13 November 2018 13:26:16 UTC-5, Jeff Vincent wrote:
>> > >
>> > > Out go-agent file systems are slowly filling up. All of the settings
>> for
>> > > cleaning up disk space seem to be geared to server-specific storage
>> of
>> > > artifacts and not agent-specific work spaces and I can't find
>> anything in
>> > > the documentation about the agent storage needs or long-term
>> maintenance.
>> > >
>> > > Can I just clear out the /var/lib/go-agent/pipelines/** folder on a
>> > > schedule, maybe via a cron job? What's the best practice?
>> > >
>> > > Thanks!
>> > >
>> >
>> > --
>> > You received this message because you are subscribed to the Google
>> Groups "go-cd" group.
>> > To unsubscribe from this group and stop receiving emails from it, send
>> an email to [email protected].
>> > For more options, visit https://groups.google.com/d/optout.
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "go-cd" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected].
>> For more options, visit https://groups.google.com/d/optout.
>>
>
--
You received this message because you are subscribed to the Google Groups
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
