Re: [Pulp-list] working directories proposal
One problem with /var/tmp/ is that some distros, like RHEL, automatically run tmpwatch to delete files in /var/tmp/ that are older than some amount of time (I think 30 days is typical). If any pulp job needed to copy in old files that retained their atime, and tmpwatch happened to run during that job, those files could get deleted out from underneath pulp. It would certainly be safer to use a part of the filesystem that doesn't have a separate tool auto-deleting stuff. Michael - Original Message - From: Randy Barlow rbar...@redhat.com To: pulp-list@redhat.com Sent: Wednesday, December 17, 2014 9:29:14 AM Subject: Re: [Pulp-list] working directories proposal On 12/15/2014 02:54 PM, Michael Hrivnak wrote: - Add a new config value in the 'server' section called 'working_directory'. It's default value would be /var/lib/storage We should have pulp in the path. /var/cache/pulp would be a good choice and works well with the FHS. I'll suggest /var/tmp/pulp as possibly slightly more appropriate than cache, since it is a temporary working space. ___ Pulp-list mailing list Pulp-list@redhat.com https://www.redhat.com/mailman/listinfo/pulp-list ___ Pulp-list mailing list Pulp-list@redhat.com https://www.redhat.com/mailman/listinfo/pulp-list
Re: [Pulp-list] working directories proposal
On 12/19/2014 04:19 PM, Michael Hrivnak wrote: One problem with /var/tmp/ is that some distros, like RHEL, automatically run tmpwatch to delete files in /var/tmp/ that are older than some amount of time (I think 30 days is typical). If any pulp job needed to copy in old files that retained their atime, and tmpwatch happened to run during that job, those files could get deleted out from underneath pulp. It would certainly be safer to use a part of the filesystem that doesn't have a separate tool auto-deleting stuff. Good point. signature.asc Description: OpenPGP digital signature ___ Pulp-list mailing list Pulp-list@redhat.com https://www.redhat.com/mailman/listinfo/pulp-list
Re: [Pulp-list] working directories proposal
On 12/15/2014 02:54 PM, Michael Hrivnak wrote: - Add a new config value in the 'server' section called 'working_directory'. It's default value would be /var/lib/storage We should have pulp in the path. /var/cache/pulp would be a good choice and works well with the FHS. I'll suggest /var/tmp/pulp as possibly slightly more appropriate than cache, since it is a temporary working space. signature.asc Description: OpenPGP digital signature ___ Pulp-list mailing list Pulp-list@redhat.com https://www.redhat.com/mailman/listinfo/pulp-list
Re: [Pulp-list] working directories proposal
- Original Message - - Original Message - From: Dennis Kliban dkli...@redhat.com To: pulp-list pulp-list@redhat.com Sent: Monday, December 15, 2014 2:37:35 PM Subject: [Pulp-list] working directories proposal What we have now: - Storage directory defined here [0] - The above value is used to create a path for working directories here [1] Proposed changes: - Add a new config value in the 'server' section called 'working_directory'. It's default value would be /var/lib/storage We should have pulp in the path. /var/cache/pulp would be a good choice and works well with the FHS. - Change common utils [1] to use ['server']['working_dirctory'] config as base path. - Create 'working_directories' collection in database. - Add a check to repository_working_dir [2] and the rest of *_working_dir methods to determine if this method is called from a task. If it is, add the task id, worker id, and path to 'working_directories' collection. - Create a period task that will check for any tasks in final state with existing working directories and delete them. This will need to take into account that the directories exist on a specific worker. This may be complicated. The directory exists on a specific machine, which may have several workers. Dispatching the cleanup task to a specific worker or specific machine may be difficult. Here's another option that wouldn't require a DB collection. Assume that /var/cache/pulp/ will actually be the value of the working_directory setting. When a worker starts, it creates the directory /var/cache/pulp/worker_id/. If it already exists, any content gets blown away. On exit, the whole directory named worker_id gets deleted. If a task requests temporary storage, the worker creates /var/cache/pulp/worker_id/task_id/ and makes a new temp directory within that. Multiple directories can be requested per task. When the task completes, the worker blows away the entire directory named task_id. I think that covers all of our bases. Thoughts? This is a much more straight forward solution. Can anyone think of situations where we wouldn't want to clean up the working directories when a worker is (re)started? ___ Pulp-list mailing list Pulp-list@redhat.com https://www.redhat.com/mailman/listinfo/pulp-list
