Re: [Pulp-list] Change directory that rpms are deposited into?
Hi Sean, Are you referring to the directory where the content is stored long term or the working directory that is used as scratch space as files are downloaded? Generally the recommended means by which the long term storage is recommended to be addressed is to change the mount point for /var/lib/pulp. For the working directory there is an open feature request that should be addressed in Pulp 2.7 but but in the meantime you could change the mount point for /var/lib/pulp/working. If I misunderstood the question entirely please provide more details and i'll give it another shot. Regards, -Barnaby - Original Message - From: Sean Waite swa...@tracelink.com To: pulp-list@redhat.com Sent: Friday, December 12, 2014 6:07:19 PM Subject: [Pulp-list] Change directory that rpms are deposited into? Hi, Is it possible to change the directory that an rpm repo sync places the rpm files into? Thanks, -- Sean Waite swa...@tracelink.com Cloud Operations Engineer GPG 17F91B3A TraceLink, Inc. Be Excellent to Each Other ___ Pulp-list mailing list Pulp-list@redhat.com https://www.redhat.com/mailman/listinfo/pulp-list ___ Pulp-list mailing list Pulp-list@redhat.com https://www.redhat.com/mailman/listinfo/pulp-list
Re: [Pulp-list] Change directory that rpms are deposited into?
Hi Sean, FWIWI that's what we do.. /var/lib/pulp is an NFS mount. And recently did the same for /var/lib/mongodb, where mongo stores its data, as it became too much for the root partition. -Original Message- From: pulp-list-boun...@redhat.com [mailto:pulp-list-boun...@redhat.com] On Behalf Of Barnaby Court Sent: Monday, December 15, 2014 10:01 AM To: Sean Waite Cc: pulp-list@redhat.com Subject: Re: [Pulp-list] Change directory that rpms are deposited into? Hi Sean, Are you referring to the directory where the content is stored long term or the working directory that is used as scratch space as files are downloaded? Generally the recommended means by which the long term storage is recommended to be addressed is to change the mount point for /var/lib/pulp. For the working directory there is an open feature request that should be addressed in Pulp 2.7 but but in the meantime you could change the mount point for /var/lib/pulp/working. If I misunderstood the question entirely please provide more details and i'll give it another shot. Regards, -Barnaby - Original Message - From: Sean Waite swa...@tracelink.com To: pulp-list@redhat.com Sent: Friday, December 12, 2014 6:07:19 PM Subject: [Pulp-list] Change directory that rpms are deposited into? Hi, Is it possible to change the directory that an rpm repo sync places the rpm files into? Thanks, -- Sean Waite swa...@tracelink.com Cloud Operations Engineer GPG 17F91B3A TraceLink, Inc. Be Excellent to Each Other ___ Pulp-list mailing list Pulp-list@redhat.com https://www.redhat.com/mailman/listinfo/pulp-list ___ Pulp-list mailing list Pulp-list@redhat.com https://www.redhat.com/mailman/listinfo/pulp-list Information in this e-mail may be confidential. It is intended only for the addressee(s) identified above. If you are not the addressee(s), or an employee or agent of the addressee(s), please note that any dissemination, distribution, or copying of this communication is strictly prohibited. If you have received this e-mail in error, please notify the sender of the error. ___ Pulp-list mailing list Pulp-list@redhat.com https://www.redhat.com/mailman/listinfo/pulp-list
Re: [Pulp-list] Change directory that rpms are deposited into?
Same here /var/lib/pulp/content and /var/lib/pulp/working are symbolic links on a network volume (provided by Cinder and supported by Netapp storage bay) mongo database is also stored on a network volume On Mon, Dec 15, 2014 at 4:10 PM, Ashby, Jason (IMS) ash...@imsweb.com wrote: Hi Sean, FWIWI that's what we do.. /var/lib/pulp is an NFS mount. And recently did the same for /var/lib/mongodb, where mongo stores its data, as it became too much for the root partition. -Original Message- From: pulp-list-boun...@redhat.com [mailto:pulp-list-boun...@redhat.com] On Behalf Of Barnaby Court Sent: Monday, December 15, 2014 10:01 AM To: Sean Waite Cc: pulp-list@redhat.com Subject: Re: [Pulp-list] Change directory that rpms are deposited into? Hi Sean, Are you referring to the directory where the content is stored long term or the working directory that is used as scratch space as files are downloaded? Generally the recommended means by which the long term storage is recommended to be addressed is to change the mount point for /var/lib/pulp. For the working directory there is an open feature request that should be addressed in Pulp 2.7 but but in the meantime you could change the mount point for /var/lib/pulp/working. If I misunderstood the question entirely please provide more details and i'll give it another shot. Regards, -Barnaby - Original Message - From: Sean Waite swa...@tracelink.com To: pulp-list@redhat.com Sent: Friday, December 12, 2014 6:07:19 PM Subject: [Pulp-list] Change directory that rpms are deposited into? Hi, Is it possible to change the directory that an rpm repo sync places the rpm files into? Thanks, -- Sean Waite swa...@tracelink.com Cloud Operations Engineer GPG 17F91B3A TraceLink, Inc. Be Excellent to Each Other ___ Pulp-list mailing list Pulp-list@redhat.com https://www.redhat.com/mailman/listinfo/pulp-list ___ Pulp-list mailing list Pulp-list@redhat.com https://www.redhat.com/mailman/listinfo/pulp-list Information in this e-mail may be confidential. It is intended only for the addressee(s) identified above. If you are not the addressee(s), or an employee or agent of the addressee(s), please note that any dissemination, distribution, or copying of this communication is strictly prohibited. If you have received this e-mail in error, please notify the sender of the error. ___ Pulp-list mailing list Pulp-list@redhat.com https://www.redhat.com/mailman/listinfo/pulp-list ___ Pulp-list mailing list Pulp-list@redhat.com https://www.redhat.com/mailman/listinfo/pulp-list
[Pulp-list] working directories proposal
What we have now: - Storage directory defined here [0] - The above value is used to create a path for working directories here [1] Proposed changes: - Add a new config value in the 'server' section called 'working_directory'. It's default value would be /var/lib/storage - Change common utils [1] to use ['server']['working_dirctory'] config as base path. - Create 'working_directories' collection in database. - Add a check to repository_working_dir [2] and the rest of *_working_dir methods to determine if this method is called from a task. If it is, add the task id, worker id, and path to 'working_directories' collection. - Create a period task that will check for any tasks in final state with existing working directories and delete them. This will need to take into account that the directories exist on a specific worker. Does this make sense? [0] https://github.com/dkliban/pulp/blob/master/server/pulp/server/config.py#L85 [1] https://github.com/pulp/pulp/blob/master/server/pulp/server/managers/repo/_common.py#L264 [2] https://github.com/pulp/pulp/blob/master/server/pulp/server/managers/repo/_common.py#L82 ___ Pulp-list mailing list Pulp-list@redhat.com https://www.redhat.com/mailman/listinfo/pulp-list
Re: [Pulp-list] working directories proposal
- Original Message - - Original Message - From: Dennis Kliban dkli...@redhat.com To: pulp-list pulp-list@redhat.com Sent: Monday, December 15, 2014 2:37:35 PM Subject: [Pulp-list] working directories proposal What we have now: - Storage directory defined here [0] - The above value is used to create a path for working directories here [1] Proposed changes: - Add a new config value in the 'server' section called 'working_directory'. It's default value would be /var/lib/storage We should have pulp in the path. /var/cache/pulp would be a good choice and works well with the FHS. - Change common utils [1] to use ['server']['working_dirctory'] config as base path. - Create 'working_directories' collection in database. - Add a check to repository_working_dir [2] and the rest of *_working_dir methods to determine if this method is called from a task. If it is, add the task id, worker id, and path to 'working_directories' collection. - Create a period task that will check for any tasks in final state with existing working directories and delete them. This will need to take into account that the directories exist on a specific worker. This may be complicated. The directory exists on a specific machine, which may have several workers. Dispatching the cleanup task to a specific worker or specific machine may be difficult. Here's another option that wouldn't require a DB collection. Assume that /var/cache/pulp/ will actually be the value of the working_directory setting. When a worker starts, it creates the directory /var/cache/pulp/worker_id/. If it already exists, any content gets blown away. On exit, the whole directory named worker_id gets deleted. If a task requests temporary storage, the worker creates /var/cache/pulp/worker_id/task_id/ and makes a new temp directory within that. Multiple directories can be requested per task. When the task completes, the worker blows away the entire directory named task_id. I think that covers all of our bases. Thoughts? This is a much more straight forward solution. Can anyone think of situations where we wouldn't want to clean up the working directories when a worker is (re)started? ___ Pulp-list mailing list Pulp-list@redhat.com https://www.redhat.com/mailman/listinfo/pulp-list
