Hi Dennis, Justin, On Mon, Dec 18, 2017 at 04:37:23PM -0500, Dennis Kliban wrote: > On Mon, Dec 18, 2017 at 2:49 PM, Justin Sherrill > <[1][email protected]> wrote: > > On 12/18/2017 02:31 PM, Dennis Kliban wrote: > > The use case you are describing is not one that Pulp 2 currently > supports. However, I can picture this feature working in 2 different > ways: > 1) A publisher for a repository with an 'immediate' download policy on > an importer should make sure that all the content it is publishing is > actually downloaded. > 2) A copy of content from an 'on_demand' repo to an 'immediate' repo > should schedule the copied units to be downloaded. > > Could you issue a 'download repo' task to one of the secondary repos to > force those units to download? > pulp-admin repo download > Note that this is *different* than 'pulp-admin rpm repo sync' > It wouldn't be automatic, but could be scheduled. > > Justin, thank you for reminding me of this feature. The REST API is > documented here[0]. Make this request against your 'destination' > repository and this will spawn a task to download all content units > that have not been downloaded yet in that repository.
Thank you for pointing this out! I did not know 'pulp-admin repo' and somehow overlooked the download API call when I searched for a way to trigger downloading in the documentation. It is not automatic, but doing an additional API request is no problem at all for us. I did a quick test and it seems to do what I was looking for: after the call, all necessary content units are available locally and publishing works including the rsync distributors. - Simon > [0] > [2]https://docs.pulpproject.org/dev-guide/integration/rest-api/repo/syn > c.html#download-a-repository > -Dennis > > -Justin > > What do others think? Is there another way to solve this problem? > -Dennis > > On Mon, Dec 18, 2017 at 8:09 AM, Simon Baatz <[3][email protected]> > wrote: > > We are trying to optimize one of our use cases using Pulp's > 'on_demand' download policy, but it does not work as expected (I > reckon that's because the deferred download feature was designed for > a > different use case). > We have one or even multiple large 'source' RPM repos from which we > associate a much smaller number of RPMs into a dedicated > 'destination' > repo. We then publish (locally and using rsync distributors) the > 'destination' repo, which is used for installation on our servers. > Up to now, the source repos sync from their feeds using 'immediate' > as > download policy. As we only need a small subset of the RPMs, we > tested > to set the download policy to 'on_demand' on the source repos in > order > to download the needed RPMs only. We kept 'immediate' for the > 'destination' repo we are associating RPMs into. > However, associating an RPM into that repo does not trigger a > download > (neither immediately nor by the 'deferred_download' task). > Publishing > the 'destination' repo works and content can be downloaded even > though > the download policy is set to 'immediate' (i.e. squid/pulp_streamer > will deliver the content unit as long as it is not present). The > rsync > publisher fails however, because most of the symlinks are pointing > to non-existent destinations. > As a workaround we tried to use another repo whose feed points to > the > 'destination' repo and sync it. This will cause Pulp to download and > store the needed RPMs. > Basically, I have two questions: > - Is this how association between repos with different download > policies is supposed to work? (I would have expected this to not > work at all or to trigger downloading) > - If so, is there a more direct way to ensure or trigger the > download of > all RPMs in our destination repo? > Btw., we tested this on Pulp 2.14.3. > - Simon > _______________________________________________ > Pulp-list mailing list > [4][email protected] > [5]https://www.redhat.com/mailman/listinfo/pulp-list > > _______________________________________________ > Pulp-list mailing list > [6][email protected] > [7]https://www.redhat.com/mailman/listinfo/pulp-list > > _______________________________________________ > Pulp-list mailing list > [8][email protected] > [9]https://www.redhat.com/mailman/listinfo/pulp-list > > References > > 1. mailto:[email protected] > 2. > https://docs.pulpproject.org/dev-guide/integration/rest-api/repo/sync.html#download-a-repository > 3. mailto:[email protected] > 4. mailto:[email protected] > 5. https://www.redhat.com/mailman/listinfo/pulp-list > 6. mailto:[email protected] > 7. https://www.redhat.com/mailman/listinfo/pulp-list > 8. mailto:[email protected] > 9. https://www.redhat.com/mailman/listinfo/pulp-list > _______________________________________________ > Pulp-list mailing list > [email protected] > https://www.redhat.com/mailman/listinfo/pulp-list _______________________________________________ Pulp-list mailing list [email protected] https://www.redhat.com/mailman/listinfo/pulp-list
