Re: [Pulp-list] Pulp 2: Docker rsync distributors & Crane
Actually, let's move this discussion to the issue tracker[0].
[0] https://pulp.plan.io/issues/3761#note-2
On Thu, Jun 28, 2018 at 11:07 AM, Dennis Kliban wrote:
> The POC for Crane looks good to me.
>
> Story 3761 seems to be all about making a new rsync distributor for
> Docker. Do you still see that as necessary?
>
> On Fri, Jun 15, 2018 at 11:59 AM, Simon Baatz wrote:
>
>> On Mon, Jun 11, 2018 at 08:25:39AM -0400, Dennis Kliban wrote:
>> >On Wed, Jun 6, 2018 at 9:11 AM, Simon Baatz <[1]gmbno...@gmail.com>
>> >wrote:
>> ...
>> >Is there a way to enable protection for both the redirections and
>> >content? (I know that crane 3.2.0 supports Akamai CDN tokens, but
>> >that does not help with a local server.)
>> >
>> >There is not a way to add content protection for the content itself
>> >right now.
>>
>> We prepared proof-of-concept code to deliver content from Crane/httpd
>> directly without redirecting. This allows to use basic
>> authentication for all content.
>>
>> Commit is at [0] ("/v2" API endpoint only, no "X-Sendfile" support
>> yet). This turned out to be pretty simple. Any feedback
>> is welcome, of course.
>>
>>
>> [0]: https://github.com/Telekom-PD/crane/commit/7b065b1dd96281e31
>> c96de61a03b3293b5c2bd89
>>
>
>
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list
Re: [Pulp-list] Pulp 2: Docker rsync distributors & Crane
The POC for Crane looks good to me.
Story 3761 seems to be all about making a new rsync distributor for Docker.
Do you still see that as necessary?
On Fri, Jun 15, 2018 at 11:59 AM, Simon Baatz wrote:
> On Mon, Jun 11, 2018 at 08:25:39AM -0400, Dennis Kliban wrote:
> >On Wed, Jun 6, 2018 at 9:11 AM, Simon Baatz <[1]gmbno...@gmail.com>
> >wrote:
> ...
> >Is there a way to enable protection for both the redirections and
> >content? (I know that crane 3.2.0 supports Akamai CDN tokens, but
> >that does not help with a local server.)
> >
> >There is not a way to add content protection for the content itself
> >right now.
>
> We prepared proof-of-concept code to deliver content from Crane/httpd
> directly without redirecting. This allows to use basic
> authentication for all content.
>
> Commit is at [0] ("/v2" API endpoint only, no "X-Sendfile" support
> yet). This turned out to be pretty simple. Any feedback
> is welcome, of course.
>
>
> [0]: https://github.com/Telekom-PD/crane/commit/
> 7b065b1dd96281e31c96de61a03b3293b5c2bd89
>
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list
Re: [Pulp-list] Pulp 2: Docker rsync distributors & Crane
On Mon, Jun 11, 2018 at 08:25:39AM -0400, Dennis Kliban wrote:
>On Wed, Jun 6, 2018 at 9:11 AM, Simon Baatz <[1]gmbno...@gmail.com>
>wrote:
...
>Is there a way to enable protection for both the redirections and
>content? (I know that crane 3.2.0 supports Akamai CDN tokens, but
>that does not help with a local server.)
>
>There is not a way to add content protection for the content itself
>right now.
We prepared proof-of-concept code to deliver content from Crane/httpd
directly without redirecting. This allows to use basic
authentication for all content.
Commit is at [0] ("/v2" API endpoint only, no "X-Sendfile" support
yet). This turned out to be pretty simple. Any feedback
is welcome, of course.
[0]:
https://github.com/Telekom-PD/crane/commit/7b065b1dd96281e31c96de61a03b3293b5c2bd89
___
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list
Re: [Pulp-list] Pulp 2: Docker rsync distributors & Crane
On Mon, Jun 11, 2018 at 08:25:39AM -0400, Dennis Kliban wrote: >On Wed, Jun 6, 2018 at 9:11 AM, Simon Baatz <[1]gmbno...@gmail.com> >... > >We did not have a use case for distributing the redirect files. This >would be a good feature to add. If you are interested in adding this >functionality, you should start by filing a Story on [2]pulp.plan.io. >Reply with the issue link here and we can work out the details on the >ticket. I am not sure whether we can contribute this feature (in the end this depends on the complexity). As suggested, I created story #3761 (at [1]) to find out the details and how complex this will be. > - The documentation [0] describes authentication for Crane, but this >authenticates only the redirects delivered by Crane. When adding >basic authentication to the actual content, the Docker daemon will >fail. Apparently, it does not add the credentials when following > the >redirections. >Is there a way to enable protection for both the redirections and >content? (I know that crane 3.2.0 supports Akamai CDN tokens, but >that does not help with a local server.) > >There is not a way to add content protection for the content itself >right now. We found a possible solution: Basic authentication works for content if Crane serves the content directly instead of redirecting. We found that it is surprisingly simple to let Crane do that. As Flask supports "X-Sendfile" out of the box, this should be efficient as well (even more efficient than redirecting. The client does not need the additional round-trip for every artifact.) I think we could post some code soon, which allows to switch between "redirect" and "local content" mode. Should we do the same here and create a story? [1] https://pulp.plan.io/issues/3761 ___ Pulp-list mailing list Pulp-list@redhat.com https://www.redhat.com/mailman/listinfo/pulp-list
Re: [Pulp-list] Pulp 2: Docker rsync distributors & Crane
On Wed, Jun 6, 2018 at 9:11 AM, Simon Baatz wrote: > We looked into Pulp's Docker support recently and ran into surprising > problems. > > Our setup is probably not the usual Pulp & Crane setup: We have > detached content servers to which Pulp pushes yum and iso repositories > using rsync distributors. The content servers are static web servers > that make the repos available to clients. > > We planned to run Crane directly on the content servers using the new > URL rewriting feature (we would like to avoid using a full blown Pulp > installation on those servers). However, this does not seem to work > out of the box: > > - For rpm and iso repos, the rsync publisher uses the output of the > web publisher (pre-distributor). In contrast, the docker rsync > distributor has the web distributor as post-distributor. The > generated tree on the rsync destination can not be used by Crane as > the redirect files are missing. > > I understand that it makes sense to have a web post-distributor if > Crane runs on the Pulp node (or a node with a shared file > system). But is there a reason why the docker rsync distributor > does not distribute the redirect files? > > We did not have a use case for distributing the redirect files. This would be a good feature to add. If you are interested in adding this functionality, you should start by filing a Story on pulp.plan.io. Reply with the issue link here and we can work out the details on the ticket. > - The documentation [0] describes authentication for Crane, but this > authenticates only the redirects delivered by Crane. When adding > basic authentication to the actual content, the Docker daemon will > fail. Apparently, it does not add the credentials when following the > redirections. > > Is there a way to enable protection for both the redirections and > content? (I know that crane 3.2.0 supports Akamai CDN tokens, but > that does not help with a local server.) > > There is not a way to add content protection for the content itself right now. > > [0] https://docs.pulpproject.org/plugins/crane/index.html#user- > authentication > > ___ > Pulp-list mailing list > Pulp-list@redhat.com > https://www.redhat.com/mailman/listinfo/pulp-list > ___ Pulp-list mailing list Pulp-list@redhat.com https://www.redhat.com/mailman/listinfo/pulp-list
