Re: [foreman-dev] Re: [RFC] HTTP proxy for requests

2017-06-22 Thread Sebastian Gräßl
I haven't thought as far as having it as an official plugin yet and not 
considered all the necessary parts for that.

I can also add it into core.

My thoughts on preferring to have it as a plugin were that, 
even though the code should be relatively small,
as it would probably only be initialiser code to set the HTTP proxy for 
various http libraries,
it does have a big impact on the whole application and 
having it as a plugin allows to disable all of it's doing by uninstalling 
it.

I actually also briefly thought that it be a generic rubygem, but that 
would have gone way too far. :D

On Thursday, June 22, 2017 at 2:08:13 PM UTC+2, Marek Hulán wrote:

> Why do you want to introduce it in a new plugin? I can see the global 
> proxy 
> setting in core. When we get to the point when we want to have separate 
> proxy 
> per library/plugin/communication then I think we should start using the 
> foreman_http_proxies plugin and enhance it if needed. From my experience, 
> starting a plugin brings a lot of extra maintenance effort, such as test 
> infrastructure, redmine setup, plugin manual write-up, hammer plugin from 
> scratch, support in installer. Why not simply add a new setting in 
> Foreman? 
>
> -- 
> Marek 
>
> On středa 21. června 2017 16:28:38 CEST Sebastian Gräßl wrote: 
> > The biggest part of this all is actually ensuring that all requests made 
> by 
> > the application are actually going through the HTTP proxy. 
> > 
> > As a solution to that, I was thinking of starting a plugin that 
> configures 
> > the proxy for HTTP libraries (Net::HTTP, Excon, RestClient, etc.) used. 
> > At first it would only make sure that we have all requests covered. 
> > As a nicety it could also have a debug mode to show outgoing requests 
> via 
> > something like httplog[1]. 
> > 
> > The HTTP proxy would at first just be one global setting, but later on 
> by 
> > extending the used libraries' underlying request methods it could 
> > also allow for dynamically choosing the appropriate proxy per request. 
> > This then can be used by the foreman_http_proxies[2] plugin. 
> > 
> > [1] https://github.com/trusche/httplog 
> > [2] https://github.com/jlsherrill/foreman_http_proxies 
> > 
> > On Thursday, May 25, 2017 at 11:58:12 AM UTC+2, Marek Hulán wrote: 
> > > sorry for typo, it should have been: 
> > > 
> > > AFAIK we don't *have* the solution *implemented* atm. 
> > > 
> > > On čtvrtek 25. května 2017 11:49:24 CEST Marek Hulán wrote: 
> > > > Sorry for being late to the party, sending my 2c: 
> > > > 
> > > > I agree with having more complicated solution where users could have 
> > > > separate proxies per service is good long-term goal. AFAIK we don't 
> the 
> > > > solution atm. Therefore I think introducing support for single, 
> global 
> > > > proxy sounds as improvement already to what we have now (nothing). 
> > > 
> > > What's 
> > > 
> > > > good on this, migrating to specific proxies should be easy, the RFC 
> > > > explicitly[1] mentions it. The global proxy can have granular rules 
> of 
> > > 
> > > what 
> > > 
> > > > communication should be passed through untouched and what should be 
> sent 
> > > > through maybe other proxies. Another advantage I see is that the 
> global 
> > > > proxy offloads the configuration from Foreman/Katello, which does 
> not 
> > > > really belongs into our domain. 
> > > > 
> > > > Later, when the RFC is implemented via foreman_http_proxies plugin, 
> I'm 
> > > > happy to stop using global proxy and improve plugins to use 
> > > > foreman_http_proxies if it makes sense. It will take some time 
> before 
> > > > everyone adopts it. But meanwhile we'd still have the option to let 
> user 
> > > > configure their master proxy according to their needs. 
> > > > 
> > > > [1] https://github.com/theforeman/rfcs/pull/18/ 
> > > > files#diff-12584a6580dac145ae55c2b5d67088dfR45 
> > > > 
> > > > > On 05/17/2017 07:57 AM, Tom McKay wrote: 
> > > > > > After reading the RFC I think that more robust and adaptable 
> > > 
> > > solution 
> > > 
> > > > > > would be better. A single env var is not going to cover the 
> needs of 
> > > > > > all the scenarios. A simple example may be accessing both 
> > > > > > registry.access.redhat.com  
> > > > > > (through proxy) and myopenshift:5000 (no proxy). 
> > > > > > 
> > > > > > As @jlsherrill noted on the PR, the temporary solution for the 
> > > > > > foreman-docker plugin is alright for the moment. 
> > > > > 
> > > > > I'd like to echo what tom said, we've had many users that want to 
> > > 
> > > access 
> > > 
> > > > > content externally through a proxy and internally (where the proxy 
> is 
> > > > > not controlled by them and does not properly proxy internal 
> requests). 
> > > > > Its happened enough for me to say that a simple solution is not 
> good 
> > > > > enough long term. 
> > > > > 
> > > > > > On Wed, May 17, 2017 at 3:08 AM, Sebastian Gräßl 
> > > > > > 
> > > > > >  

Re: [foreman-dev] Re: [RFC] HTTP proxy for requests

2017-06-22 Thread Marek Hulán
Why do you want to introduce it in a new plugin? I can see the global proxy 
setting in core. When we get to the point when we want to have separate proxy 
per library/plugin/communication then I think we should start using the 
foreman_http_proxies plugin and enhance it if needed. From my experience, 
starting a plugin brings a lot of extra maintenance effort, such as test 
infrastructure, redmine setup, plugin manual write-up, hammer plugin from 
scratch, support in installer. Why not simply add a new setting in Foreman?

--
Marek

On středa 21. června 2017 16:28:38 CEST Sebastian Gräßl wrote:
> The biggest part of this all is actually ensuring that all requests made by
> the application are actually going through the HTTP proxy.
> 
> As a solution to that, I was thinking of starting a plugin that configures
> the proxy for HTTP libraries (Net::HTTP, Excon, RestClient, etc.) used.
> At first it would only make sure that we have all requests covered.
> As a nicety it could also have a debug mode to show outgoing requests via
> something like httplog[1].
> 
> The HTTP proxy would at first just be one global setting, but later on by
> extending the used libraries' underlying request methods it could
> also allow for dynamically choosing the appropriate proxy per request.
> This then can be used by the foreman_http_proxies[2] plugin.
> 
> [1] https://github.com/trusche/httplog
> [2] https://github.com/jlsherrill/foreman_http_proxies
> 
> On Thursday, May 25, 2017 at 11:58:12 AM UTC+2, Marek Hulán wrote:
> > sorry for typo, it should have been:
> > 
> > AFAIK we don't *have* the solution *implemented* atm.
> > 
> > On čtvrtek 25. května 2017 11:49:24 CEST Marek Hulán wrote:
> > > Sorry for being late to the party, sending my 2c:
> > > 
> > > I agree with having more complicated solution where users could have
> > > separate proxies per service is good long-term goal. AFAIK we don't the
> > > solution atm. Therefore I think introducing support for single, global
> > > proxy sounds as improvement already to what we have now (nothing).
> > 
> > What's
> > 
> > > good on this, migrating to specific proxies should be easy, the RFC
> > > explicitly[1] mentions it. The global proxy can have granular rules of
> > 
> > what
> > 
> > > communication should be passed through untouched and what should be sent
> > > through maybe other proxies. Another advantage I see is that the global
> > > proxy offloads the configuration from Foreman/Katello, which does not
> > > really belongs into our domain.
> > > 
> > > Later, when the RFC is implemented via foreman_http_proxies plugin, I'm
> > > happy to stop using global proxy and improve plugins to use
> > > foreman_http_proxies if it makes sense. It will take some time before
> > > everyone adopts it. But meanwhile we'd still have the option to let user
> > > configure their master proxy according to their needs.
> > > 
> > > [1] https://github.com/theforeman/rfcs/pull/18/
> > > files#diff-12584a6580dac145ae55c2b5d67088dfR45
> > > 
> > > > On 05/17/2017 07:57 AM, Tom McKay wrote:
> > > > > After reading the RFC I think that more robust and adaptable
> > 
> > solution
> > 
> > > > > would be better. A single env var is not going to cover the needs of
> > > > > all the scenarios. A simple example may be accessing both
> > > > > registry.access.redhat.com 
> > > > > (through proxy) and myopenshift:5000 (no proxy).
> > > > > 
> > > > > As @jlsherrill noted on the PR, the temporary solution for the
> > > > > foreman-docker plugin is alright for the moment.
> > > > 
> > > > I'd like to echo what tom said, we've had many users that want to
> > 
> > access
> > 
> > > > content externally through a proxy and internally (where the proxy is
> > > > not controlled by them and does not properly proxy internal requests).
> > > > Its happened enough for me to say that a simple solution is not good
> > > > enough long term.
> > > > 
> > > > > On Wed, May 17, 2017 at 3:08 AM, Sebastian Gräßl
> > > > > 
> > > > >   > 
> > >> wrote:
> > > > > There was some feedback regarding this on the PR[1] mentioned in
> > > > > the beginning.
> > > > > There is already a RFC[2] regarding this and a plugin[3] to
> > > > > implement the solution proposed in the RFC.
> > > > > 
> > > > > The solution proposed by jlsherrill allows to add multiple
> > > > > HTTP-proxies in Foreman and use these in plugins and allow to
> > > > > configure what HTTP-proxy should be used for what requests.
> > > > > So far the plugin only adds the ability to add HTTP proxies and
> > > > > misses a essential part, which is applying the HTTP proxies to
> > > > > requests.
> > > > > 
> > > > > While looking at how other applications handle this and also
> > > > > considering typical HTTP proxy configurations, it feels that
> > 
> > such
> > 
> > > > > a solution would make it rather complex in practice to apply.

Re: [foreman-dev] Re: [RFC] HTTP proxy for requests

2017-06-21 Thread Sebastian Gräßl
The biggest part of this all is actually ensuring that all requests made by 
the application are actually going through the HTTP proxy.

As a solution to that, I was thinking of starting a plugin that configures 
the proxy for HTTP libraries (Net::HTTP, Excon, RestClient, etc.) used.
At first it would only make sure that we have all requests covered. 
As a nicety it could also have a debug mode to show outgoing requests via 
something like httplog[1].

The HTTP proxy would at first just be one global setting, but later on by 
extending the used libraries' underlying request methods it could 
also allow for dynamically choosing the appropriate proxy per request.
This then can be used by the foreman_http_proxies[2] plugin.

[1] https://github.com/trusche/httplog
[2] https://github.com/jlsherrill/foreman_http_proxies

On Thursday, May 25, 2017 at 11:58:12 AM UTC+2, Marek Hulán wrote:
>
> sorry for typo, it should have been: 
>
> AFAIK we don't *have* the solution *implemented* atm. 
>
> -- 
> Marek 
>
> On čtvrtek 25. května 2017 11:49:24 CEST Marek Hulán wrote: 
> > Sorry for being late to the party, sending my 2c: 
> > 
> > I agree with having more complicated solution where users could have 
> > separate proxies per service is good long-term goal. AFAIK we don't the 
> > solution atm. Therefore I think introducing support for single, global 
> > proxy sounds as improvement already to what we have now (nothing). 
> What's 
> > good on this, migrating to specific proxies should be easy, the RFC 
> > explicitly[1] mentions it. The global proxy can have granular rules of 
> what 
> > communication should be passed through untouched and what should be sent 
> > through maybe other proxies. Another advantage I see is that the global 
> > proxy offloads the configuration from Foreman/Katello, which does not 
> > really belongs into our domain. 
> > 
> > Later, when the RFC is implemented via foreman_http_proxies plugin, I'm 
> > happy to stop using global proxy and improve plugins to use 
> > foreman_http_proxies if it makes sense. It will take some time before 
> > everyone adopts it. But meanwhile we'd still have the option to let user 
> > configure their master proxy according to their needs. 
> > 
> > [1] https://github.com/theforeman/rfcs/pull/18/ 
> > files#diff-12584a6580dac145ae55c2b5d67088dfR45 
> > 
> > -- 
> > Marek 
> > 
> > On středa 17. května 2017 14:22:28 CEST Justin Sherrill wrote: 
> > > On 05/17/2017 07:57 AM, Tom McKay wrote: 
> > > > After reading the RFC I think that more robust and adaptable 
> solution 
> > > > would be better. A single env var is not going to cover the needs of 
> > > > all the scenarios. A simple example may be accessing both 
> > > > registry.access.redhat.com  
> > > > (through proxy) and myopenshift:5000 (no proxy). 
> > > > 
> > > > As @jlsherrill noted on the PR, the temporary solution for the 
> > > > foreman-docker plugin is alright for the moment. 
> > > 
> > > I'd like to echo what tom said, we've had many users that want to 
> access 
> > > content externally through a proxy and internally (where the proxy is 
> > > not controlled by them and does not properly proxy internal requests). 
> > > Its happened enough for me to say that a simple solution is not good 
> > > enough long term. 
> > > 
> > > > On Wed, May 17, 2017 at 3:08 AM, Sebastian Gräßl 
> > > > 
> > > >   >> wrote: 
> > > > There was some feedback regarding this on the PR[1] mentioned in 
> > > > the beginning. 
> > > > There is already a RFC[2] regarding this and a plugin[3] to 
> > > > implement the solution proposed in the RFC. 
> > > > 
> > > > The solution proposed by jlsherrill allows to add multiple 
> > > > HTTP-proxies in Foreman and use these in plugins and allow to 
> > > > configure what HTTP-proxy should be used for what requests. 
> > > > So far the plugin only adds the ability to add HTTP proxies and 
> > > > misses a essential part, which is applying the HTTP proxies to 
> > > > requests. 
> > > > 
> > > > While looking at how other applications handle this and also 
> > > > considering typical HTTP proxy configurations, it feels that 
> such 
> > > > a solution would make it rather complex in practice to apply. 
> > > > Configuring rules for requests or just ensuring the proper 
> request 
> > > > is using the right HTTP proxy is better configurable in the HTTP 
> > > > proxy itself. 
> > > > 
> > > > I believe a very bold simple solution would be the better, which 
> > > > in my opinion would be to ensure all parts respect a HTTP proxy 
> > > > configuration and have good documentation around it to advice on 
> > > > how to configure the HTTP proxy correctly. Taken other 
> > > > applications in the same area the HTTP_PROXY environment 
> variable 
> > > > seems to be the common standard to use. 
> > > > 
> > > >

Re: [foreman-dev] Re: [RFC] HTTP proxy for requests

2017-05-25 Thread Marek Hulán
sorry for typo, it should have been:

AFAIK we don't *have* the solution *implemented* atm.

--
Marek

On čtvrtek 25. května 2017 11:49:24 CEST Marek Hulán wrote:
> Sorry for being late to the party, sending my 2c:
> 
> I agree with having more complicated solution where users could have
> separate proxies per service is good long-term goal. AFAIK we don't the
> solution atm. Therefore I think introducing support for single, global
> proxy sounds as improvement already to what we have now (nothing). What's
> good on this, migrating to specific proxies should be easy, the RFC
> explicitly[1] mentions it. The global proxy can have granular rules of what
> communication should be passed through untouched and what should be sent
> through maybe other proxies. Another advantage I see is that the global
> proxy offloads the configuration from Foreman/Katello, which does not
> really belongs into our domain.
> 
> Later, when the RFC is implemented via foreman_http_proxies plugin, I'm
> happy to stop using global proxy and improve plugins to use
> foreman_http_proxies if it makes sense. It will take some time before
> everyone adopts it. But meanwhile we'd still have the option to let user
> configure their master proxy according to their needs.
> 
> [1] https://github.com/theforeman/rfcs/pull/18/
> files#diff-12584a6580dac145ae55c2b5d67088dfR45
> 
> --
> Marek
> 
> On středa 17. května 2017 14:22:28 CEST Justin Sherrill wrote:
> > On 05/17/2017 07:57 AM, Tom McKay wrote:
> > > After reading the RFC I think that more robust and adaptable solution
> > > would be better. A single env var is not going to cover the needs of
> > > all the scenarios. A simple example may be accessing both
> > > registry.access.redhat.com 
> > > (through proxy) and myopenshift:5000 (no proxy).
> > > 
> > > As @jlsherrill noted on the PR, the temporary solution for the
> > > foreman-docker plugin is alright for the moment.
> > 
> > I'd like to echo what tom said, we've had many users that want to access
> > content externally through a proxy and internally (where the proxy is
> > not controlled by them and does not properly proxy internal requests).
> > Its happened enough for me to say that a simple solution is not good
> > enough long term.
> > 
> > > On Wed, May 17, 2017 at 3:08 AM, Sebastian Gräßl
> > > 
> > > mailto:sebast...@validcode.me>> wrote:
> > > There was some feedback regarding this on the PR[1] mentioned in
> > > the beginning.
> > > There is already a RFC[2] regarding this and a plugin[3] to
> > > implement the solution proposed in the RFC.
> > > 
> > > The solution proposed by jlsherrill allows to add multiple
> > > HTTP-proxies in Foreman and use these in plugins and allow to
> > > configure what HTTP-proxy should be used for what requests.
> > > So far the plugin only adds the ability to add HTTP proxies and
> > > misses a essential part, which is applying the HTTP proxies to
> > > requests.
> > > 
> > > While looking at how other applications handle this and also
> > > considering typical HTTP proxy configurations, it feels that such
> > > a solution would make it rather complex in practice to apply.
> > > Configuring rules for requests or just ensuring the proper request
> > > is using the right HTTP proxy is better configurable in the HTTP
> > > proxy itself.
> > > 
> > > I believe a very bold simple solution would be the better, which
> > > in my opinion would be to ensure all parts respect a HTTP proxy
> > > configuration and have good documentation around it to advice on
> > > how to configure the HTTP proxy correctly. Taken other
> > > applications in the same area the HTTP_PROXY environment variable
> > > seems to be the common standard to use.
> > > 
> > > Please, I would love to hear more feedback on this and what are
> > > common HTTP proxy setups.
> > > 
> > > [1] https://github.com/theforeman/foreman-docker/pull/189
> > > 
> > > [2] https://github.com/theforeman/rfcs/pull/18
> > > 
> > > [3] https://github.com/jlsherrill/foreman_http_proxies
> > > 
> > > 
> > > On Thursday, April 20, 2017 at 1:07:33 PM UTC+2, Sebastian Gräßl
> > > 
> > > wrote:
> > > Hej,
> > > 
> > > at the moment there is a PR[1] open on foreman-docker to set a
> > > HTTP proxy for requests to registries.
> > > The PR allows to set a HTTP proxy on the HTTP client, in this
> > > case deep down Excon, only for registry requests.
> > > 
> > > A HTTP proxy won't be set on requests if a `HTTP_PROXY`
> > > environment variable is available, since it is an unlikely
> > > setup to have registry request routed over a different proxy

Re: [foreman-dev] Re: [RFC] HTTP proxy for requests

2017-05-25 Thread Marek Hulán
Sorry for being late to the party, sending my 2c:

I agree with having more complicated solution where users could have separate 
proxies per service is good long-term goal. AFAIK we don't the solution atm. 
Therefore I think introducing support for single, global proxy sounds as 
improvement already to what we have now (nothing). What's good on this, 
migrating to specific proxies should be easy, the RFC explicitly[1] mentions 
it. The global proxy can have granular rules of what communication should be 
passed through untouched and what should be sent through maybe other proxies. 
Another advantage I see is that the global proxy offloads the configuration 
from Foreman/Katello, which does not really belongs into our domain.

Later, when the RFC is implemented via foreman_http_proxies plugin, I'm happy 
to stop using global proxy and improve plugins to use foreman_http_proxies if 
it makes sense. It will take some time before everyone adopts it. But 
meanwhile we'd still have the option to let user configure their master proxy 
according to their needs.

[1] https://github.com/theforeman/rfcs/pull/18/
files#diff-12584a6580dac145ae55c2b5d67088dfR45

--
Marek

On středa 17. května 2017 14:22:28 CEST Justin Sherrill wrote:
> On 05/17/2017 07:57 AM, Tom McKay wrote:
> > After reading the RFC I think that more robust and adaptable solution
> > would be better. A single env var is not going to cover the needs of
> > all the scenarios. A simple example may be accessing both
> > registry.access.redhat.com 
> > (through proxy) and myopenshift:5000 (no proxy).
> > 
> > As @jlsherrill noted on the PR, the temporary solution for the
> > foreman-docker plugin is alright for the moment.
> 
> I'd like to echo what tom said, we've had many users that want to access
> content externally through a proxy and internally (where the proxy is
> not controlled by them and does not properly proxy internal requests).
> Its happened enough for me to say that a simple solution is not good
> enough long term.
> 
> > On Wed, May 17, 2017 at 3:08 AM, Sebastian Gräßl
> > 
> > mailto:sebast...@validcode.me>> wrote:
> > There was some feedback regarding this on the PR[1] mentioned in
> > the beginning.
> > There is already a RFC[2] regarding this and a plugin[3] to
> > implement the solution proposed in the RFC.
> > 
> > The solution proposed by jlsherrill allows to add multiple
> > HTTP-proxies in Foreman and use these in plugins and allow to
> > configure what HTTP-proxy should be used for what requests.
> > So far the plugin only adds the ability to add HTTP proxies and
> > misses a essential part, which is applying the HTTP proxies to
> > requests.
> > 
> > While looking at how other applications handle this and also
> > considering typical HTTP proxy configurations, it feels that such
> > a solution would make it rather complex in practice to apply.
> > Configuring rules for requests or just ensuring the proper request
> > is using the right HTTP proxy is better configurable in the HTTP
> > proxy itself.
> > 
> > I believe a very bold simple solution would be the better, which
> > in my opinion would be to ensure all parts respect a HTTP proxy
> > configuration and have good documentation around it to advice on
> > how to configure the HTTP proxy correctly. Taken other
> > applications in the same area the HTTP_PROXY environment variable
> > seems to be the common standard to use.
> > 
> > Please, I would love to hear more feedback on this and what are
> > common HTTP proxy setups.
> > 
> > [1] https://github.com/theforeman/foreman-docker/pull/189
> > 
> > [2] https://github.com/theforeman/rfcs/pull/18
> > 
> > [3] https://github.com/jlsherrill/foreman_http_proxies
> > 
> > 
> > On Thursday, April 20, 2017 at 1:07:33 PM UTC+2, Sebastian Gräßl
> > 
> > wrote:
> > Hej,
> > 
> > at the moment there is a PR[1] open on foreman-docker to set a
> > HTTP proxy for requests to registries.
> > The PR allows to set a HTTP proxy on the HTTP client, in this
> > case deep down Excon, only for registry requests.
> > 
> > A HTTP proxy won't be set on requests if a `HTTP_PROXY`
> > environment variable is available, since it is an unlikely
> > setup to have registry request routed over a different proxy
> > than other requests. However setting it via the environment
> > variable will allow requests to succeed to resources available
> > by the HTTP proxy, but will fail for those inside and possible
> > blocked.
> > 
> > The `HTTP_PROXY` environment variable seems to be a standard,
> >  

Re: [foreman-dev] Re: [RFC] HTTP proxy for requests

2017-05-17 Thread Justin Sherrill


On 05/17/2017 07:57 AM, Tom McKay wrote:
After reading the RFC I think that more robust and adaptable solution 
would be better. A single env var is not going to cover the needs of 
all the scenarios. A simple example may be accessing both 
registry.access.redhat.com  
(through proxy) and myopenshift:5000 (no proxy).


As @jlsherrill noted on the PR, the temporary solution for the 
foreman-docker plugin is alright for the moment.
I'd like to echo what tom said, we've had many users that want to access 
content externally through a proxy and internally (where the proxy is 
not controlled by them and does not properly proxy internal requests).  
Its happened enough for me to say that a simple solution is not good 
enough long term.




On Wed, May 17, 2017 at 3:08 AM, Sebastian Gräßl 
mailto:sebast...@validcode.me>> wrote:


There was some feedback regarding this on the PR[1] mentioned in
the beginning.
There is already a RFC[2] regarding this and a plugin[3] to
implement the solution proposed in the RFC.

The solution proposed by jlsherrill allows to add multiple
HTTP-proxies in Foreman and use these in plugins and allow to
configure what HTTP-proxy should be used for what requests.
So far the plugin only adds the ability to add HTTP proxies and
misses a essential part, which is applying the HTTP proxies to
requests.

While looking at how other applications handle this and also
considering typical HTTP proxy configurations, it feels that such
a solution would make it rather complex in practice to apply.
Configuring rules for requests or just ensuring the proper request
is using the right HTTP proxy is better configurable in the HTTP
proxy itself.

I believe a very bold simple solution would be the better, which
in my opinion would be to ensure all parts respect a HTTP proxy
configuration and have good documentation around it to advice on
how to configure the HTTP proxy correctly. Taken other
applications in the same area the HTTP_PROXY environment variable
seems to be the common standard to use.

Please, I would love to hear more feedback on this and what are
common HTTP proxy setups.

[1] https://github.com/theforeman/foreman-docker/pull/189

[2] https://github.com/theforeman/rfcs/pull/18

[3] https://github.com/jlsherrill/foreman_http_proxies


On Thursday, April 20, 2017 at 1:07:33 PM UTC+2, Sebastian Gräßl
wrote:

Hej,

at the moment there is a PR[1] open on foreman-docker to set a
HTTP proxy for requests to registries.
The PR allows to set a HTTP proxy on the HTTP client, in this
case deep down Excon, only for registry requests.

A HTTP proxy won't be set on requests if a `HTTP_PROXY`
environment variable is available, since it is an unlikely
setup to have registry request routed over a different proxy
than other requests. However setting it via the environment
variable will allow requests to succeed to resources available
by the HTTP proxy, but will fail for those inside and possible
blocked.

The `HTTP_PROXY` environment variable seems to be a standard,
and therefore Excon is built to use it when available.
Excon is used by docker-api as well as fog, it might be used
by other components and there might be other parts that use
another HTTP client like RestClient, which also respects the
variable.

This means at the moment with that environment variable set
some requests would already rely on it.
In any case this should be in mentioned in the manual to be
aware of, also because some operating systems set this globally.

The question is should we make an afford to ensure deployment
behind a HTTP proxy on a system with HTTP blocked works
without issues and provide a way to configure it properly?

I've tested Foreman with HTTP blocked and `HTTP_PROXY` set,
but in a very basic setup, with the only external requests
being to Docker registries outside and squid configured to
just pass requests through regardless there to.

It didn't show any apparent issue, but there are for sure
issues with a more robust configured HTTP proxy.
This raises another question: How common is a setup where
external resources requiring HTTP are used with Foreman behind
a HTTP proxy?

Comments?

All the best,
Sebastian

[1] https://github.com/theforeman/foreman-docker/pull/189


-- 
You received this message because you are subscribed to the Google


Re: [foreman-dev] Re: [RFC] HTTP proxy for requests

2017-05-17 Thread Tom McKay
After reading the RFC I think that more robust and adaptable solution would
be better. A single env var is not going to cover the needs of all the
scenarios. A simple example may be accessing both registry.access.redhat.com
(through proxy) and myopenshift:5000 (no proxy).

As @jlsherrill noted on the PR, the temporary solution for the
foreman-docker plugin is alright for the moment.

On Wed, May 17, 2017 at 3:08 AM, Sebastian Gräßl 
wrote:

> There was some feedback regarding this on the PR[1] mentioned in the
> beginning.
> There is already a RFC[2] regarding this and a plugin[3] to implement the
> solution proposed in the RFC.
>
> The solution proposed by jlsherrill allows to add multiple HTTP-proxies in
> Foreman and use these in plugins and allow to configure what HTTP-proxy
> should be used for what requests.
> So far the plugin only adds the ability to add HTTP proxies and misses a
> essential part, which is applying the HTTP proxies to requests.
>
> While looking at how other applications handle this and also considering
> typical HTTP proxy configurations, it feels that such a solution would make
> it rather complex in practice to apply. Configuring rules for requests or
> just ensuring the proper request is using the right HTTP proxy is better
> configurable in the HTTP proxy itself.
>
> I believe a very bold simple solution would be the better, which in my
> opinion would be to ensure all parts respect a HTTP proxy configuration and
> have good documentation around it to advice on how to configure the HTTP
> proxy correctly. Taken other applications in the same area the HTTP_PROXY
> environment variable seems to be the common standard to use.
>
> Please, I would love to hear more feedback on this and what are common
> HTTP proxy setups.
>
> [1] https://github.com/theforeman/foreman-docker/pull/189
> [2] https://github.com/theforeman/rfcs/pull/18
> [3] https://github.com/jlsherrill/foreman_http_proxies
>
> On Thursday, April 20, 2017 at 1:07:33 PM UTC+2, Sebastian Gräßl wrote:
>>
>> Hej,
>>
>> at the moment there is a PR[1] open on foreman-docker to set a HTTP proxy
>> for requests to registries.
>> The PR allows to set a HTTP proxy on the HTTP client, in this case deep
>> down Excon, only for registry requests.
>>
>> A HTTP proxy won't be set on requests if a `HTTP_PROXY` environment
>> variable is available, since it is an unlikely setup to have registry
>> request routed over a different proxy than other requests. However setting
>> it via the environment variable will allow requests to succeed to resources
>> available by the HTTP proxy, but will fail for those inside and possible
>> blocked.
>>
>> The `HTTP_PROXY` environment variable seems to be a standard, and
>> therefore Excon is built to use it when available.
>> Excon is used by docker-api as well as fog, it might be used by other
>> components and there might be other parts that use another HTTP client like
>> RestClient, which also respects the variable.
>>
>> This means at the moment with that environment variable set some requests
>> would already rely on it.
>> In any case this should be in mentioned in the manual to be aware of,
>> also because some operating systems set this globally.
>>
>> The question is should we make an afford to ensure deployment behind a
>> HTTP proxy on a system with HTTP blocked works without issues and provide a
>> way to configure it properly?
>>
>> I've tested Foreman with HTTP blocked and `HTTP_PROXY` set, but in a very
>> basic setup, with the only external requests being to Docker registries
>> outside and squid configured to just pass requests through regardless there
>> to.
>>
>> It didn't show any apparent issue, but there are for sure issues with a
>> more robust configured HTTP proxy.
>> This raises another question: How common is a setup where external
>> resources requiring HTTP are used with Foreman behind a HTTP proxy?
>>
>> Comments?
>>
>> All the best,
>> Sebastian
>>
>> [1] https://github.com/theforeman/foreman-docker/pull/189
>>
>> --
> You received this message because you are subscribed to the Google Groups
> "foreman-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to foreman-dev+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


[foreman-dev] Re: [RFC] HTTP proxy for requests

2017-05-17 Thread Sebastian Gräßl
 There was some feedback regarding this on the PR[1] mentioned in the 
beginning. 
There is already a RFC[2] regarding this and a plugin[3] to implement the 
solution proposed in the RFC. 

The solution proposed by jlsherrill allows to add multiple HTTP-proxies in 
Foreman and use these in plugins and allow to configure what HTTP-proxy 
should be used for what requests. 
So far the plugin only adds the ability to add HTTP proxies and misses a 
essential part, which is applying the HTTP proxies to requests. 

While looking at how other applications handle this and also considering 
typical HTTP proxy configurations, it feels that such a solution would make 
it rather complex in practice to apply. Configuring rules for requests or 
just ensuring the proper request is using the right HTTP proxy is better 
configurable in the HTTP proxy itself. 

I believe a very bold simple solution would be the better, which in my 
opinion would be to ensure all parts respect a HTTP proxy configuration and 
have good documentation around it to advice on how to configure the HTTP 
proxy correctly. Taken other applications in the same area the HTTP_PROXY 
environment variable seems to be the common standard to use. 

Please, I would love to hear more feedback on this and what are common HTTP 
proxy setups. 

[1] https://github.com/theforeman/foreman-docker/pull/189 
[2] https://github.com/theforeman/rfcs/pull/18 
[3] https://github.com/jlsherrill/foreman_http_proxies

On Thursday, April 20, 2017 at 1:07:33 PM UTC+2, Sebastian Gräßl wrote:
>
> Hej,
>
> at the moment there is a PR[1] open on foreman-docker to set a HTTP proxy 
> for requests to registries.
> The PR allows to set a HTTP proxy on the HTTP client, in this case deep 
> down Excon, only for registry requests.
>
> A HTTP proxy won't be set on requests if a `HTTP_PROXY` environment 
> variable is available, since it is an unlikely setup to have registry 
> request routed over a different proxy than other requests. However setting 
> it via the environment variable will allow requests to succeed to resources 
> available by the HTTP proxy, but will fail for those inside and possible 
> blocked.
>
> The `HTTP_PROXY` environment variable seems to be a standard, and 
> therefore Excon is built to use it when available. 
> Excon is used by docker-api as well as fog, it might be used by other 
> components and there might be other parts that use another HTTP client like 
> RestClient, which also respects the variable.
>
> This means at the moment with that environment variable set some requests 
> would already rely on it.
> In any case this should be in mentioned in the manual to be aware of, also 
> because some operating systems set this globally.
>
> The question is should we make an afford to ensure deployment behind a 
> HTTP proxy on a system with HTTP blocked works without issues and provide a 
> way to configure it properly?
>
> I've tested Foreman with HTTP blocked and `HTTP_PROXY` set, but in a very 
> basic setup, with the only external requests being to Docker registries 
> outside and squid configured to just pass requests through regardless there 
> to.
>
> It didn't show any apparent issue, but there are for sure issues with a 
> more robust configured HTTP proxy. 
> This raises another question: How common is a setup where external 
> resources requiring HTTP are used with Foreman behind a HTTP proxy?
>
> Comments?
>
> All the best,
> Sebastian
>
> [1] https://github.com/theforeman/foreman-docker/pull/189
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.