The biggest part of this all is actually ensuring that all requests made by the application are actually going through the HTTP proxy.
As a solution to that, I was thinking of starting a plugin that configures the proxy for HTTP libraries (Net::HTTP, Excon, RestClient, etc.) used. At first it would only make sure that we have all requests covered. As a nicety it could also have a debug mode to show outgoing requests via something like httplog[1]. The HTTP proxy would at first just be one global setting, but later on by extending the used libraries' underlying request methods it could also allow for dynamically choosing the appropriate proxy per request. This then can be used by the foreman_http_proxies[2] plugin. [1] https://github.com/trusche/httplog [2] https://github.com/jlsherrill/foreman_http_proxies On Thursday, May 25, 2017 at 11:58:12 AM UTC+2, Marek Hulán wrote: > > sorry for typo, it should have been: > > AFAIK we don't *have* the solution *implemented* atm. > > -- > Marek > > On čtvrtek 25. května 2017 11:49:24 CEST Marek Hulán wrote: > > Sorry for being late to the party, sending my 2c: > > > > I agree with having more complicated solution where users could have > > separate proxies per service is good long-term goal. AFAIK we don't the > > solution atm. Therefore I think introducing support for single, global > > proxy sounds as improvement already to what we have now (nothing). > What's > > good on this, migrating to specific proxies should be easy, the RFC > > explicitly[1] mentions it. The global proxy can have granular rules of > what > > communication should be passed through untouched and what should be sent > > through maybe other proxies. Another advantage I see is that the global > > proxy offloads the configuration from Foreman/Katello, which does not > > really belongs into our domain. > > > > Later, when the RFC is implemented via foreman_http_proxies plugin, I'm > > happy to stop using global proxy and improve plugins to use > > foreman_http_proxies if it makes sense. It will take some time before > > everyone adopts it. But meanwhile we'd still have the option to let user > > configure their master proxy according to their needs. > > > > [1] https://github.com/theforeman/rfcs/pull/18/ > > files#diff-12584a6580dac145ae55c2b5d67088dfR45 > > > > -- > > Marek > > > > On středa 17. května 2017 14:22:28 CEST Justin Sherrill wrote: > > > On 05/17/2017 07:57 AM, Tom McKay wrote: > > > > After reading the RFC I think that more robust and adaptable > solution > > > > would be better. A single env var is not going to cover the needs of > > > > all the scenarios. A simple example may be accessing both > > > > registry.access.redhat.com <http://registry.access.redhat.com> > > > > (through proxy) and myopenshift:5000 (no proxy). > > > > > > > > As @jlsherrill noted on the PR, the temporary solution for the > > > > foreman-docker plugin is alright for the moment. > > > > > > I'd like to echo what tom said, we've had many users that want to > access > > > content externally through a proxy and internally (where the proxy is > > > not controlled by them and does not properly proxy internal requests). > > > Its happened enough for me to say that a simple solution is not good > > > enough long term. > > > > > > > On Wed, May 17, 2017 at 3:08 AM, Sebastian Gräßl > > > > > > > > <seba...@validcode.me <javascript:> <mailto:seba...@validcode.me > <javascript:>>> wrote: > > > > There was some feedback regarding this on the PR[1] mentioned in > > > > the beginning. > > > > There is already a RFC[2] regarding this and a plugin[3] to > > > > implement the solution proposed in the RFC. > > > > > > > > The solution proposed by jlsherrill allows to add multiple > > > > HTTP-proxies in Foreman and use these in plugins and allow to > > > > configure what HTTP-proxy should be used for what requests. > > > > So far the plugin only adds the ability to add HTTP proxies and > > > > misses a essential part, which is applying the HTTP proxies to > > > > requests. > > > > > > > > While looking at how other applications handle this and also > > > > considering typical HTTP proxy configurations, it feels that > such > > > > a solution would make it rather complex in practice to apply. > > > > Configuring rules for requests or just ensuring the proper > request > > > > is using the right HTTP proxy is better configurable in the HTTP > > > > proxy itself. > > > > > > > > I believe a very bold simple solution would be the better, which > > > > in my opinion would be to ensure all parts respect a HTTP proxy > > > > configuration and have good documentation around it to advice on > > > > how to configure the HTTP proxy correctly. Taken other > > > > applications in the same area the HTTP_PROXY environment > variable > > > > seems to be the common standard to use. > > > > > > > > Please, I would love to hear more feedback on this and what are > > > > common HTTP proxy setups. > > > > > > > > [1] https://github.com/theforeman/foreman-docker/pull/189 > > > > <https://github.com/theforeman/foreman-docker/pull/189> > > > > [2] https://github.com/theforeman/rfcs/pull/18 > > > > <https://github.com/theforeman/rfcs/pull/18> > > > > [3] https://github.com/jlsherrill/foreman_http_proxies > > > > <https://github.com/jlsherrill/foreman_http_proxies> > > > > > > > > On Thursday, April 20, 2017 at 1:07:33 PM UTC+2, Sebastian Gräßl > > > > > > > > wrote: > > > > Hej, > > > > > > > > at the moment there is a PR[1] open on foreman-docker to set > a > > > > HTTP proxy for requests to registries. > > > > The PR allows to set a HTTP proxy on the HTTP client, in > this > > > > case deep down Excon, only for registry requests. > > > > > > > > A HTTP proxy won't be set on requests if a `HTTP_PROXY` > > > > environment variable is available, since it is an unlikely > > > > setup to have registry request routed over a different proxy > > > > than other requests. However setting it via the environment > > > > variable will allow requests to succeed to resources > available > > > > by the HTTP proxy, but will fail for those inside and > possible > > > > blocked. > > > > > > > > The `HTTP_PROXY` environment variable seems to be a > standard, > > > > and therefore Excon is built to use it when available. > > > > Excon is used by docker-api as well as fog, it might be used > > > > by other components and there might be other parts that use > > > > another HTTP client like RestClient, which also respects the > > > > variable. > > > > > > > > This means at the moment with that environment variable set > > > > some requests would already rely on it. > > > > In any case this should be in mentioned in the manual to be > > > > aware of, also because some operating systems set this > globally. > > > > > > > > The question is should we make an afford to ensure > deployment > > > > behind a HTTP proxy on a system with HTTP blocked works > > > > without issues and provide a way to configure it properly? > > > > > > > > I've tested Foreman with HTTP blocked and `HTTP_PROXY` set, > > > > but in a very basic setup, with the only external requests > > > > being to Docker registries outside and squid configured to > > > > just pass requests through regardless there to. > > > > > > > > It didn't show any apparent issue, but there are for sure > > > > issues with a more robust configured HTTP proxy. > > > > This raises another question: How common is a setup where > > > > external resources requiring HTTP are used with Foreman > behind > > > > a HTTP proxy? > > > > > > > > Comments? > > > > > > > > All the best, > > > > Sebastian > > > > > > > > [1] https://github.com/theforeman/foreman-docker/pull/189 > > > > <https://github.com/theforeman/foreman-docker/pull/189> > > > -- You received this message because you are subscribed to the Google Groups "foreman-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to foreman-dev+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
