[go-nuts] Re: How to implement localhost proxy which injects Proxy-Authorization header to incoming request and sends it to another remote proxy with Go?

[Hugo Bollon]

I fixed the 400 Server Error issue and add a request checking before inject 
*Proxy-Authorization 
*header in order to alter only GET or CONNECT requests: 
https://github.com/hbollon/IGopher/blob/proxy/internal/proxy/pipe.go
But now I'm getting this error after curl request: *OpenSSL SSL_read: 
error:1409442E:SSL routines:ssl3_read_bytes:tlsv1 alert protocol version, 
errno 0*

Maybe my approach is completely crazy or wrong but I feel can succeed lmao

Le mercredi 17 mars 2021 à 16:00:22 UTC+1, Hugo Bollon a écrit :

> Yeah, I'm not very comfortable with networking so it's possible I'm not 
> using the right terms...
>
> I want to be able to allow my users to use, with Selenium (and therefore 
> Chrome), a proxy with authentication. The problem is that I cannot send the 
> proxy credentials to the Chrome instance or interact with the proxy 
> connection alert box.
>
> So basically, what I want to do, is a localhost proxy managed by my 
> program which will receive all Chrome CONNECT requests, inject the 
> Proxy-Authorization header to authenticate requests and send them back to 
> the user remote proxy.
>
> I managed to get something with a TCP tunnel, I relied on the source code 
> of io.Copy source code.
> I managed to intercept the request from the incoming net.Conn, decode the 
> http request from the tcp packet, add the Proxy-Authorization at the end, 
> re-encode the packet and finally write it to the outgoing net.Conn (to the 
> remote proxy )
> But I'm getting a 400 Server Error on the remote proxy side...
>
> You can check all of that here: 
> https://github.com/hbollon/IGopher/tree/proxy/internal/proxy
> (It's WIP stuff, so it's not cleaned or optimized yet. I just pushed it so 
> you could check it out)
>
> Hope I am clear enough ...
> Anyways, thanks for your help! :)
>
> Le mercredi 17 mars 2021 à 09:30:22 UTC+1, vlad...@varank.in a écrit :
>
>> I think I didn't get what you're building right. Now, it looks like, 
>> instead of implementing a custom RR's director, you need to configure its 
>> Transport [1], which will be aware of your auth proxy in the middle. Have a 
>> look at net/http.Transport.Proxy field [2] for that.
>>
>> [1]: https://pkg.go.dev/net/http/httputil?utm_source=godoc#ReverseProxy
>> [2]: https://pkg.go.dev/net/http?utm_source=godoc#Transport
>>
>> On Tuesday, March 16, 2021 at 11:17:49 AM UTC+1 hugo@gmail.com wrote:
>>
>>> Thank you for your advice that I applied.
>>> But now I have a *407 Proxy Authentication Required* error, while the 
>>> header is added to the request...
>>> Here is the output:
>>>
>>> INFO[0019] Pre-Edited request: &{Method:CONNECT URL://google.com:443 
>>> Proto:HTTP/1.1 ProtoMajor:1 ProtoMinor:1 
>>> Header:map[Proxy-Connection:[Keep-Alive] User-Agent:[curl/7.68.0]] 
>>> Body: GetBody: ContentLength:0 TransferEncoding:[] Close:false 
>>> Host:google.com:443 Form:map[] PostForm:map[] MultipartForm: 
>>> Trailer:map[] RemoteAddr:127.0.0.1:45382 RequestURI:google.com:443 
>>> TLS: Cancel: Response: ctx:0xc000814240}  function=func1 
>>> line=60
>>>
>>> INFO[0019] Edited Request: &{Method:CONNECT URL:
>>> http://51.178.xx.xx:3128/ Proto:HTTP/1.1 ProtoMajor:1 ProtoMinor:1 
>>> Header:map[Proxy-Authorization:[Basic ] Proxy-Connection:[Keep-Alive] 
>>> User-Agent:[curl/7.68.0]] Body: GetBody: ContentLength:0 
>>> TransferEncoding:[] Close:false Host:google.com:443 Form:map[] 
>>> PostForm:map[] MultipartForm: Trailer:map[] RemoteAddr:
>>> 127.0.0.1:45382 RequestURI:google.com:443 TLS: Cancel: 
>>> Response: ctx:0xc000814240}  function=func1 line=69
>>>
>>> INFO[0019] Scheme: http, Host: 51.178.xx.xx:3128, Port: 3128  
>>> function=func1 line=70
>>>
>>> INFO[0019] Response: &{Status:407 Proxy Authentication Required 
>>> StatusCode:407 Proto:HTTP/1.1 ProtoMajor:1 ProtoMinor:1 
>>> Header:map[Content-Language:[en] Content-Length:[3520] 
>>> Content-Type:[text/html;charset=utf-8] Date:[Tue, 16 Mar 2021 10:03:44 GMT] 
>>> Mime-Version:[1.0] Server:[squid/3.5.27] Vary:[Accept-Language] Via:[1.1 
>>> vps799016 (squid/3.5.27)] X-Cache:[MISS from vps799xxx] 
>>> X-Cache-Lookup:[NONE from vps799xxx:3128] 
>>> X-Squid-Error:[ERR_CACHE_ACCESS_DENIED 0]] Body:0xc0004de180 
>>> ContentLength:3520 TransferEncoding:[] Close:false Uncompressed:false 
>>> Trailer:map[] Request:0xc0005ee100 TLS:}  function=PrintResponse 
>>> line=33
>>>
>>>
>>> PS: Is it possible on Google Groups to format code snippets? Markdown 
>>> not seems to be sup

[go-nuts] Re: How to implement localhost proxy which injects Proxy-Authorization header to incoming request and sends it to another remote proxy with Go?

[Hugo Bollon]

Yeah, I'm not very comfortable with networking so it's possible I'm not 
using the right terms...

I want to be able to allow my users to use, with Selenium (and therefore 
Chrome), a proxy with authentication. The problem is that I cannot send the 
proxy credentials to the Chrome instance or interact with the proxy 
connection alert box.

So basically, what I want to do, is a localhost proxy managed by my program 
which will receive all Chrome CONNECT requests, inject the 
Proxy-Authorization header to authenticate requests and send them back to 
the user remote proxy.

I managed to get something with a TCP tunnel, I relied on the source code 
of io.Copy source code.
I managed to intercept the request from the incoming net.Conn, decode the 
http request from the tcp packet, add the Proxy-Authorization at the end, 
re-encode the packet and finally write it to the outgoing net.Conn (to the 
remote proxy )
But I'm getting a 400 Server Error on the remote proxy side...

You can check all of that here: 
https://github.com/hbollon/IGopher/tree/proxy/internal/proxy
(It's WIP stuff, so it's not cleaned or optimized yet. I just pushed it so 
you could check it out)

Hope I am clear enough ...
Anyways, thanks for your help! :)

Le mercredi 17 mars 2021 à 09:30:22 UTC+1, vlad...@varank.in a écrit :

> I think I didn't get what you're building right. Now, it looks like, 
> instead of implementing a custom RR's director, you need to configure its 
> Transport [1], which will be aware of your auth proxy in the middle. Have a 
> look at net/http.Transport.Proxy field [2] for that.
>
> [1]: https://pkg.go.dev/net/http/httputil?utm_source=godoc#ReverseProxy
> [2]: https://pkg.go.dev/net/http?utm_source=godoc#Transport
>
> On Tuesday, March 16, 2021 at 11:17:49 AM UTC+1 hugo@gmail.com wrote:
>
>> Thank you for your advice that I applied.
>> But now I have a *407 Proxy Authentication Required* error, while the 
>> header is added to the request...
>> Here is the output:
>>
>> INFO[0019] Pre-Edited request: &{Method:CONNECT URL://google.com:443 
>> Proto:HTTP/1.1 ProtoMajor:1 ProtoMinor:1 
>> Header:map[Proxy-Connection:[Keep-Alive] User-Agent:[curl/7.68.0]] 
>> Body: GetBody: ContentLength:0 TransferEncoding:[] Close:false 
>> Host:google.com:443 Form:map[] PostForm:map[] MultipartForm: 
>> Trailer:map[] RemoteAddr:127.0.0.1:45382 RequestURI:google.com:443 
>> TLS: Cancel: Response: ctx:0xc000814240}  function=func1 
>> line=60
>>
>> INFO[0019] Edited Request: &{Method:CONNECT URL:http://51.178.xx.xx:3128/ 
>> Proto:HTTP/1.1 ProtoMajor:1 ProtoMinor:1 
>> Header:map[Proxy-Authorization:[Basic ] Proxy-Connection:[Keep-Alive] 
>> User-Agent:[curl/7.68.0]] Body: GetBody: ContentLength:0 
>> TransferEncoding:[] Close:false Host:google.com:443 Form:map[] 
>> PostForm:map[] MultipartForm: Trailer:map[] RemoteAddr:
>> 127.0.0.1:45382 RequestURI:google.com:443 TLS: Cancel: 
>> Response: ctx:0xc000814240}  function=func1 line=69
>>
>> INFO[0019] Scheme: http, Host: 51.178.xx.xx:3128, Port: 3128  
>> function=func1 line=70
>>
>> INFO[0019] Response: &{Status:407 Proxy Authentication Required 
>> StatusCode:407 Proto:HTTP/1.1 ProtoMajor:1 ProtoMinor:1 
>> Header:map[Content-Language:[en] Content-Length:[3520] 
>> Content-Type:[text/html;charset=utf-8] Date:[Tue, 16 Mar 2021 10:03:44 GMT] 
>> Mime-Version:[1.0] Server:[squid/3.5.27] Vary:[Accept-Language] Via:[1.1 
>> vps799016 (squid/3.5.27)] X-Cache:[MISS from vps799xxx] 
>> X-Cache-Lookup:[NONE from vps799xxx:3128] 
>> X-Squid-Error:[ERR_CACHE_ACCESS_DENIED 0]] Body:0xc0004de180 
>> ContentLength:3520 TransferEncoding:[] Close:false Uncompressed:false 
>> Trailer:map[] Request:0xc0005ee100 TLS:}  function=PrintResponse 
>> line=33
>>
>>
>> PS: Is it possible on Google Groups to format code snippets? Markdown not 
>> seems to be supported
>>
>> Le mardi 16 mars 2021 à 09:04:31 UTC+1, vlad...@varank.in a écrit :
>>
>>> Hey there,
>>>
>>> Seems the issue hides in the chunk, where you overwrite reverse proxy's 
>>> "Director" method, which NewSingleHostReverseProxy creates internally. 
>>> Since your own director doesn't set the client request's Schema and Host, 
>>> you have to either do that manually or make sure you call the original 
>>> director.
>>>
>>> Try doing the following:
>>>
>>> proxyDirector := proxy.Director // ← keep the original director
>>> d := func(req *http.Request) {
>>> logrus.Infof("Pre-Edited request: %+v\n", req)
>>>
>>> proxyDirector(req) // ← call the original director to make sure 
>>> the request will go through the proxy
>>>
>>> // Inject proxy authentication headers to outgoing request into 
>>> new Header
>>> basicAuth := "Basic " + 
>>> base64.StdEncoding.EncodeToString([]byte(remoteServerAuth))
>>> req.Header.Set("Proxy-Authorization", basicAuth)
>>> logrus.Infof("Edited Request: %+v\n", req)
>>> logrus.Infof("Scheme: %s, Host: %s, Port: %s\n", req.URL.Scheme, 
>>> req.URL.Host, 

[go-nuts] Re: How to implement localhost proxy which injects Proxy-Authorization header to incoming request and sends it to another remote proxy with Go?

[Hugo Bollon]

Thank you for your advice that I applied.
But now I have a *407 Proxy Authentication Required* error, while the 
header is added to the request...
Here is the output:

INFO[0019] Pre-Edited request: &{Method:CONNECT URL://google.com:443 
Proto:HTTP/1.1 ProtoMajor:1 ProtoMinor:1 
Header:map[Proxy-Connection:[Keep-Alive] User-Agent:[curl/7.68.0]] 
Body: GetBody: ContentLength:0 TransferEncoding:[] Close:false 
Host:google.com:443 Form:map[] PostForm:map[] MultipartForm: 
Trailer:map[] RemoteAddr:127.0.0.1:45382 RequestURI:google.com:443 
TLS: Cancel: Response: ctx:0xc000814240}  function=func1 
line=60

INFO[0019] Edited Request: &{Method:CONNECT URL:http://51.178.xx.xx:3128/ 
Proto:HTTP/1.1 ProtoMajor:1 ProtoMinor:1 
Header:map[Proxy-Authorization:[Basic ] Proxy-Connection:[Keep-Alive] 
User-Agent:[curl/7.68.0]] Body: GetBody: ContentLength:0 
TransferEncoding:[] Close:false Host:google.com:443 Form:map[] 
PostForm:map[] MultipartForm: Trailer:map[] RemoteAddr:127.0.0.1:45382 
RequestURI:google.com:443 TLS: Cancel: Response: 
ctx:0xc000814240}  function=func1 line=69

INFO[0019] Scheme: http, Host: 51.178.xx.xx:3128, Port: 3128  
function=func1 line=70

INFO[0019] Response: &{Status:407 Proxy Authentication Required 
StatusCode:407 Proto:HTTP/1.1 ProtoMajor:1 ProtoMinor:1 
Header:map[Content-Language:[en] Content-Length:[3520] 
Content-Type:[text/html;charset=utf-8] Date:[Tue, 16 Mar 2021 10:03:44 GMT] 
Mime-Version:[1.0] Server:[squid/3.5.27] Vary:[Accept-Language] Via:[1.1 
vps799016 (squid/3.5.27)] X-Cache:[MISS from vps799xxx] 
X-Cache-Lookup:[NONE from vps799xxx:3128] 
X-Squid-Error:[ERR_CACHE_ACCESS_DENIED 0]] Body:0xc0004de180 
ContentLength:3520 TransferEncoding:[] Close:false Uncompressed:false 
Trailer:map[] Request:0xc0005ee100 TLS:}  function=PrintResponse 
line=33


PS: Is it possible on Google Groups to format code snippets? Markdown not 
seems to be supported

Le mardi 16 mars 2021 à 09:04:31 UTC+1, vlad...@varank.in a écrit :

> Hey there,
>
> Seems the issue hides in the chunk, where you overwrite reverse proxy's 
> "Director" method, which NewSingleHostReverseProxy creates internally. 
> Since your own director doesn't set the client request's Schema and Host, 
> you have to either do that manually or make sure you call the original 
> director.
>
> Try doing the following:
>
> proxyDirector := proxy.Director // ← keep the original director
> d := func(req *http.Request) {
> logrus.Infof("Pre-Edited request: %+v\n", req)
>
> proxyDirector(req) // ← call the original director to make sure 
> the request will go through the proxy
>
> // Inject proxy authentication headers to outgoing request into 
> new Header
> basicAuth := "Basic " + 
> base64.StdEncoding.EncodeToString([]byte(remoteServerAuth))
> req.Header.Set("Proxy-Authorization", basicAuth)
> logrus.Infof("Edited Request: %+v\n", req)
> logrus.Infof("Scheme: %s, Host: %s, Port: %s\n", req.URL.Scheme, 
> req.URL.Host, req.URL.Port())
> }
> proxy.Director = d
>
> Also, have a look at the implementation of NewSingleHostReverseProxy 
> https://go.googlesource.com/go/+/go1.16/src/net/http/httputil/reverseproxy.go#142
>
> Cheers,
> V.
>
> On Monday, March 15, 2021 at 11:37:51 PM UTC+1 hugo@gmail.com wrote:
>
>> Hi!
>> I'm actually building an automation tool based on Selenium with Go called 
>> IGopher and I have had a few requests to implement native proxy support.  
>> However, I am facing an issue with those with authentication...  
>> I can't send the proxy credentials to Chrome and without them it asks 
>> through an alert box for authentication that I can hardly interact with 
>> through Selenium (I'm not even sure it's possible in headless mode) .
>>
>> So I thought of an intermediary proxy system hosted locally by my program 
>> which will add the *Proxy-Authorization* header and transfer the request 
>> to the remote proxy:
>>
>> [image: IGopher_proxies.jpg]
>>
>> Something like this:  proxy-login-automator 
>> 
>>
>> I'm not very familiar with proxies to be honest, but I tried this 
>> approach using *NewSingleHostReverseProxy*: 
>> ```go
>> var (
>> localServerHost  string
>> remoteServerHost string
>> remoteServerAuth string
>> )
>>
>> // ProxyConfig store all remote proxy configuration
>> type ProxyConfig struct {
>> IP   string `yaml:"ip"`
>> Port int`yaml:"port"`
>> Username string `yaml:"username"`
>> Password string `yaml:"password"`
>> Enabled  bool   `yaml:"activated"`
>> }
>>
>> func PrintResponse(r *http.Response) error {
>> logrus.Infof("Response: %+v\n", r)
>> return nil
>> }
>>
>> // LaunchForwardingProxy launch forward server used to inject proxy 
>> authentication header
>> // into outgoing requests
>> func LaunchForwardingProxy(localPort uint16, remoteProxy ProxyConfig) 
>> error {
>> localServerHost = fmt.Sprintf("localhost:%d", localPort)
>> remoteServerHost = fmt.Sprintf(
>> "http://%s:%d;,

[go-nuts] How to implement localhost proxy which injects Proxy-Authorization header to incoming request and sends it to another remote proxy with Go?

[Hugo Bollon]

Hi!
I'm actually building an automation tool based on Selenium with Go called 
IGopher and I have had a few requests to implement native proxy support.  
However, I am facing an issue with those with authentication...  
I can't send the proxy credentials to Chrome and without them it asks 
through an alert box for authentication that I can hardly interact with 
through Selenium (I'm not even sure it's possible in headless mode) .

So I thought of an intermediary proxy system hosted locally by my program 
which will add the *Proxy-Authorization* header and transfer the request to 
the remote proxy:

[image: IGopher_proxies.jpg]

Something like this:  proxy-login-automator 


I'm not very familiar with proxies to be honest, but I tried this approach 
using *NewSingleHostReverseProxy*: 
```go
var (
localServerHost  string
remoteServerHost string
remoteServerAuth string
)

// ProxyConfig store all remote proxy configuration
type ProxyConfig struct {
IP   string `yaml:"ip"`
Port int`yaml:"port"`
Username string `yaml:"username"`
Password string `yaml:"password"`
Enabled  bool   `yaml:"activated"`
}

func PrintResponse(r *http.Response) error {
logrus.Infof("Response: %+v\n", r)
return nil
}

// LaunchForwardingProxy launch forward server used to inject proxy 
authentication header
// into outgoing requests
func LaunchForwardingProxy(localPort uint16, remoteProxy ProxyConfig) error 
{
localServerHost = fmt.Sprintf("localhost:%d", localPort)
remoteServerHost = fmt.Sprintf(
"http://%s:%d;,
remoteProxy.IP,
remoteProxy.Port,
)
remoteServerAuth = fmt.Sprintf(
"%s:%s",
remoteProxy.Username,
remoteProxy.Password,
)

remote, err := url.Parse(remoteServerHost)
if err != nil {
panic(err)
}

proxy := httputil.NewSingleHostReverseProxy(remote)
d := func(req *http.Request) {
logrus.Infof("Pre-Edited request: %+v\n", req)
// Inject proxy authentication headers to outgoing request into new Header
basicAuth := "Basic " + 
base64.StdEncoding.EncodeToString([]byte(remoteServerAuth))
req.Header.Set("Proxy-Authorization", basicAuth)
logrus.Infof("Edited Request: %+v\n", req)
logrus.Infof("Scheme: %s, Host: %s, Port: %s\n", req.URL.Scheme, 
req.URL.Host, req.URL.Port())
}
proxy.Director = d
proxy.ModifyResponse = PrintResponse
http.ListenAndServe(localServerHost, proxy)

return nil
}
```

With this code snippet, I'm able to intercept the request and update the 
header. 
However, resending the CONNECT request fails with the following output:

```
INFO[0028] Pre-Edited request: &{Method:CONNECT URL://google.com:443 
Proto:HTTP/1.1 ProtoMajor:1 ProtoMinor:1 
Header:map[Proxy-Connection:[Keep-Alive] User-Agent:[curl/7.68.0]] 
Body: GetBody: ContentLength:0 TransferEncoding:[] Close:false 
Host:google.com:443 Form:map[] PostForm:map[] MultipartForm: 
Trailer:map[] RemoteAddr:127.0.0.1:35610 RequestURI:google.com:443 
TLS: Cancel: Response: ctx:0xc000164300}  function=func1 
line=59

INFO[0028] Edited Request: &{Method:CONNECT URL://google.com:443 
Proto:HTTP/1.1 ProtoMajor:1 ProtoMinor:1 
Header:map[Proxy-Authorization:[Basic ] Proxy-Connection:[Keep-Alive] 
User-Agent:[curl/7.68.0]] Body: GetBody: ContentLength:0 
TransferEncoding:[] Close:false Host:google.com:443 Form:map[] 
PostForm:map[] MultipartForm: Trailer:map[] RemoteAddr:127.0.0.1:35610 
RequestURI:google.com:443 TLS: Cancel: Response: 
ctx:0xc000164300}  function=func1 line=63

INFO[0028] Scheme: , Host: google.com:443, Port: 443 function=func1 
line=64

*2021/03/15 21:35:11 http: proxy error: unsupported protocol scheme ""*
```

What am I doing wrong? Is there a way to send a CONNECT request without 
scheme in Go?
Maybe I'm doing something wrong or my approach to this problem is wrong. 
Are there better methods to achieve my goals?

If you have an idea to complete what I did or any other method, please let 
me know! :)

You can find all IGopher sources here: GitHub repository 
 (Proxy stuff excluded)

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/ededf8bc-f42d-4ee1-bad4-4ef690549f8dn%40googlegroups.com.