Re: Lua Shell letsencrypt

[joris dedieu]

2015-12-05 23:42 GMT+01:00  :
> On Fri, 4 Dec 2015 00:23:53 -0700
> Mela Luca  wrote:
>
>> I am looking to automate letsencrypt with lua, the process would be to 
>> detect to see if the domain has a cert already, if not it would execute 
>> letsencrypt on the domain.
>> Any thought if this would be possible to do with lua. I am guessing using 
>> the os.execute.
>>
>
> I'm not sure that you're using the good way to do this:
>
>  - I don't known letsencrypt very well, but I heard that the
>letsecncrypt framework expect a confirmation that the requester is
>the real owner of the web site. It requires to the owner to add a
>special webpage at a special url. So the process is very slow and it
>cannot done during an http request timing.

Also don't forget that you can be flood by bots using arbitrary Host headers.

>
>  - os.execute() is a blocking action. While HAProxy is waiting for the
>script response, it does nothing, and all the traffic is blocked.
>
> Actuelly, the Lua in HAProxy only communicates with other process with
> the Socket provided by the Lua/HAProxy API.

IMHO the right approach is to use async communication (any ASMQ
middleware, 0MQ, IRC, what else ...) between haproxy and the
letsencrypt client or any ACME protocol implementation.

It also should be useful for other stuff.

Joris


>
> Thierry
>

Re: lua authentication

[joris dedieu]

2015-12-06 3:44 GMT+01:00 Grant Haywood :
> I found a pretty good starting point
>
> https://github.com/morganfainberg/HAProxyKeystoneMiddlware
>
> if i do anything with ldap ill post it...
>
> - Original Message -
> From: "Grant Haywood" 
> To: "thierry fournier" 
> Cc: "Igor Cicimov" , "HAProxy" 
> , "Baptiste" 
> Sent: Saturday, December 5, 2015 6:48:52 PM
> Subject: Re: lua authentication
>
> I see.
> Still, is there an example of authenticating an Http connection in lua?
>
> im fairly certan i can do a JWT implementation
>
> and for LDAP, it may still easyer to proxy a simple (non-ldap) message over a 
> socket, and write a bridge to ldap daemon in something thats not lua. (use at 
> your own risk/understanding/vetting)
>
> kind of like this https://doc.powerdns.com/md/authoritative/backend-pipe/
> (i know thats not for auth, but same concept)
>
> - Original Message -
> From: "thierry fournier" 
> To: "Baptiste" 
> Cc: "Grant Haywood" , "Igor Cicimov" 
> , "HAProxy" 
> Sent: Saturday, December 5, 2015 3:36:32 PM
> Subject: Re: lua authentication
>
> Hi,
>
> I complement, I would say, that the Lua bindings for the standard
> Openldap client exists, but unfortunately, the operation is blocking,
> and doesn't run very well with HAProxy.
>
> It seems that a Lua rewrite of the LDAP protocol using standard Lua
> HAProxy socket is a solution, but this is a big development. Maybe a
> partial implementation (juste the binding) will be usefull.
>
> Thierry
>
>
>
> On Fri, 4 Dec 2015 08:35:41 +0100
> Baptiste  wrote:
>
>> current Lua implementation already allows asynchronous network sockets.
>> Now, what you need to do is to code a basic LDAP auth request in Lua
>> and be able to parse the response.
>>
>> Baptiste
>>
>>
>>
>> On Thu, Dec 3, 2015 at 11:58 PM, Grant Haywood  
>> wrote:
>> > Thats exactly what I am wanting to code, I just need an example of how to 
>> > do auth, like userlist, inside of lua.
>> >
>> > - Original Message -
>> > From: "Igor Cicimov" 
>> > To: "Grant Haywood" 
>> > Cc: "HAProxy" 
>> > Sent: Thursday, December 3, 2015 3:58:28 PM
>> > Subject: Re: lua authentication
>> >
>> >
>> >
>> >
>> > Hi Grant,
>> >
>> >
>> >
>> > On Fri, Dec 4, 2015 at 7:46 AM, Grant Haywood < gr...@iowntheinter.net > 
>> > wrote:
>> >
>> >
>> > Hello,
>> >
>> > I was wondering if there is a basic example of using lua to do 
>> > authentication?
>> >
>> > I am specificaly interested in constructing 'ldap' and 'jwt' versions of 
>> > the 'userlist' block
>> >
>> > thx in advance for your time
>> >
>> >
>> >
>> > Excellent question. One feature I would love to see in haproxy is support 
>> > for ldap authentication. It would be awesome If that could be done via lua.

IHMO it should be easier to use  SASL.

Joris

>> >
>> >
>> > Thanks,
>> >
>> > Igor
>> >
>>
>

Re: SSL handshake failure when using "send-proxy" on HTTPS backend

[PiBa-NL]

Hi,

Ive never used nginx and have little experience with proxy_protocol.. 
But could it be an issue that on the same port your both using and not 
using proxy protocol? What happens if you remove the first server 
definition there?


server {
listen 10.0.80.1:8443;
server {
listen 10.0.80.1:8443 default_server ssl proxy_protocol;

Just a thought..

Regards,
PiBa-NL

Op 6-12-2015 om 12:25 schreef Lukas Erlacher:

Hi,

On 12/04/2015 04:27 PM, Jonathan Leroy - Inikup wrote:

2015-12-04 13:23 GMT+01:00 Lukas Erlacher :

Please show the nginx config.


Hi Luke,

Here's the Nginx config :
https://gist.githubusercontent.com/jleroy/ab45c328263731c46ec1/raw/69af9edc154329c113aad588ff5f9501edfd61b1/gistfile1.txt 



Thanks,



I can't find an obvious error with this. When I tried combining SSL 
and proxy protocol in Postfix, it didn't work due to a bug in Postfix. 
Maybe you should try to ask an nginx support list instead.


Best,
Luke

外贸公司与SOHO一族的必备神器---外贸客户自主开发与搜索软件

[todayls2]

  
  
您好
 
双喜软件是一款专门帮助企业开发全球客户的软件,集合全球各大主流商业搜索引擎，及全球各国当地常用的区域引擎.
 
利用产品及目标客户群的特征设置相应的关键词，通过这些搜索引擎来搜索定位到目标客户的官方网站、提取联系方式.
 
将全球有意向的行业客户都搜索出来，并一对一发送开发信，每天联系上万的目标客户，帮助外贸企业在快速成交一些订单。
 
 
 
更多详细咨询 173561765 (QQ) 免费在线演示软件的功能和效果-若不需要此类邮件请设置拒收，抱歉打挠
 
  

Re: SSL handshake failure when using "send-proxy" on HTTPS backend

[Lukas Erlacher]

Hi,

On 12/04/2015 04:27 PM, Jonathan Leroy - Inikup wrote:

2015-12-04 13:23 GMT+01:00 Lukas Erlacher :

Please show the nginx config.


Hi Luke,

Here's the Nginx config :
https://gist.githubusercontent.com/jleroy/ab45c328263731c46ec1/raw/69af9edc154329c113aad588ff5f9501edfd61b1/gistfile1.txt

Thanks,



I can't find an obvious error with this. When I tried combining SSL and proxy 
protocol in Postfix, it didn't work due to a bug in Postfix. Maybe you should 
try to ask an nginx support list instead.

Best,
Luke

Re: [1.6.1] Utilizing http-reuse

[Krishna Kumar (Engineering)]

Hi Willy, Baptiste,

Apologies for the delay in reproducing this issue and in responding.

I am using HAProxy 1.6.2 and am still finding that connection reuse is not
happening in my setup. Attaching the configuration file, command line
arguments, and the tcpdump (80 packets in all), in case it helps. HAProxy
is configured with a single backend. The same client makes two requests,
one a telnet with a GET request for a 128 byte file, and the second 'ab -k'
command to retrieve the same file.

172.20.97.36: Client
10.34.73.174: HAProxy
10.32.121.94: Server

Telnet from client with GET:
GET /128 HTTP/1.1
Host: www.example.com
User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)

Followed immediately with:
ab -k -n 10 -c 1 http://10.34.73.174/1K

Packets #1-7: Telnet to haproxy, and a GET request made
Packets #8-9: HAProxy opens connection to single backend
Packets #10-15: Response from server, relays data back to the client,
connection from Client->HAProxy and HAProxy->server is
kept open.
Packets #16-19 (5 seconds later): Same client, run 'ab -k'
Packet #20-72: New connection to same backend, and data transfer.
Packet #73: 'ab' closes connection to HAProxy
Packet #74: HAProxy closes connection to 'ab'.
Packet #75: HAProxy closes connection to backend.
Packets #77-81: Telnet closes connection

Configuration file:
--
global
daemon
maxconn 1

defaults
mode http
option http-keep-alive
balance leastconn
option splice-response
option clitcpka
option srvtcpka
option tcp-smart-accept
option tcp-smart-connect
option contstats
timeout http-keep-alive 1800s
timeout http-request 1800s
timeout connect 60s
timeout client 1800s
timeout server 1800s

frontend private-frontend
mode http
maxconn 1
bind 10.34.73.174:80
default_backend private-backend

backend private-backend
http-reuse always
server 10.32.121.94 10.32.121.94:80 maxconn 1

>From the above, it is seen that HAProxy opens a second connection to the
server on same GET request from the client.

Can you please take a look and suggest what needs to be done to get reuse
working?

$ haproxy -vv
HA-Proxy version 1.6.2 2015/11/03
Copyright 2000-2015 Willy Tarreau 

Build options :
  TARGET  = linux2628
  CPU = generic
  CC  = gcc
  CFLAGS  = -O3 -g -fno-strict-aliasing -Wdeclaration-after-statement
  OPTIONS = USE_ZLIB=1 USE_OPENSSL=1 USE_PCRE=1

Default settings :
  maxconn = 2000, bufsize = 16384, maxrewrite = 1024, maxpollevents = 200

Encrypted password support via crypt(3): yes
Built with zlib version : 1.2.8
Compression algorithms supported : identity("identity"),
deflate("deflate"), raw-deflate("deflate"), gzip("gzip")
Built with OpenSSL version : OpenSSL 1.0.1f 6 Jan 2014
Running on OpenSSL version : OpenSSL 1.0.1f 6 Jan 2014
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports prefer-server-ciphers : yes
Built with PCRE version : 8.35 2014-04-04
PCRE library supports JIT : no (USE_PCRE_JIT not set)
Built without Lua support
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT
IP_FREEBIND

Available polling systems :
  epoll : pref=300,  test result OK
   poll : pref=200,  test result OK
 select : pref=150,  test result OK
Total: 3 (3 usable), will use epoll.

Thanks,
- Krishna Kumar


On Thu, Nov 12, 2015 at 12:50 PM, Willy Tarreau  wrote:

> Hi Krishna,
>
> On Wed, Nov 11, 2015 at 03:22:54PM +0530, Krishna Kumar (Engineering)
> wrote:
> > I just tested with 128K byte file (run 4 wgets
> > in parallel each retrieving 128K). Here, I see 4 connections being
> opened, and
> > lots of data packets in the middle, finally followed by 4 connections
> > being closed. I
> > can test with "ab -k" option to simulate a browser, I guess, will try
> that.
>
> In my tests, ab -k definitely works.
>
> > > Is wget advertising HTTP/1.1 in the request ? If not that could
> >
> > Yes, tcpdump shows HTTP/1.1 in the GET request.
>
> OK.
>
> > >   - proxy protocol used to the server
> > >   - SNI sent to the server
> > >   - source IP binding to client's IP address
> > >   - source IP binding to anything dynamic (eg: header)
> > >   - 401/407 received on a server connection.
> >
> > I am not doing any of these specifically. Its a very simple setup where
> the
> > client@ip1 connects to haproxy@ip2 and requests 128 byte file, which
> > is handled by server@ip3.
>
> OK. I don't see any reason for this not to work then.
>
> > I was doing this in telnet:
> >
> > GET /128 HTTP/1.1
> > Host: www.example.com
> > User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)
>
> Looks fine as well. Very strange. I have no idea what would not at the
> moment, I suspect this is something stupid and obvious but am failing
> to spot it :-/
>
> Willy
>
>


packets.pcap
Description: application/vnd.tcpdump.pcap

Re: SSL handshake failure when using "send-proxy" on HTTPS backend

[Jonathan Leroy - Inikup]

2015-12-06 12:25 GMT+01:00 Lukas Erlacher :
> I can't find an obvious error with this. When I tried combining SSL and
> proxy protocol in Postfix, it didn't work due to a bug in Postfix. Maybe you
> should try to ask an nginx support list instead.

Thanks, I'll try that.


-- 
Jonathan Leroy
http://www.inikup.com/
Tel: +33 (0)9 74 77 41 72

Re: [1.6.1] Utilizing http-reuse

[Willy Tarreau]

Hi Krishna,

On Mon, Dec 07, 2015 at 08:31:19AM +0530, Krishna Kumar (Engineering) wrote:
> Hi Willy, Baptiste,
> 
> Apologies for the delay in reproducing this issue and in responding.
> 
> I am using HAProxy 1.6.2 and am still finding that connection reuse is not
> happening in my setup. Attaching the configuration file, command line
> arguments, and the tcpdump (80 packets in all), in case it helps. HAProxy
> is configured with a single backend. The same client makes two requests,
> one a telnet with a GET request for a 128 byte file, and the second 'ab -k'
> command to retrieve the same file.
(...)
> Can you please take a look and suggest what needs to be done to get reuse
> working?

Thank you for this detailed report. I agree that your config shows that
it should work and the pcap shows that it doesn't. I've taken a quick
look at the code and have no idea why it does this. I'm going to investigate
and will keep you informed.

Thanks!
Willy

Re: [1.6.1] Utilizing http-reuse

[Krishna Kumar (Engineering)]

Thanks a lot, Willy.

Regards,
- Krishna

On Mon, Dec 7, 2015 at 11:59 AM, Willy Tarreau  wrote:

> Hi Krishna,
>
> On Mon, Dec 07, 2015 at 08:31:19AM +0530, Krishna Kumar (Engineering)
> wrote:
> > Hi Willy, Baptiste,
> >
> > Apologies for the delay in reproducing this issue and in responding.
> >
> > I am using HAProxy 1.6.2 and am still finding that connection reuse is
> not
> > happening in my setup. Attaching the configuration file, command line
> > arguments, and the tcpdump (80 packets in all), in case it helps. HAProxy
> > is configured with a single backend. The same client makes two requests,
> > one a telnet with a GET request for a 128 byte file, and the second 'ab
> -k'
> > command to retrieve the same file.
> (...)
> > Can you please take a look and suggest what needs to be done to get reuse
> > working?
>
> Thank you for this detailed report. I agree that your config shows that
> it should work and the pcap shows that it doesn't. I've taken a quick
> look at the code and have no idea why it does this. I'm going to
> investigate
> and will keep you informed.
>
> Thanks!
> Willy
>
>

Re: SSL handshake failure when using "send-proxy" on HTTPS backend

[Jonathan Leroy - Inikup]

2015-12-06 16:14 GMT+01:00 PiBa-NL :
> Hi,
>
> Ive never used nginx and have little experience with proxy_protocol.. But
> could it be an issue that on the same port your both using and not using
> proxy protocol? What happens if you remove the first server definition
> there?
>
> server {
> listen 10.0.80.1:8443;
> server {
> listen 10.0.80.1:8443 default_server ssl proxy_protocol;
>
> Just a thought..

Hi,

See my previous response to Lukas Tribus. With Nginx, listening
options must be specified only once for the same address+port
combinations.



-- 
Jonathan Leroy
http://www.inikup.com/
Tel: +33 (0)9 74 77 41 72

Re: what's the difference between rspdel and http-response del-header

[Ruoshan Huang]

> On Dec 4, 2015, at 2:41 AM, Bryan Talbot  wrote:
> 
> rspdel is older and remains for backwards compatibility.

if so, maybe the document should flag those directives `deprecated` :)

Thank you for explaining.

--
Good day!
ruoshan