Re: Help

2021-07-07 Thread Shawn Heisey

On 7/7/2021 6:45 AM, Anilton Silva Fernandes wrote:


Hi there.

Can I get some help from you.

I’m configuring HAProxy as a frontend on HTTPS with centified and I 
want clients to be redirect to BACKEND on HTTPS as well (443) but I 
want clients to see only HAProxy certificate, as the backend one is 
not valid.





The second one is what we would like, but does not work and says some 
erros:


[ALERT] 187/114337 (7823) : parsing [/etc/haproxy/haproxy.cfg:85] : 
'bind *:443' : unable to load SSL private key from PEM file 
'/etc/ssl/cvt.cv/accounts_cvt.pem'.



The error is shown clearly.  It's telling you that the private key is 
not contained in the file you mentioned for your certificate.


For haproxy, the certificate file must contain three things:  1) The 
server certificate.  2) Any intermediate certificates.  3) The private 
key for the server certificate.  Order is not important for 2 and 3, but 
I'm pretty sure the first certificate in the file must be the server 
cert.  The root certificate is usually not required -- the end-user's 
browser should already contain that.  I am not aware of any way to 
specify the private key in a separate file, but one might exist that I 
have never seen.


My certificate files also contain a fourth item - "DH PARAMETERS", 
generated with "openssl dhparam 2048".  Each certificate gets its own 
dhparam, and it is regenerated each time I renew the cert.


Thanks,
Shawn




Re: Help

2021-07-07 Thread Emerson Gomes
Hello Anilton,

In the "bind *:443" line, do not specify a PEM file directly, but only the
directory where your PEM file(s) resides.
Also, make sure that both the certificate and private key are contained
within the same PEM file.

It should look like this:

-BEGIN CERTIFICATE-
   xxx
-END CERTIFICATE-
-BEGIN PRIVATE KEY-
  xxx
-END PRIVATE KEY-

BR.,
Emerson

Em qua., 7 de jul. de 2021 às 14:47, Anilton Silva Fernandes <
anilton.fernan...@cvt.cv> escreveu:

> Hi there.
>
>
>
> Can I get some help from you.
>
>
>
> I’m configuring HAProxy as a frontend on HTTPS with centified and I want
> clients to be redirect to BACKEND on HTTPS as well (443) but I want clients
> to see only HAProxy certificate, as the backend one is not valid.
>
>
>
> Bellow the schematic of my design:
>
>
>
>
>
>
>
> So, on
>
>
>
> This is the configuration file I’m using:
>
>
>
> [image: frontend haproxy mode http bind *:80 bind *:443 ssl crt
> /etc/ssl/cvt.cv/accounts_cvt.pem default_backend wso2 backend wso2 mode
> http option forwardfor redirect scheme https if !{ ssl_fc } server my-api
> 10.16.18.128:443 check ssl verify none http-request set-header
> X-Forwarded-Port %[dst_port] http-request add-header X-Forwarded-Proto
> https if { ssl_fc }]
> [image: frontend web_accounts mode tcp bind 192.168.1.214:443
> default_backend accounts_servers frontend web_apimanager mode tcp bind
> 192.168.1.215:443 default_backend apimanager_servers backend
> accounts_servers balance roundrobin server accounts1 10.16.18.128:443 check
> server accounts2 10.16.18.128:443 check backend apimanager_servers balance
> roundrobin server accounts1 10.16.18.128:443 check server accounts2
> 10.16.18.128:443 check]
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> The first one is what works but we got SSL problems due to invalid
> certificates on Backend;
>
>
>
> The second one is what we would like, but does not work and says some
> erros:
>
> [ALERT] 187/114337 (7823) : parsing [/etc/haproxy/haproxy.cfg:85] : 'bind
> *:443' : unable to load SSL private key from PEM file '/etc/ssl/
> cvt.cv/accounts_cvt.pem'.
>
> [ALERT] 187/114337 (7823) : Error(s) found in configuration file :
> /etc/haproxy/haproxy.cfg
>
> [ALERT] 187/114337 (7823) : Proxy 'haproxy': no SSL certificate specified
> for bind '*:443' at [/etc/haproxy/haproxy.cfg:85] (use 'crt').
>
> [ALERT] 187/114337 (7823) : Fatal errors found in configuration.
>
> Errors in configuration file, check with haproxy check.
>
>
>
>
>
> This is on CentOS 6
>
>
>
> Thank you
>
>
>
>
>
>
>
>
>
> Melhores Cumprimentos
>
>
>
> *Anilton Fernandes | Plataformas, Sistemas e Infraestruturas*
>
> Cabo Verde Telecom, SA
>
> Group Cabo Verde Telecom
>
> Rua Cabo Verde Telecom, 1, Edificio CVT
>
> 198, Praia, Santiago, República de Cabo Verde
>
> Phone: +238 3503934 | Mobile: +238 9589123 | Email –
> anilton.fernan...@cvt.cv
>
>
>
> [image: cid:image001.jpg@01D5997A.B9848FB0]
>
>
>
>
>


Re: Proposal about new default SSL log format

2021-07-07 Thread Remi Tricot-Le Breton

Hello,

On 06/07/2021 14:55, Willy Tarreau wrote:

On Tue, Jul 06, 2021 at 12:19:56PM +0200, Tim Düsterhus wrote:

Willy,

On 7/6/21 12:12 PM, Willy Tarreau wrote:

A few points first, that are needed to address various concerns. The
goal here is to defined an HTTPS log format because that's what the
vast majority of users are dealing with day-to-day. For specific usages,
everyone already redefines log formats. But for a very basic setup, right
now it's true that httplog is a bit limited, and all it shows to help guess
there was SSL involved is to append this '~' after the frontend's name.

It was not clear to me that this log format is meant to apply to HTTPS
requests, because the example given by Remi does not include the HTTP verb
and neither the request URI (unless I need glasses). I thought it was a
format for TLS errors or something like this.

At least it was my understanding that it was mostly aimed at HTTPS given
that the discussion we had on the subject started around this :-)

Is this a mistake in the examples? Or is HAProxy going to emit multiple log
lines: One for the TLS connection and one for each HTTP request passing
through this TLS connection?

There was a discussion around logging at various layers, I don't remember
if it was on the list or in a github issue, but I remember having been
involved in this some time ago. I think that it makes sense to ultimately
be able to emit error logs at various levels (i.e. when being certain that
no other event will happen) but it's particularly confusing to log successes
at various levels. Thus typically seeing TLS handshake failures is important
but their success should not be logged as all the info will instead be
available with the traffic logs. There are already enough users complaining
about noise caused by client-aborted connections :-)

Willy


I was indeed more focused on logging SSL related information only, with 
the idea that an SSL log line could be displayed after a completed 
handshake, hence the lack of upper layer information in the log line. 
But it would indeed bring a new level of complexity in the log because 
correlating the SSL and HTTP log lines of a specific session might be a 
pain. After talking with Willy, an https log was deemed more useful. It 
would only be an extension of the existing HTTP log with SSL specific 
information added. This log format would concern every log line raised 
by the frontends using this log format (one line per HTTP response of 
the SSL session).


A quick recap of the topics raised by the multiple conversations had in 
this thread :
- we won't used tabs as separators in order to remain consistent with 
the existing log format (especially since this new format will only be 
an extension of the HTTP one)

- the date format won't change right now, it is a whole new subject
- the logged lines will all have the same "date+process_name+pid" prefix 
as the already existing logs
- the HTTP log format won't be touched yet, changing it would be a whole 
new subject as well



The log would then be as follows :


    >>> Jul  1 18:11:31 haproxy[143338]: 127.0.0.1:37740 
[01/Jul/2021:18:11:31.517] \

  ssl_frontend~ ssl_frontend/s2 0/0/0/7/+7 200 2750 - -  \
  1/1/1/1/0 0/0 {1wt.eu} {} "GET /index.html HTTP/1.1 \
  0/0/0/0 TLSv1.3/TLS_AES_256_GCM_SHA384

  Field   Format    Extract from the 
example above

  1   process_name '[' pid ']:' haproxy[143338]:
  2   client_ip ':' client_port 127.0.0.1:37740
  3   '[' request_date ']' [01/Jul/2021:18:11:31.517]
  4   frontend_name ssl_frontend~
  5   backend_name '/' server_name ssl_frontend/s2
  6   TR '/' Tw '/' Tc '/' Tr '/' Ta*   
0/0/0/7/+7

  7 status_code 200
  8 bytes_read* 2750
  9 captured_request_cookie -
 10 captured_response_cookie -
 11 termination_state 
 12   actconn '/' feconn '/' beconn '/' srv_conn '/' retries*    
1/1/1/1/0
 13   srv_queue '/' 
backend_queue  0/0

 14   '{' captured_request_headers* '}' {haproxy.1wt.eu}
 15   '{' captured_response_headers* 
'}'    {}
 16   '"' http_request '"'  "GET /index.html 
HTTP/1.1"
 17 *conn_status '/' SSL fe hsk error '/' SSL vfy '/' SSL CA vfy*  
0/0/0/0

 18 *ssl_version '/' ssl_ciphers* TLSv1.3/TLS_AES_256_GCM_SHA384

and the corresponding log-format :

    %ci:%cp [%tr] %ft %b/%s %TR/%Tw/%Tc/%Tr/%Ta %ST %B %CC \
    %CS %tsc %ac/%fc/%bc/%sc/%rc %sq/%bq %hr %hs %{+Q}r \
*%[conn_err_code]/%[ssl_fc_hsk_err]/%[ssl_c_err]/%[ssl_c_ca_err]* \
*%sslv/%sslc*

Thanks

Rémi





[ANNOUNCE] haproxy-2.4.2

2021-07-07 Thread Christopher Faulet

Hi,

HAProxy 2.4.2 was released on 2021/07/07. It added 35 new commits
after version 2.4.1.

The main purpose of this release is to fix a possible deadlock introduced
into the previous release when the maxconn of a server was changed via an
agent-check. In this case, the server lock was held twice leading to a
deadlock, the first time in the agent-check itself, the second time in the
agent response parsing when the maxconn setting was changed. Note the 2.3.10
is also affected by this bug and a new release will be emitted with the fix
very soon.

Some bugs in the resolvers were also fixed. First one was a possible ABBA
deadlock when the server's FQDN was set from the CLI socket. In this case,
the server lock was held before the resolver lock while the opposite is done
when a resolution is performed. It is a quite old bug only discovered
recently by chance. In addition, to avoid any ambiguities, it is now
forbidden to set server's FQDN on the CLI if SRV resolution is enabled for
the server. Second bug is about the SRV resolution when the server state was
loaded from a file. In this case, it was possible to never renew the server
information loaded during startup if the DNS server no longer announced the
corresponding SRV record. To work around this bug, a task is attached to
servers relying on SRV resolution to purge outdated information, if any. Two
regressions of the 2.4.1 were also fixed. First, the first server of a
template based on a SRV resolution was not resolved anymore. The same bug
existed for single servers relying on SRV resolution. Second, a server might
be ignored during resolution if its IP was set by the libc during
startup. Finally, information about SRV resolution found in a server state
file are now ignored if the corresponding server is no longer configured to
rely on the same SRV resolution.

Willy fixed a bug in the sock part leading to high CPU usage because some
early connection failures might be missed if a connect() reported an error
directly via the poller without ever reporting send readiness. It is an old
bug revealed by recent changes.

Amaury implemented the scheme-based URI normalization as described in
rfc3986 6.3.2. It means the port of an URI is removed if it is a default
port according to the URI scheme (80/http and 443/https). On HTTP/1, the
normalization is only performed on requests using an absolute-form target
URI. On HTTP/2, It is performed on requests with a scheme and an
authority. It is the most common case, except CONNECT. This change will be
notably useful to not confuse users who are accustomed to use the host for
routing without specifying default ports. This problem was recently
encountered with Firefox which specify the 443 default port for HTTP/2
WebSocket Extended CONNECT.

Some may have noticed the support of "set-src" adn "set-sr-port" actions for
"tcp-request content" rules was first added then reverted. While this
support must be added, it should be delayed to fix a design problem by
setting client source address from the L7 layer. This problem already exists
because these actions are supported by "http-request" rules (See #90 on
github for details). So instead of adding more confusion, we have chosen to
wait a bit and delayed the feature.

Other commits are regular bug fixes and cleanups, mainly:

  * The MQTT parser was fixed to properly handle large client ID or empty
one in CONNECT message.

  * Tim fixed a bug in the cache to properly handle empty 'accept-encoding'
header.

  * The "show fd" command output was fixed to displayed the server name as
/ instead of the reverse.

  * The configuration manual was fixed to add missing documentation of some
keywords and to refresh "mysql-check" description.

As said at the beginning of this announce, a new 2.3 release will be emitted
very soon. We are really annoyed to have delayed so much this release. The
same is true for the last 2.2 and 2.0 releases. For these versions, we will
try to emit new releases the next week.

Thanks everyone for your help and your contributions!

Please find the usual URLs below :
   Site index   : http://www.haproxy.org/
   Discourse: http://discourse.haproxy.org/
   Slack channel: https://slack.haproxy.org/
   Issue tracker: https://github.com/haproxy/haproxy/issues
   Wiki : https://github.com/haproxy/wiki/wiki
   Sources  : http://www.haproxy.org/download/2.4/src/
   Git repository   : http://git.haproxy.org/git/haproxy-2.4.git/
   Git Web browsing : http://git.haproxy.org/?p=haproxy-2.4.git
   Changelog: http://www.haproxy.org/download/2.4/src/CHANGELOG
   Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/


---
Complete changelog :
Amaury Denoyelle (7):
  BUG/MAJOR: server: fix deadlock when changing maxconn via agent-check
  REGTESTS: fix maxconn update with agent-check
  MINOR: http: implement http_get_scheme
  MEDIUM: http: implement scheme-based normalization
  MEDIUM: 

Help

2021-07-07 Thread Anilton Silva Fernandes
Hi there.

Can I get some help from you.

I'm configuring HAProxy as a frontend on HTTPS with centified and I want 
clients to be redirect to BACKEND on HTTPS as well (443) but I want clients to 
see only HAProxy certificate, as the backend one is not valid.

Bellow the schematic of my design:

[cid:image004.png@01D77325.9DBECB70]


So, on

This is the configuration file I'm using:


[frontend haproxy mode http bind *:80 bind *:443 
ssl crt /etc/ssl/cvt.cv/accounts_cvt.pem default_backend wso2backend 
wso2 mode http option forwardfor redirect scheme https if !{ ssl_fc 
} server my-api 10.16.18.128:443 check ssl verify none http-request 
set-header X-Forwarded-Port %[dst_port] http-request add-header 
X-Forwarded-Proto https if { ssl_fc }]
[frontend web_accounts  mode tcp  bind 192.168.1.214:443  
default_backend accounts_serversfrontend web_apimanager  mode tcp  
bind 192.168.1.215:443  default_backend apimanager_serversbackend 
accounts_servers  balance roundrobin  server  accounts1 
10.16.18.128:443 check  server  accounts2 10.16.18.128:443 checkbackend 
apimanager_servers  balance roundrobin  server  accounts1 
10.16.18.128:443 check  server  accounts2 10.16.18.128:443 check]



























The first one is what works but we got SSL problems due to invalid certificates 
on Backend;

The second one is what we would like, but does not work and says some erros:
[ALERT] 187/114337 (7823) : parsing [/etc/haproxy/haproxy.cfg:85] : 'bind 
*:443' : unable to load SSL private key from PEM file 
'/etc/ssl/cvt.cv/accounts_cvt.pem'.
[ALERT] 187/114337 (7823) : Error(s) found in configuration file : 
/etc/haproxy/haproxy.cfg
[ALERT] 187/114337 (7823) : Proxy 'haproxy': no SSL certificate specified for 
bind '*:443' at [/etc/haproxy/haproxy.cfg:85] (use 'crt').
[ALERT] 187/114337 (7823) : Fatal errors found in configuration.
Errors in configuration file, check with haproxy check.


This is on CentOS 6

Thank you




Melhores Cumprimentos

Anilton Fernandes | Plataformas, Sistemas e Infraestruturas
Cabo Verde Telecom, SA
Group Cabo Verde Telecom
Rua Cabo Verde Telecom, 1, Edificio CVT
198, Praia, Santiago, República de Cabo Verde
Phone: +238 3503934 | Mobile: +238 9589123 | Email - 
anilton.fernan...@cvt.cv

[cid:image001.jpg@01D5997A.B9848FB0]




image003.emz
Description: image003.emz


oledata.mso
Description: oledata.mso


image007.emz
Description: image007.emz


image008.emz
Description: image008.emz


Re: Proposal about new default SSL log format

2021-07-07 Thread Tim Düsterhus

Remi,

On 7/7/21 4:43 PM, Remi Tricot-Le Breton wrote:
A quick recap of the topics raised by the multiple conversations had in 
this thread :
- we won't used tabs as separators in order to remain consistent with 
the existing log format (especially since this new format will only be 
an extension of the HTTP one)

- the date format won't change right now, it is a whole new subject
- the logged lines will all have the same "date+process_name+pid" prefix 
as the already existing logs
- the HTTP log format won't be touched yet, changing it would be a whole 
new subject as well



The log would then be as follows :


     >>> Jul  1 18:11:31 haproxy[143338]: 127.0.0.1:37740 
[01/Jul/2021:18:11:31.517] \

   ssl_frontend~ ssl_frontend/s2 0/0/0/7/+7 200 2750 - -  \
   1/1/1/1/0 0/0 {1wt.eu} {} "GET /index.html HTTP/1.1 \
   0/0/0/0 TLSv1.3/TLS_AES_256_GCM_SHA384

*snip*

and the corresponding log-format :

     %ci:%cp [%tr] %ft %b/%s %TR/%Tw/%Tc/%Tr/%Ta %ST %B %CC \
     %CS %tsc %ac/%fc/%bc/%sc/%rc %sq/%bq %hr %hs %{+Q}r \
*%[conn_err_code]/%[ssl_fc_hsk_err]/%[ssl_c_err]/%[ssl_c_ca_err]* \
*%sslv/%sslc*



Looks good to me, this is a natural extension of the existing format.

Best regards
Tim Düsterhus



[ANNOUNCE] haproxy-2.3.11

2021-07-07 Thread Christopher Faulet

Hi,

HAProxy 2.3.11 was released on 2021/07/07. It added 106 new commits
after version 2.3.10.

This release is quite huge, mainly because the previous release is a bit
old. Several major bugs were fixed:

  * Amaury fixed a possible deadlock if "set maxconn server" command was used
when there was a pending connection ready to be dequeued. This fix
introduced another possible deadlock that was also fixed.

  * A possible infinite loop in process_stream() was fixed because a
connection error was reported while the stream was waiting for a
retry. The bug was happening because the previous server connection was
only released when a new connection attempt was performed, leaving a
place to catch unexpected connection error. Now the server connection is
release ASAP on retry.

  * A possible race between free() and pool_alloc() in the pools lockless
variant was fixed. The bug was always there. It is a bit tricky. For
details see the corresponding commit and the issue #1275.

  * A bug in the HTX defragmentation was fixed. It was happening when the
defragmentation was performed to be able to expand an HTX block. The bug
might be encountered in the HTTP compression filter or in HTTP header
replacement.

  * An old bug preventing the dequeuing process was fixed, with Pierre
Cheynier's help. The bug was happening for servers with a very low
maxconn because the load balancing was not skipped when a new connection
was picked from the proxy's or server's queue.

Less important but still noticeable fixes:

  * Willy fixed a bug in the sock part leading to high CPU usage because
some early connection failures might be missed if a connect() reported
an error directly via the poller without ever reporting send
readiness. It is an old bug revealed by recent changes.

  * The SHCTX code, used for the cache and the SSL sessions, was fixed to
use thread-based locking when USE_PRIVATE_CACHE was enabled.

  * Emeric fixed several bugs on the peers to improve the synchronization
process. He also fixed resolvers performance issues leading to watchdog
panics when dealing with large DNS responses. In addition, many bugs
were fixed in this area, most of them about the SRV resolution.

  * A bug in the HTTP compression that could occasionally lead to truncated
or corrupted responses was fixed.

  * The abortonclose option was fixed.

  * A bug in lua HTTP applet was fixed to be sure to notify the producer
side when some data are consumed by the applet, to not block data
receipt.

  * The FCGI multiplexer was slightly improved to send a relative path
instead of a normalized URI to an application and to expose
SERVER_SOFTWARE parameter by default.

  * http-ignore-probes is now respected for H2 connections. When this option
is set, no errors are reported anymore when connections are aborted
during preface.

  * As a consequence of the bug fixed in the pools, the code was
simplified. The lockless implementation is used everywhere, resulting in
the removal of the very old locked implementation that was kept for
non-capable architectures. As a result, threads will now be faster on
less common architectures (e.g. i686, MIPS, PPC64, ...)

  * The "show fd" command output was fixed to displayed the server name as
/ instead of the reverse.

  * Synchronous connect when a tcpcheck is started are now properly
handled. This fixes a bug that was happening when several connections
come one after another.

  * "url_ip" and "url_port" sample fetches were fixed to properly handled
url parsing error.

The rest is less visible but contains, as usual, cleanups, small fixes here and
there, improvements ... It is strongly advised to update to this version. Thanks
everyone for your help and your contributions!

Please find the usual URLs below :
   Site index   : http://www.haproxy.org/
   Discourse: http://discourse.haproxy.org/
   Slack channel: https://slack.haproxy.org/
   Issue tracker: https://github.com/haproxy/haproxy/issues
   Wiki : https://github.com/haproxy/wiki/wiki
   Sources  : http://www.haproxy.org/download/2.3/src/
   Git repository   : http://git.haproxy.org/git/haproxy-2.3.git/
   Git Web browsing : http://git.haproxy.org/?p=haproxy-2.3.git
   Changelog: http://www.haproxy.org/download/2.3/src/CHANGELOG
   Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/


---
Complete changelog :
Alex (1):
  DOC: use the req.ssl_sni in examples

Alexandar Lazic (1):
  DOC/MINOR: move uuid in the configuration to the right alphabetical order

Amaury Denoyelle (5):
  BUG/MINOR: http_fetch: fix possible uninit sockaddr in fetch_url_ip/port
  BUG/MAJOR: server: prevent deadlock when using 'set maxconn server'
  BUG/MINOR: stick-table: insert srv in used_name tree even with fixed id
  BUG/MAJOR: server: fix deadlock when changing 

Re: ratio spam/useful message

2021-07-07 Thread Willy Tarreau
Hi Julien,

On Tue, Jul 06, 2021 at 11:06:05AM +0200, Julien Pivotto wrote:
> Hello,
> 
> Lately, the ratio spam/useful message on the ML has been quite high.

Well, that's not what I'm seeing in the archives here:

   https://www.mail-archive.com/haproxy@formilux.org/

I had to manually delete 9 messages this month and 29 last month from
my mbox, or 1 per day, I don't find it high at all, quite the opposite!
I've added the recurrent "NFP workshops" ones and the occasional
"sock-raw.org" to the block list, but I'm not seeing that much come in.
Are you getting more from the list ? Am I the only one getting so few ?
Even checking the local archives here didn't reveal anything exceptional.

Willy