Thanks for responding Jarno.
So yes, as I stated Rsyslog is looking at the UNIX socket for haproxy in the 
/etc/rsyslog.d/49-haproxy.conf file, and rsyslog.conf has the default option 
enabled to load /etc/rsyslog.d/*.conf contents.

And Haproxy is logging, I just feel.. or no I wanted to make sure I was logging 
in a proper manner to not miss anything I guess.  

Since sending this, I did discover the tcplog option, as well as the 
dontlognull. Removing the latter actually helped me discover a Chinese source 
trying to hit one of these servers, resulting in Networking find out there was 
an issue with the Block rule in the firewall! So that was cool.  I did re-add 
the dontlognull just because the Route53 healthc hecks were very annoying and 
filling the log.

Anyway so as of right now, I'm getting better HTTP logs.  The LDAPS ones 
though, IDK. I've got the SNI/requested header server name so that's a big 
bonus. But I wonder if I'm still not getting everything.

Attached is a revised haproxy.cfg, I look forward to your advice and 
suggestions.
Thanks! 


On 11/23/21, 1:57 AM, "Jarno Huuskonen" <jarno.huusko...@uef.fi> wrote:

    Hi,

    On 11/22/21 16:33, Ben Hart wrote:
    > Hey there! I’ve got a handful of Haproxy servers that are serving LDAPS 
    > and HTTPS front/back ends.  I am new to this, so I built these and 
    > reused the config from the older Haproxy servers we had.
    > 
    > Anyway I mention that because I likely have little idea what I should be 
    > done here. So far everything is working.. we are able to bind and 
    > perform lookups successfully. What’s not working like I think it should 
    > is logging. I have Firewalld setup that is blocking all traffic inbound 
    > from the same internal subnet as the server, and allowing 0.0.0.0/0 in 
    > from all other sources for ports 636 and 443.
    > 
    > Rsyslog is matching on program name ‘haproxy’ and the default UNIX 
    > socket /dev/log and forwarding all info to /var/log/haproxy.log
    > 
    > Rsyslog is matching on program name ‘firewalld’ and sending all info to 
    > /var/log/firewalld.log
    > 
    > If I tail both files, I see many inbound connections allowed to port 
    > 636, but no corresponding events in the haproxy.log file.  So I’m hoping 

    Do you get any logs in haproxy.log ? (Any logs from "frontend 
    ecorp_https" ?)

    > that maybe I have something on the Haproxy side that’s not quite what it 
    > should be.  The thought is, Maybe the connection attempts are coming in, 
    > but Haproxy is not fulfilling them for some reason. And I don’t have the 
    > appropriate log options or formats setup to determine that.
    > 
    > Attached is my sanitized haproxy.cfg

     > global
     >         log /dev/log    local0
     >         log /dev/log    local1 notice
     > #   log 127.0.0.1   local1
     >         chroot /var/lib/haproxy


    You're using chroot, is rsyslog configured to listen to 
    /var/lib/haproxy/dev/log ? (And if this is centos/rhel based system 
    selinux allows rsyslog to create the socket and haproxy to connect to it).

    Have you checked that haproxy sends logs for example with
    enable logging to 127.0.0.1 and use tcpdump -nn -XX -i lo port 514 or 
    something similar ?

     > frontend ecorp_https
     >  option                tcplog

    You probably don't want to use 
    
tcplog(https://cbonte.github.io/haproxy-dconv/2.4/configuration.html#4-option%20tcplog)
 
    with mode http. AFAIK it overrides your custom log-format.

    -Jarno

    -- 
    Jarno Huuskonen

global
        log /dev/log    local0
        log /dev/log    local1 notice
#   log 127.0.0.1   local1
        chroot /var/lib/haproxy
        stats socket /run/haproxy/admin.sock mode 660 level admin
        stats timeout 30s
        maxconn 2048
        user haproxy
        group haproxy
        daemon

        # Default SSL material locations
        ca-base /etc/ssl/certs
        crt-base /etc/ssl/private

        # Default ciphers to use on SSL-enabled listening sockets.
        # For more information, see ciphers(1SSL). This list is from:
        #  https://hynek.me/articles/hardening-your-web-servers-ssl-ciphers/
        # An alternative list with additional directives can be obtained from
        #  
https://mozilla.github.io/server-side-tls/ssl-config-generator/?server=haproxy
        ssl-default-bind-ciphers 
ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:RSA+AESGCM:RSA+AES:!aNULL:!MD5:!DSS
        ssl-default-bind-options no-sslv3
        tune.ssl.default-dh-param 2048

defaults
        log     global
        mode    http
       # option  httplog
        log-format "%{+Q}o client_ip=%ci client_port=%cp 
datetime_of_request=[%tr] frontend_name_transport=%ft backend_name=%b 
server_name=%s time_to_receive_full_request=%TR Tw=%Tw Tc=%Tc response_time=%Tr 
active_time_of_request=%Ta status_code=%ST bytes_read=%B 
captured_request_cookie=%CC captured_response_cookie=%CS 
termination_state_with_cookie_status=%tsc actconn=%ac feconn=%fc beconn=%bc 
srv_conn=%sc retries=%rc srv_queue=%sq backend_queue=%bq 
captured_request_headers_default_style=%hr 
captured_response_headers_default_style=%hs server_ip=%si server_port=%sp 
frontend_name=%f http_method=%HM http_request_uri_without_query=%HP 
http_request_query_string=%HQ http_request_uri=%HU bytes_uploaded=%U 
ssl_ciphers=%sslc ssl_version=%sslv"
        option  dontlognull
        timeout connect 5000
        timeout client  50000
        timeout server  50000
        errorfile 400 /etc/haproxy/errors/400.http
        errorfile 403 /etc/haproxy/errors/403.http
        errorfile 408 /etc/haproxy/errors/408.http
        errorfile 500 /etc/haproxy/errors/500.http
        errorfile 502 /etc/haproxy/errors/502.http
        errorfile 503 /etc/haproxy/errors/503.http
        errorfile 504 /etc/haproxy/errors/504.http

# Enabling HAProxy Stats
listen  stats
        bind    *:8404
        mode    http
        log     global
        maxconn 10
        stats enable
        stats refresh 30s
        stats show-node
        stats auth admin:password
        stats uri  /stats

# LDAPS
frontend ldaps
  mode                  tcp
 # log                   global
  bind                  *:636 ssl crt /etc/ssl/private/hap/ldaps.net.pem crt 
/etc/ssl/private/hap/wild.ecorp.com.pem
  log-format            "%ci:%cp [%t] %ft %b/%s %Tw/%Tc/%Tt %B %ts 
%ac/%fc/%bc/%sc/%rc %sq/%bq capture0: %[capture.req.hdr(0)]"
  description           LDAPS Service
#  option                tcplog
 # option                logasap
  option                socket-stats
#  option                tcpka
  timeout client        60s
  tcp-request inspect-delay 5s
  tcp-request content accept if { req_ssl_hello_type 1 }
  use_backend ecorp_ldaps if { ssl_fc_sni_end ecorp.com }
  default_backend ldaps


# ECorp Stuff
frontend ecorp_https
  bind                  *:443 ssl crt /etc/ssl/private/hap/wild.ecorp.com.pem
  log           global
  mode                  http
  description           ECorpTech
  option                socket-stats
  default_backend       ecorp_https
#  option                tcplog
  capture request header Host len <len>
  capture request header Content-Type len <len>
  capture request header User-Agent len <len>
  capture request header Referer len <len>
  capture request header X-Forwarded-For len <len>
  capture response header Content-Type len <len>
  capture cookie Cookie_2 len <len>

# LDAPS backend
backend ldaps
  mode                  http
  log           global
  balance               leastconn
  server                rodc01 x.x.x.x:636 maxconn 100 check ssl fall 3 rise 1 
inter 2s verify none check check-ssl ca-file /etc/ssl/private/hap/ldaps.pem
  timeout server        60s
  timeout connect       60s
#  option                tcpka
#  option       ldap-check
  option                tcp-check
  tcp-check             connect port 636 ssl
  tcp-check send-binary 300c0201 # LDAP bind request "<ROOT>" simple
  tcp-check send-binary 01 # message ID
  tcp-check send-binary 6007 # protocol Op
  tcp-check send-binary 0201 # bind request
  tcp-check send-binary 03 # LDAP v3
  tcp-check send-binary 04008000 # name, simple authentication
  tcp-check expect binary 0a0100 # bind response + result code: success
  tcp-check send-binary 30050201034200 # unbind request

# ECorp backend LDAPS
backend ecorp_ldaps
  mode                  http
  log           global
  server                dc02 x.x.x.x:636 maxconn 20 check ssl fall 3 rise 1 
inter 10s verify none check check-ssl ca-file 
/etc/ssl/private/hap/wild.ecorp.com.pem
  timeout server        60s
  timeout connect       60s
  option                tcpka
  option                tcp-check
  tcp-check             connect port 636 ssl
  tcp-check send-binary 300c0201 # LDAP bind request "<ROOT>" simple
  tcp-check send-binary 01 # message ID
  tcp-check send-binary 6007 # protocol Op
  tcp-check send-binary 0201 # bind request
  tcp-check send-binary 03 # LDAP v3
  tcp-check send-binary 04008000 # name, simple authentication
  tcp-check expect binary 0a0100 # bind response + result code: success
  tcp-check send-binary 30050201034200 # unbind request

# Ecorp backend https
backend ecorp_https
  mode                  http
  Log                   global
  server                subca02.ecorp.com x.x.x.x:443 maxconn 20 fall 3 rise 1 
inter 10s verify none check check-ssl
  option                httpchk OPTIONS / HTTP/1.0

Reply via email to