Apologies, my mail client went stupid. Here's the log entry unmangled:
198.228.211.13:60848 api~ platform-push/i-84d931a5 49562/0/-1/-1/49563
0/0/0/0/0 0/0 691/212 503 CC-- 4F8E-4624 + GET
/1/sync/notifications/subscribe?sync_box_id=12496&sender=D7A9F93D-F653-4527-A022-383AD55A1943
HTTP/1.1
-Patrick
------------------------------------------------------------------------
*From: *Patrick Hemmer <hapr...@stormcloud9.net>
*Sent: * 2014-04-01 15:20:15 E
*To: *haproxy@formilux.org
*Subject: *haproxy intermittently not connecting to backend
> We have an issue with haproxy (1.5-dev22-1a34d57) where it is
> intermittently not connecting to the backend server. However the
> behavior it is exhibiting seems strange.
> The reason I say strange is that in one example, it logged that the
> client disconnected after ~49 seconds with a connection flags of
> "CC--". However our config has "timeout connect 5000", so it should
> have timed out connecting to the backend server after 5 seconds.
> Additionally we have "retries 3" in the config, so upon timing out, it
> should have tried another backend server, but it never did (the
> retries counter in the log shows "0").
> At the time of this log entry, the backend server is responding
> properly. For the ~49 seconds prior to the log entry, the backend
> server has taken other requests. The backend server is also another
> haproxy (same version).
>
> Here's an example of one such log entry:
>
> 198.228.211.13:60848 api~ platform-push/i-84d931a5 49562/0/-1/-1/49563
> 0/0/0/0/0 0/0 691/212
> &
> lt;
> span class="t" style="border-color: rgb(204, 204, 204); font-style: normal;
> cursor: pointer;">503 CC-- 4F8E-4624 + GET
> /1/sync/notifications/subscribe?sync_box_
> id=12345&sender=27B9A93C-F473-4385-A662-352AD34A2453 HTTP/1.1
>
> The log format is defined as:
> %ci:%cp\ %ft\ %b/%s\ %Tq/%Tw/%Tc/%Tr/%Tt\ %ac/%fc/%bc/%sc/%rc\
> %sq/%bq\ %U/%B\ %ST\ %tsc\ %ID\ +\ %r
>
> Running a "show errors" on the stats socket did not return any
> relevant results.
>
> Here's the relevant portions of the haproxy config. It is not the
> entire thing as the whole config is 1,513 lines long.
>
> global
> log 127.0.0.1 local0
> maxconn 20480
> user haproxy
> group haproxy
> daemon
> stats socket /var/run/hapi/haproxy/haproxy.sock level admin
>
> defaults
> log global
> mode http
> option httplog
> option dontlognull
> option log-separate-errors
> retries 3
> option redispatch
> timeout connect 5000
> timeout client 60000
> timeout server 170000
> option clitcpka
> option srvtcpka
> option abortonclose
> option splice-auto
> monitor-uri /haproxy/ping
> stats enable
> stats uri /haproxy/stats
> stats refresh 15
> stats auth user:pass
>
> frontend api
> bind *:80
> bind *:443 ssl crt /etc/haproxy/server.pem
> maxconn 20000
> option httpclose
> option forwardfor
> acl internal src 10.0.0.0/8
> acl have_request_id req.fhdr(X-Request-Id) -m found
> http-request set-nice -100 if internal
> http-request add-header X-API-URL %[path] if !internal
> http-request add-header X-Request-Timestamp %Ts.%ms
> http-request add-header X-Request-Id %[req.fhdr(X-Request-Id)] if
> internal have_request_id
> http-request set-header X-Request-Id %{+X}o%pid-%rt if !internal ||
> !have_request_id
> http-request add-header X-API-Host i-4a3b1c6a
> unique-id-format %{+X}o%pid-%rt
> log-format %ci:%cp\ %ft\ %b/%s\ %Tq/%Tw/%Tc/%Tr/%Tt\
> %ac/%fc/%bc/%sc/%rc\ %sq/%bq\ %U/%B\ %ST\ %tsc\ %ID\ +\ %r
> default_backend DEFAULT_404
>
> acl rewrite-found req.hdr(X-Rewrite-ID,1) -m found
>
> acl nqXn_path path_reg ^/1/sync/notifications/subscribe/([^\ ?]*)$
> acl nqXn_method method OPTIONS GET HEAD POST PUT DELETE TRACE
> CONNECT PATCH
> http-request set-header X-Rewrite-Id nqXn if !rewrite-found
> nqXn_path nqXn_method
> acl rewrite-nqXn req.hdr(X-Rewrite-Id) -m str nqXn
> use_backend platform-push if rewrite-nqXn
> reqrep ^(OPTIONS|GET|HEAD|POST|PUT|DELETE|TRACE|CONNECT|PATCH)\
> /1/sync/notifications/subscribe/([^\ ?]*)([\ ?].*|$) \1\
> /1/sync/subscribe/\2\3 if rewrite-nqXn
>
>
> backend platform-push
> option httpchk GET /ping
> default-server inter 15s fastinter 1s
> server i-6eaf724d 10.230.23.64:80 check observe layer4
> server i-84d931a5 10.230.42.8:80 check observe layer4
>
