On 22/07/2016 11:53 πμ, Willy Tarreau wrote: > Hi Cyrus, > > On Thu, Jul 21, 2016 at 11:22:06PM -0700, Cyrus Katrak wrote: >> Greetings from Slack Technologies, >> >> We are evaluating HAProxy as a load balancing solution and so far are quite >> pleased with >> it. One feature we require is the ability to log when a connection started >> (option >> logasap), and again when it is terminated. We have many long lived websocket >> connections, >> and this level of introspection is invaluable for debugging. >> >> We are currently accomplishing this with the following patch: diff -u -r >> haproxy-1.6.5-orig/src/proto_http.c haproxy-1.6.5/src/proto_http.c --- >> haproxy-1.6.5-orig/src/proto_http.c 2016-05-10 06:42:00.000000000 -0700 +++ >> haproxy-1.6.5/src/proto_http.c 2016-07-21 14:19:10.131437264 -0700 @@ >> -6926,6 +6926,7 >> @@ s->logs.bytes_out = txn->rsp.eoh; s->do_log(s); s->logs.bytes_out = 0; + >> s->logs.logwait |= LW_BYTES; } return 1; } >> >> Questions: 1. Is this correct? Are there unforeseen consequences of this >> change? > > No, no bad consequences, it will work for you in this situation. > >> 2. Would this functionality be welcomed upstream? If so, I can look into >> creating a >> separate option for this. > > We found that there are various situations where people would like to have > different places > to produce logs. The most common demands I've collected over the years were : > - after > request is full received - after server starts to respond - after server > finishes. > > I had an experimental patch adding up to 6 locations (more or less similar to > yours above). > But I found that a side effect of this was that logformat doesn't play nicely > with it since > some elements are not necessarily present. So either we stop emitting > warnings for bad log > formats (I'd prefer not), or better, we plan on having different log formats > for each step, > which requires different configuration settings. >
For those setups which forward logs to ElasticSearch/druid/similar products it may mess up dashboards as information for single request is spread across multiple log lines. For the past 3 weeks I have been working on 'dashboarding' HAProxy logs using Graphite and ElasticSearch (soon with druid). I combine time series and event based information, thus I have a lot of data available to me. I am now in the process to go beyond simply measuring num of req/secs and apply 'behavioral analytics'. The limiting factor for doing this is *not* HAProxy logs but how Graphite and ElasticSearch work/store data. Druid should help on this, I hope... What is 'behavioral analytics' in the context of HAProxy logs? Let's say you enable support for ECC and RSA SSL certs, you know by fact the less CPU cycles will be utilized. But, what else is effected by that, either positively or negatively. Right now, you can't easily answer this question. Why do you care to know about it ? For multiple reasons: #1 You want to measure ROI of the change #2 You want to reduce the 'surprise' factor. That is, a change occurred in area that you didn't think of. To sum up, HAProxy logs are very reach in information and any effort to make them to provide more information will be very much appreciated. > In addition, some people would like to be able to emit a log from an > http-request rule, > which also comes with the idea of a separate log format description. > +1 as it very handy during debug sessions. > And the last point is that we definitely need to be able to log the step > we're in to > distinguish the various log lines. But it could simply be a log format tag so > that users > place it where they want. I am doing that to separate logs between TCP and HTTP frontends: mode tcp option tcplog log-tag haproxy-tcp and I have template on rsyslogd to pass fields to ElasticSearch based on the mode. Adding another tag is an easy win. Cheers, Pavlos
signature.asc
Description: OpenPGP digital signature
