Thanks for the guidance and quick response. I do not think what you are proposing is overkill, a generalized solution for logging in N places, with N custom formats seems ideal. Using a tag to identify the source of a log line sounds sufficient. Being able to specify custom log formats using aliases, and additionally being able to concatenate/extend aliases (e.g. "log_source_tag" + base_log_format_alias + "more tags") would certainly make configuration a little more pleasant.
Unfortunately I can't currently commit the time to a change of this magnitude. If someone added it, we'd absolutely use it. On Fri, Jul 22, 2016 at 2:53 AM, Willy Tarreau <[email protected]> wrote: > Hi Cyrus, > > On Thu, Jul 21, 2016 at 11:22:06PM -0700, Cyrus Katrak wrote: > > Greetings from Slack Technologies, > > > > We are evaluating HAProxy as a load balancing solution and so far are > quite > > pleased with it. One feature we require is the ability to log when a > > connection started (option logasap), and again when it is terminated. We > > have many long lived websocket connections, and this level of > introspection > > is invaluable for debugging. > > > > We are currently accomplishing this with the following patch: > > diff -u -r haproxy-1.6.5-orig/src/proto_http.c > > haproxy-1.6.5/src/proto_http.c > > --- haproxy-1.6.5-orig/src/proto_http.c 2016-05-10 06:42:00.000000000 > -0700 > > +++ haproxy-1.6.5/src/proto_http.c 2016-07-21 14:19:10.131437264 > -0700 > > @@ -6926,6 +6926,7 @@ > > s->logs.bytes_out = txn->rsp.eoh; > > s->do_log(s); > > s->logs.bytes_out = 0; > > + s->logs.logwait |= LW_BYTES; > > } > > return 1; > > } > > > > Questions: > > 1. Is this correct? Are there unforeseen consequences of this change? > > No, no bad consequences, it will work for you in this situation. > > > 2. Would this functionality be welcomed upstream? If so, I can look into > > creating a separate option for this. > > We found that there are various situations where people would like to have > different places to produce logs. The most common demands I've collected > over the years were : > - after request is full received > - after server starts to respond > - after server finishes. > > I had an experimental patch adding up to 6 locations (more or less > similar to yours above). But I found that a side effect of this was > that logformat doesn't play nicely with it since some elements are > not necessarily present. So either we stop emitting warnings for bad > log formats (I'd prefer not), or better, we plan on having different > log formats for each step, which requires different configuration > settings. > > In addition, some people would like to be able to emit a log from an > http-request rule, which also comes with the idea of a separate log > format description. > > And the last point is that we definitely need to be able to log the > step we're in to distinguish the various log lines. But it could > simply be a log format tag so that users place it where they want. > > With all this in mind, I think it will be interesting to think the > solution in its entirety and *not* to implement just one part of it > as in your example above. > > One idea I had on the subject would be to provide the ability to declare > various log formats in advance for later reuse (because they're a real > pain to copy-paste everywhere), then to specify on the "log" lines a > list of the steps where we want to emit some logs. We would then have > a directive to indicate what alternate log format to use for what step. > > Maybe this is overkill and could be done differently, I'm all open ears. > > Willy > >
