Hi Willie,
Thanks for your note. Yes, I remember you mentioning that you had plans to do
this at some point. I just didn't realize it was included in 1.5. The
flexible log format is nice too. I had hard-wired a format that made sense for
us, but your way lets people pick whatever makes sense for them.
Now that the official haproxy code has this, there's really no point in me
continuing to maintain our version. It was nice to get to write some C code
again, however. These days, I mostly live in Python. It's good to get close
to the metal once in a while, so you don't forget how computers really work.
I totally agree about keeping the clocks in sync. And, I should add, running
everything in UTC. I never again want to try to remember if I'm +4 or +5, or
deal with the fact that we rolled over from summer time to winter time in the
middle of a log file :-)
We're really looking forward to the SSL support in 1.5. We currently terminate
our SSL sessions with nginx (in addition to also using nginx as our application
front-end), so our data flow looks like nginx -> haproxy -> nginx. It'll be
nice to shorten that chain.
Have you experienced any issues running your SSL code on Ubuntu precise? We
tried to upgrade our SSL hosts from lucid to precise and started getting
intermittent errors with users authenticating to our application. We were
never able to figure out what was going on. We eventually solved the problem
by retreating back to lucid. We don't know if it's an nginx issue, or a
problem with the underlying SSL library. I'm curious if you've seen anything
similar?
On Dec 11, 2012, at 5:54 PM, Willy Tarreau wrote:
> Hi Roy,
>
> On Tue, Dec 11, 2012 at 05:08:17PM -0500, Roy Smith wrote:
>> Hmmm, yeah, this looks like exactly what my patch does. Cool.
>
> Indeed, now I remember this old discussion where we were different ones
> having different requirements. As I said at that time, we (Exceliance)
> had plans for developing this. After we had redesigned the log format,
> it was quite easy to extend the feature to define an HTTP header format.
>
> That way anyone can define the format he needs. It is very possible
> that some of the things you're currently doing are not possible with
> the current format (I have no idea what), but you're really invited
> to extend it if something is missing.
>
> Now to respond your initial questions :
>
>>>> I've just finished patching haproxy-1.5-dev14 to use my unique-id header
>>>> code, which we discussed a year or so ago. There's one thing I'm not 100%
>>>> sure about. You changed the interface of http_find_header2().
>>>> Previously, the third argument was an int, now it's a char*. I'm assuming
>>>> that where I used to do:
>>>>
>>>> + if (! http_find_header2(hdr, hdr_len, msg->sol,
>>>> &txn->hdr_idx, &ctx)) {
>>>>
>>>> I now want
>>>>
>>>> + if (! http_find_header2(hdr, hdr_len, msg->chn->buf->p +
>>>> msg->sol, &txn->hdr_idx, &ctx)) {
>>>>
>>>> yes? This is in proto_http.c, http_wait_for_request().
>
> From what I'm seeing, no, you just need msg->chn->buf->p, which is the
> pointer to the beginning of current message in the request buffer. Please
> have a look to doc/internal/buffer-ops.fig to get a better idea. The
> "entities.fig" file in the same directory might also be of interest to you.
>
>>>> Have you given any more thought to picking up this patch for the official
>>>> version? We've been running it in production for well over a year with no
>>>> problems. Yesterdays' traffic volume was 73 million HTTP requests, so I
>>>> think we've given it a fair workout. The feature has proven invaluable
>>>> for debugging problems in the site, as it lets us easily correlate lines
>>>> in our haproxy, nginx, and application log files. I'm sure many other
>>>> haproxy users would find it equally useful.
>
> I'm totally convinced by the usefulness of such a feature. I've insisted
> that we put this in our Aloha roadmap years ago because of this. But only
> people who have previously used such a feature in production know its value.
> Once you have this, you can completely rethink the log workflow, processing
> and analysis. And the second best thing to have is perfectly synchronized
> clocks on all your servers :-)
>
> Best regards!
> Willy
>
---
Roy Smith
[email protected]
