Re: Capture sequencing in logs
Nenad, thank you so much for your help; I was able to run that successfully
on a test instance, I think that may fix the majority of my issue!
Thanks again,
Phillip
On Mon, Jul 6, 2015 at 4:55 PM, Nenad Merdanovic wrote:
> Hello Phillip,
>
> First, let me apologize for misspelling your name the first time.
>
> On 7/6/2015 10:46 PM, Phillip Decker wrote:
> > Hi Nenad, thanks for the reply!
> >
> > Okay, insofar as the first link - I can see where I can retrieve
> > captures for use in acls or elsewhere, but is there a way to use these
> > directly in the custom log format (
> > http://cbonte.github.io/haproxy-dconv/configuration-1.5.html#8.2.4 )?
> > Being able to intersperse captures and the usual custom log format
> > entries on the same log line is my main problem.
> >
> > Yes, I was excited to use the new declared captures, but it came down to
> > the same question as above; is there a way to use a declared capture in
> > a custom log format line? If I can, it solves my problems...
> >
>
> http://cbonte.github.io/haproxy-dconv/configuration-1.5.html#8.2.4
>
> As you can see here:
> If a variable is named between square brackets ('[' .. ']') then it is
> used as a sample expression rule (see section 7.3). This it useful to
> add some less common information such as the client's SSL certificate's
> DN, or to log the key that would be used to store an entry into a stick
> table.
>
> So essentially what you have to do is add something like
> %[capture.req.hdr(0)] to your log-format string. And then just change
> the 0 to whatever capture you want depending on the capture ordering.
>
> Hope this helps.
>
> Nenda
>
> > In log.c I see that %hr is doing a for loop on the session's front end
> > proxy struct's list of capture fields, but doesn't look like it can take
> > any arguments or anything for declared captures. Looking through the
> > diffs on that file back two years, I see a reference in April to moving
> > the captures from http_txn to stream, but a look at the diffs show that
> > doesn't change this issue.
> >
> > I'm looking through
> > log.c : void parse_logformat_string(const char *fmt, struct
> > proxy *curproxy, struct list *list_format, int options, int cap, const
> > char *file, int line)
> > for an indication of a way to define this, and I don't see it so far...
> > am I missing it?
> >
> > Thanks again for any help!
> >
> > Phillip
> >
> >
> >
> > On Mon, Jul 6, 2015 at 4:10 PM, Nenad Merdanovic > > wrote:
> >
> > Hello Philip
> >
> > On 7/6/2015 9:47 PM, Phillip Decker wrote:
> > > Hello again all,
> > >
> > > I have a configuration problem, and I have a workaround that I
> don't
> > > like, so I'm hoping someone here might have a better solution -
> > >
> > > I have a number of capture fields such as ssl_c_s_dn, request
> header
> > > referrer, request header X-Forwarded-For, etc that get logged
> using %hr,
> > > among another 12 or so fields such as %t, %H, %B, %ci and so on.
> > >
> > > The problem is the client has an established log format which goes
> to an
> > > enterprise centralized log db. I'd love to be able to write out
> this
> > > format directly with haproxy, but since all the captured fields in
> > > haproxy get appended together by %hr and %hs, I don't know of a
> way to
> > > put one captured field in spot #4 of a log line, and then another
> > > captured field in spot #7, etc.
> > >
> > > In order to work around this, I have haproxy instead outputting
> all this
> > > data to an "intermediate" log format, and then have another process
> > > running which picks these up and rearranges them into the
> arbitrary log
> > > format that the enterprise accepts.
> >
> > Well, it seems like you are looking to reorder the captures, so take
> a
> > look at:
> >
> http://cbonte.github.io/haproxy-dconv/configuration-1.5.html#7.3.6-capture.req.hdr
> >
> > You can use this in your log format to specify a different ordering
> than
> > the one how the captures are sorted in the config.
> >
> > In 1.6 you could also use:
> >
> http://cbonte.github.io/haproxy-dconv/configuration-1.6.html#declare%20capture
> >
> > Let me know if this helps.
> >
> > Regards,
> > Nenad
> >
> >
>
Air Freight China
2015-7-7 5:09 haproxy Air freight quotation from china: qu...@worldfreightforwarding.com
Re: Capture sequencing in logs
Hello Phillip,
First, let me apologize for misspelling your name the first time.
On 7/6/2015 10:46 PM, Phillip Decker wrote:
> Hi Nenad, thanks for the reply!
>
> Okay, insofar as the first link - I can see where I can retrieve
> captures for use in acls or elsewhere, but is there a way to use these
> directly in the custom log format (
> http://cbonte.github.io/haproxy-dconv/configuration-1.5.html#8.2.4 )?
> Being able to intersperse captures and the usual custom log format
> entries on the same log line is my main problem.
>
> Yes, I was excited to use the new declared captures, but it came down to
> the same question as above; is there a way to use a declared capture in
> a custom log format line? If I can, it solves my problems...
>
http://cbonte.github.io/haproxy-dconv/configuration-1.5.html#8.2.4
As you can see here:
If a variable is named between square brackets ('[' .. ']') then it is
used as a sample expression rule (see section 7.3). This it useful to
add some less common information such as the client's SSL certificate's
DN, or to log the key that would be used to store an entry into a stick
table.
So essentially what you have to do is add something like
%[capture.req.hdr(0)] to your log-format string. And then just change
the 0 to whatever capture you want depending on the capture ordering.
Hope this helps.
Nenda
> In log.c I see that %hr is doing a for loop on the session's front end
> proxy struct's list of capture fields, but doesn't look like it can take
> any arguments or anything for declared captures. Looking through the
> diffs on that file back two years, I see a reference in April to moving
> the captures from http_txn to stream, but a look at the diffs show that
> doesn't change this issue.
>
> I'm looking through
> log.c : void parse_logformat_string(const char *fmt, struct
> proxy *curproxy, struct list *list_format, int options, int cap, const
> char *file, int line)
> for an indication of a way to define this, and I don't see it so far...
> am I missing it?
>
> Thanks again for any help!
>
> Phillip
>
>
>
> On Mon, Jul 6, 2015 at 4:10 PM, Nenad Merdanovic > wrote:
>
> Hello Philip
>
> On 7/6/2015 9:47 PM, Phillip Decker wrote:
> > Hello again all,
> >
> > I have a configuration problem, and I have a workaround that I don't
> > like, so I'm hoping someone here might have a better solution -
> >
> > I have a number of capture fields such as ssl_c_s_dn, request header
> > referrer, request header X-Forwarded-For, etc that get logged using %hr,
> > among another 12 or so fields such as %t, %H, %B, %ci and so on.
> >
> > The problem is the client has an established log format which goes to an
> > enterprise centralized log db. I'd love to be able to write out this
> > format directly with haproxy, but since all the captured fields in
> > haproxy get appended together by %hr and %hs, I don't know of a way to
> > put one captured field in spot #4 of a log line, and then another
> > captured field in spot #7, etc.
> >
> > In order to work around this, I have haproxy instead outputting all this
> > data to an "intermediate" log format, and then have another process
> > running which picks these up and rearranges them into the arbitrary log
> > format that the enterprise accepts.
>
> Well, it seems like you are looking to reorder the captures, so take a
> look at:
>
> http://cbonte.github.io/haproxy-dconv/configuration-1.5.html#7.3.6-capture.req.hdr
>
> You can use this in your log format to specify a different ordering than
> the one how the captures are sorted in the config.
>
> In 1.6 you could also use:
>
> http://cbonte.github.io/haproxy-dconv/configuration-1.6.html#declare%20capture
>
> Let me know if this helps.
>
> Regards,
> Nenad
>
>
Re: Capture sequencing in logs
Hi Nenad, thanks for the reply! Okay, insofar as the first link - I can see where I can retrieve captures for use in acls or elsewhere, but is there a way to use these directly in the custom log format ( http://cbonte.github.io/haproxy-dconv/configuration-1.5.html#8.2.4 )? Being able to intersperse captures and the usual custom log format entries on the same log line is my main problem. Yes, I was excited to use the new declared captures, but it came down to the same question as above; is there a way to use a declared capture in a custom log format line? If I can, it solves my problems... In log.c I see that %hr is doing a for loop on the session's front end proxy struct's list of capture fields, but doesn't look like it can take any arguments or anything for declared captures. Looking through the diffs on that file back two years, I see a reference in April to moving the captures from http_txn to stream, but a look at the diffs show that doesn't change this issue. I'm looking through log.c : void parse_logformat_string(const char *fmt, struct proxy *curproxy, struct list *list_format, int options, int cap, const char *file, int line) for an indication of a way to define this, and I don't see it so far... am I missing it? Thanks again for any help! Phillip On Mon, Jul 6, 2015 at 4:10 PM, Nenad Merdanovic wrote: > Hello Philip > > On 7/6/2015 9:47 PM, Phillip Decker wrote: > > Hello again all, > > > > I have a configuration problem, and I have a workaround that I don't > > like, so I'm hoping someone here might have a better solution - > > > > I have a number of capture fields such as ssl_c_s_dn, request header > > referrer, request header X-Forwarded-For, etc that get logged using %hr, > > among another 12 or so fields such as %t, %H, %B, %ci and so on. > > > > The problem is the client has an established log format which goes to an > > enterprise centralized log db. I'd love to be able to write out this > > format directly with haproxy, but since all the captured fields in > > haproxy get appended together by %hr and %hs, I don't know of a way to > > put one captured field in spot #4 of a log line, and then another > > captured field in spot #7, etc. > > > > In order to work around this, I have haproxy instead outputting all this > > data to an "intermediate" log format, and then have another process > > running which picks these up and rearranges them into the arbitrary log > > format that the enterprise accepts. > > Well, it seems like you are looking to reorder the captures, so take a > look at: > > http://cbonte.github.io/haproxy-dconv/configuration-1.5.html#7.3.6-capture.req.hdr > > You can use this in your log format to specify a different ordering than > the one how the captures are sorted in the config. > > In 1.6 you could also use: > > http://cbonte.github.io/haproxy-dconv/configuration-1.6.html#declare%20capture > > Let me know if this helps. > > Regards, > Nenad >
Re: Capture sequencing in logs
Hello Philip On 7/6/2015 9:47 PM, Phillip Decker wrote: > Hello again all, > > I have a configuration problem, and I have a workaround that I don't > like, so I'm hoping someone here might have a better solution - > > I have a number of capture fields such as ssl_c_s_dn, request header > referrer, request header X-Forwarded-For, etc that get logged using %hr, > among another 12 or so fields such as %t, %H, %B, %ci and so on. > > The problem is the client has an established log format which goes to an > enterprise centralized log db. I'd love to be able to write out this > format directly with haproxy, but since all the captured fields in > haproxy get appended together by %hr and %hs, I don't know of a way to > put one captured field in spot #4 of a log line, and then another > captured field in spot #7, etc. > > In order to work around this, I have haproxy instead outputting all this > data to an "intermediate" log format, and then have another process > running which picks these up and rearranges them into the arbitrary log > format that the enterprise accepts. Well, it seems like you are looking to reorder the captures, so take a look at: http://cbonte.github.io/haproxy-dconv/configuration-1.5.html#7.3.6-capture.req.hdr You can use this in your log format to specify a different ordering than the one how the captures are sorted in the config. In 1.6 you could also use: http://cbonte.github.io/haproxy-dconv/configuration-1.6.html#declare%20capture Let me know if this helps. Regards, Nenad
Capture sequencing in logs
Hello again all, I have a configuration problem, and I have a workaround that I don't like, so I'm hoping someone here might have a better solution - I have a number of capture fields such as ssl_c_s_dn, request header referrer, request header X-Forwarded-For, etc that get logged using %hr, among another 12 or so fields such as %t, %H, %B, %ci and so on. The problem is the client has an established log format which goes to an enterprise centralized log db. I'd love to be able to write out this format directly with haproxy, but since all the captured fields in haproxy get appended together by %hr and %hs, I don't know of a way to put one captured field in spot #4 of a log line, and then another captured field in spot #7, etc. In order to work around this, I have haproxy instead outputting all this data to an "intermediate" log format, and then have another process running which picks these up and rearranges them into the arbitrary log format that the enterprise accepts. I'd love to get rid of having to maintain this separate process long term. When I first saw the discussion of defined captures, I thought maybe there'd be a way to use those definitions in the log format directly, so you could have a log line that was in a certain sequence where the captures were mixed in: time, client ip, client subject dn, HTTP request uri, http status code, X-Forwarded-For IP, bytes transferred, etc (not the actual format, but you get the point). However, I don't see a way to do that. Then I saw the LUA logging calls, and so I was reading the API, trying to figure out if I could write the whole log statement using LUA, but without having tried it yet, is it a) possible to do so for every request, and b ) is there a performance penalty for doing so? Is there another way to handle this? How have other people dealt with this problem? Thanks! Phillip PS - That was an amazing find last week on the pipelining bug; thanks for your hard work! It's appreciated.
Re: Rewrite cookie path & cookie domain
Dear rickytato rickytato. Am 06-07-2015 15:32, schrieb rickytato rickytato: Hi all, I've problem to rewrite cookie path and cookie domain in HAproxy; I've a Nginx configuration but I want to move from Nginx to HAProxy for this proxy pass. Which Version of haproxy do you use? haproxy -vv ? Cheers Aleks
simple acl combination query
hi, simple question about acl's i know i can combine them in monitors eg: acl acl1 nbsrv(mysql) eq 0 acl acl2 nbsrv(mssql) eq 0 monitor-uri /dbs for and: monitor fail if acl1 acl2 for or: monitor fail if acl1 || acl2 what i'm not sure about, is for more complicated logical combinations, can multiple and's and or's be combined, eg: monitor fail unless (acl1 acl2 acl3) || (acl4 (acl5 || acl6)) || (acl1 || acl5) thanks lee.
Rewrite cookie path & cookie domain
Hi all,
I've problem to rewrite cookie path and cookie domain in HAproxy; I've a
Nginx configuration but I want to move from Nginx to HAProxy for this proxy
pass.
This is a Nginx config I want to replace:
location /~xxx/ {
proxy_cookie_domain ~.* .$site.it;
proxy_cookie_path ~.* /~xxx/;
proxy_set_headerHost $site.it;
proxy_pass http://192.168.1.2/;
}
I need same function of proxy_cookie_domain and proxy_cookie_path; I found
this:
http://blog.haproxy.com/2014/04/28/howto-write-apache-proxypass-rules-in-haproxy/
but not work form me.
Now I can change cookie path with:
rspirep ^(Set-Cookie:.*)\ path=(.*) \1\ path=/~xxx/
I need add also domain, only if exists, but with dynamic hostname; I;ve
tried with
acl hdr_set_cookie_domain_and_path res.hdr(Set-cookie) -m sub domain=
res.hdr(Set-cookie) -m sub path=
rspirep ^(Set-Cookie:.*)\ path=(.*) \1\ path=/~xxx/;\ domain=%[hdr(Host)]
if hdr_set_cookie_domain_and_path
But not work.
Anyone can help me?
Tnx,
rr
need new business leads?
Hey, Is business slow? We can help We specialize in providing target email marketing services to a number of businesses all over the world! We would like to be your marketing partners and help your business reach the new level. Let's increase your business sales 2-4 times than now. Please contact us today so I can go over options for you. Thanks, Peter Strong Contact: vanc...@sina.com
[SPAM] Why should you choose the right book printer
Dear Purchasing Manager, Are you still suffering from a lack of good printing experience? Are you feeling printing products not reaching their desired goal, or is the high price? All right, why don't we look farther? You just need a right book printer. In China, Hangzhou, there is a professional printing group so that we can provide better service. Various hardcover books, softcover books, magazines, brochures and gift boxes, etc. are achievable in here. Why do we have to spend big price to buy advanced machinery and technology? We know the value of them. Minimal human error and lower overhead cost, which adds value to you with consistently high-quality printing at favorable prices. We believe that you will make a wise choice. Best Regards Peter
