subject:"Re\: \[PATCH\] Add log\-format variable %HQ, to log HTTP query strings"

[SPAM] Re: [PATCH] Add log-format variable %HQ, to log HTTP query strings

2015-10-01 Thread jimmy1974

 
Hey guy,

LEDlight International Industrial Limited here,exporting LED lights.There
are a lot of program code.web:www.ledlightmake.com

Call me,let's talk details.

Rgds,
jimmy

Re: [PATCH] Add log-format variable %HQ, to log HTTP query strings

2015-08-04 Thread Andrew Hayworth

On Mon, Aug 3, 2015 at 8:53 AM, Willy Tarreau w...@1wt.eu wrote:
 I agree, I found this a bit awkward as well :-)

 Regards,
 Willy


Hi all -

Thanks for the feedback. I moved the string-scanning bit, but did not
use the find_param_list function. Updated patch is attached.

-- 
- Andrew Hayworth

From 63b0af8420b22376dba3bfd8d367ff9a12261edf Mon Sep 17 00:00:00 2001
From: Andrew Hayworth andrew.haywo...@getbraintree.com
Date: Fri, 31 Jul 2015 16:14:16 +
Subject: [PATCH] Add log-format variable %HQ, to log HTTP query strings

Since sample fetches are not always available in the response phase,
this patch implements %HQ such that:

  GET /foo?bar=baz HTTP/1.0

...would be logged as:

  ?bar=baz
---
 doc/configuration.txt |  1 +
 include/types/log.h   |  1 +
 src/log.c | 36 
 3 files changed, 38 insertions(+)

diff --git a/doc/configuration.txt b/doc/configuration.txt
index db97cc7..b3ba8a0 100644
--- a/doc/configuration.txt
+++ b/doc/configuration.txt
@@ -13987,6 +13987,7 @@ Please refer to the table below for currently
defined variables :
   |   | %H   | hostname  | string  |
   | H | %HM  | HTTP method (ex: POST)| string  |
   | H | %HP  | HTTP request URI without query string (path)  | string  |
+  | H | %HQ  | HTTP request URI query string (ex: ?bar=baz)  | string  |
   | H | %HU  | HTTP request URI (ex: /foo?bar=baz)   | string  |
   | H | %HV  | HTTP version (ex: HTTP/1.0)   | string  |
   |   | %ID  | unique-id | string  |
diff --git a/include/types/log.h b/include/types/log.h
index bbfe020..d0fb966 100644
--- a/include/types/log.h
+++ b/include/types/log.h
@@ -96,6 +96,7 @@ enum {
  LOG_FMT_HTTP_METHOD,
  LOG_FMT_HTTP_URI,
  LOG_FMT_HTTP_PATH,
+ LOG_FMT_HTTP_QUERY,
  LOG_FMT_HTTP_VERSION,
  LOG_FMT_HOSTNAME,
  LOG_FMT_UNIQUEID,
diff --git a/src/log.c b/src/log.c
index ffd8f10..f80de2e 100644
--- a/src/log.c
+++ b/src/log.c
@@ -111,6 +111,7 @@ static const struct logformat_type logformat_keywords[] = {
  { hsl, LOG_FMT_HDRRESPONSLIST, PR_MODE_TCP, LW_RSPHDR, NULL },  /*
header response list */
  { HM, LOG_FMT_HTTP_METHOD, PR_MODE_HTTP, LW_REQ, NULL },  /* HTTP method */
  { HP, LOG_FMT_HTTP_PATH, PR_MODE_HTTP, LW_REQ, NULL },  /* HTTP path */
+ { HQ, LOG_FMT_HTTP_QUERY, PR_MODE_HTTP, LW_REQ, NULL },  /* HTTP query */
  { HU, LOG_FMT_HTTP_URI, PR_MODE_HTTP, LW_REQ, NULL },  /* HTTP full URI */
  { HV, LOG_FMT_HTTP_VERSION, PR_MODE_HTTP, LW_REQ, NULL },  /* HTTP
version */
  { lc, LOG_FMT_LOGCNT, PR_MODE_TCP, LW_INIT, NULL }, /* log counter */
@@ -937,6 +938,7 @@ int build_logline(struct stream *s, char *dst,
size_t maxsize, struct list *list
  struct chunk chunk;
  char *uri;
  char *spc;
+ char *qmark;
  char *end;
  struct tm tm;
  int t_request;
@@ -1578,6 +1580,40 @@ int build_logline(struct stream *s, char *dst,
size_t maxsize, struct list *list
last_isspace = 0;
break;

+ case LOG_FMT_HTTP_QUERY: // %HQ
+   if (tmp-options  LOG_OPT_QUOTE)
+ LOGCHAR('');
+
+   if (!txn-uri) {
+ chunk.str = BADREQ;
+ chunk.len = strlen(BADREQ);
+   } else {
+ uri = txn-uri;
+ end = uri + strlen(uri);
+ // look for the first question mark
+ while (uri  end  *uri != '?')
+   uri++;
+
+ qmark = uri;
+ // look for first space or question mark after url
+ while (uri  end  !HTTP_IS_SPHT(*uri))
+   uri++;
+
+ chunk.str = qmark;
+ chunk.len = uri - qmark;
+   }
+
+   ret = encode_chunk(tmplog, dst + maxsize, '#', url_encode_map, chunk);
+   if (ret == NULL || *ret != '\0')
+ goto out;
+
+   tmplog = ret;
+   if (tmp-options  LOG_OPT_QUOTE)
+ LOGCHAR('');
+
+   last_isspace = 0;
+   break;
+
  case LOG_FMT_HTTP_URI: // %HU
uri = txn-uri ? txn-uri : BADREQ;

--
2.1.3


0001-Add-log-format-variable-HQ-to-log-HTTP-query-strings.patch
Description: Binary data

Re: [PATCH] Add log-format variable %HQ, to log HTTP query strings

2015-08-03 Thread Willy Tarreau

Hi Aleks,

On Sun, Aug 02, 2015 at 06:55:26PM +0200, Aleksandar Lazic wrote:
 Hi Andrew.
 
 Am 31-07-2015 18:21, schrieb Andrew Hayworth:
 Since this came up in another thread, it seems reasonable to add a
 patch that implements %HQ as a log-format variable to record the HTTP
 query string. Leaving the initial '?' is intentional, but I don't feel
 strongly one way or another.
 
 How about to use the same function as haproxy?
 
 http://git.haproxy.org/?p=haproxy-1.5.git;a=blob;f=src/proto_http.c;h=5db64b5bf6b0c02dc8d501087bbb94ec320c2c43;hb=HEAD#l11004
 
 find_param_list()

Except that his function performs the lookup into the captured URI, which
is available even at the end of the transaction processing.

(...)
 I would also suggest to check
 
 
 if (!txn-uri) {
 
 
 before you go further due to the fact that a seach on BADREQ could 
 never match ;-)

I agree, I found this a bit awkward as well :-)

Regards,
Willy

Re: [PATCH] Add log-format variable %HQ, to log HTTP query strings

2015-08-02 Thread Aleksandar Lazic


Hi Andrew.

Am 31-07-2015 18:21, schrieb Andrew Hayworth:

Since this came up in another thread, it seems reasonable to add a
patch that implements %HQ as a log-format variable to record the HTTP
query string. Leaving the initial '?' is intentional, but I don't feel
strongly one way or another.


How about to use the same function as haproxy?

http://git.haproxy.org/?p=haproxy-1.5.git;a=blob;f=src/proto_http.c;h=5db64b5bf6b0c02dc8d501087bbb94ec320c2c43;hb=HEAD#l11004

find_param_list()


Which is used in find_url_param_value()

http://git.haproxy.org/?p=haproxy-1.5.git;a=blob;f=src/proto_http.c;h=5db64b5bf6b0c02dc8d501087bbb94ec320c2c43;hb=HEAD#l11053


qmark = find_param_list(uri , strlen(uri), '?');


I would also suggest to check


if (!txn-uri) {


before you go further due to the fact that a seach on BADREQ could 
never match ;-)


jm2c

Cheers Aleks


--
- Andrew Hayworth


From b87770d5e513fc923d0d94d2b1d0de00d88acb98 Mon Sep 17 00:00:00 2001
From: Andrew Hayworth andrew.haywo...@getbraintree.com
Date: Fri, 31 Jul 2015 16:14:16 +
Subject: [PATCH 1/1] Add log-format variable %HQ, to log HTTP query 
strings


Since sample fetches are not always available in the response phase,
this patch implements %HQ such that:

  GET /foo?bar=baz HTTP/1.0

...would be logged as:

  ?bar=baz
---
 doc/configuration.txt |  1 +
 include/types/log.h   |  1 +
 src/log.c | 38 ++
 3 files changed, 40 insertions(+)

diff --git a/doc/configuration.txt b/doc/configuration.txt
index db97cc7..b3ba8a0 100644
--- a/doc/configuration.txt
+++ b/doc/configuration.txt
@@ -13987,6 +13987,7 @@ Please refer to the table below for currently
defined variables :
   |   | %H   | hostname  | string  
|
   | H | %HM  | HTTP method (ex: POST)| string  
|
   | H | %HP  | HTTP request URI without query string (path)  | string  
|
+  | H | %HQ  | HTTP request URI query string (ex: ?bar=baz)  | string  
|
   | H | %HU  | HTTP request URI (ex: /foo?bar=baz)   | string  
|
   | H | %HV  | HTTP version (ex: HTTP/1.0)   | string  
|
   |   | %ID  | unique-id | string  
|

diff --git a/include/types/log.h b/include/types/log.h
index bbfe020..d0fb966 100644
--- a/include/types/log.h
+++ b/include/types/log.h
@@ -96,6 +96,7 @@ enum {
  LOG_FMT_HTTP_METHOD,
  LOG_FMT_HTTP_URI,
  LOG_FMT_HTTP_PATH,
+ LOG_FMT_HTTP_QUERY,
  LOG_FMT_HTTP_VERSION,
  LOG_FMT_HOSTNAME,
  LOG_FMT_UNIQUEID,
diff --git a/src/log.c b/src/log.c
index ffd8f10..1112f8a 100644
--- a/src/log.c
+++ b/src/log.c
@@ -111,6 +111,7 @@ static const struct logformat_type 
logformat_keywords[] = {

  { hsl, LOG_FMT_HDRRESPONSLIST, PR_MODE_TCP, LW_RSPHDR, NULL },  /*
header response list */
  { HM, LOG_FMT_HTTP_METHOD, PR_MODE_HTTP, LW_REQ, NULL },  /* HTTP 
method */
  { HP, LOG_FMT_HTTP_PATH, PR_MODE_HTTP, LW_REQ, NULL },  /* HTTP 
path */
+ { HQ, LOG_FMT_HTTP_QUERY, PR_MODE_HTTP, LW_REQ, NULL },  /* HTTP 
query */
  { HU, LOG_FMT_HTTP_URI, PR_MODE_HTTP, LW_REQ, NULL },  /* HTTP full 
URI */

  { HV, LOG_FMT_HTTP_VERSION, PR_MODE_HTTP, LW_REQ, NULL },  /* HTTP
version */
  { lc, LOG_FMT_LOGCNT, PR_MODE_TCP, LW_INIT, NULL }, /* log counter 
*/

@@ -937,6 +938,7 @@ int build_logline(struct stream *s, char *dst,
size_t maxsize, struct list *list
  struct chunk chunk;
  char *uri;
  char *spc;
+ char *qmark;
  char *end;
  struct tm tm;
  int t_request;
@@ -1578,6 +1580,42 @@ int build_logline(struct stream *s, char *dst,
size_t maxsize, struct list *list
last_isspace = 0;
break;

+ case LOG_FMT_HTTP_QUERY: // %HQ
+   uri = txn-uri ? txn-uri : BADREQ;
+
+   if (tmp-options  LOG_OPT_QUOTE)
+ LOGCHAR('');
+
+   end = uri + strlen(uri);
+   // look for the first question mark
+   while (uri  end  *uri != '?')
+ uri++;
+
+   qmark = uri;
+
+   // look for first space or question mark after url
+   while (uri  end  !HTTP_IS_SPHT(*uri))
+ uri++;
+
+   if (!txn-uri) {
+ chunk.str = BADREQ;
+ chunk.len = strlen(BADREQ);
+   } else {
+ chunk.str = qmark;
+ chunk.len = uri - qmark;
+   }
+
+   ret = encode_chunk(tmplog, dst + maxsize, '#', url_encode_map, 
chunk);

+   if (ret == NULL || *ret != '\0')
+ goto out;
+
+   tmplog = ret;
+   if (tmp-options  LOG_OPT_QUOTE)
+ LOGCHAR('');
+
+   last_isspace = 0;
+   break;
+
  case LOG_FMT_HTTP_URI: // %HU
uri = txn-uri ? txn-uri : BADREQ;

--
2.1.3

Re: [PATCH] Add log-format variable %HQ, to log HTTP query strings

2015-07-31 Thread Phillip Decker

Andrew-

This patch works well for my use case.  Leaving the question mark in
mirrors the behavior of %q in httpd, fwiw.

Should it also print a hyphen in case the field is empty and the quotation
mode is not on? %HQ instead of %{+Q}HQ?

Phillip


On Fri, Jul 31, 2015 at 12:21 PM, Andrew Hayworth 
andrew.haywo...@getbraintree.com wrote:

 Since this came up in another thread, it seems reasonable to add a
 patch that implements %HQ as a log-format variable to record the HTTP
 query string. Leaving the initial '?' is intentional, but I don't feel
 strongly one way or another.

 --
 - Andrew Hayworth


 From b87770d5e513fc923d0d94d2b1d0de00d88acb98 Mon Sep 17 00:00:00 2001
 From: Andrew Hayworth andrew.haywo...@getbraintree.com
 Date: Fri, 31 Jul 2015 16:14:16 +
 Subject: [PATCH 1/1] Add log-format variable %HQ, to log HTTP query strings

 Since sample fetches are not always available in the response phase,
 this patch implements %HQ such that:

   GET /foo?bar=baz HTTP/1.0

 ...would be logged as:

   ?bar=baz
 ---
  doc/configuration.txt |  1 +
  include/types/log.h   |  1 +
  src/log.c | 38 ++
  3 files changed, 40 insertions(+)

 diff --git a/doc/configuration.txt b/doc/configuration.txt
 index db97cc7..b3ba8a0 100644
 --- a/doc/configuration.txt
 +++ b/doc/configuration.txt
 @@ -13987,6 +13987,7 @@ Please refer to the table below for currently
 defined variables :
|   | %H   | hostname  | string
   |
| H | %HM  | HTTP method (ex: POST)| string
   |
| H | %HP  | HTTP request URI without query string (path)  | string
   |
 +  | H | %HQ  | HTTP request URI query string (ex: ?bar=baz)  | string
   |
| H | %HU  | HTTP request URI (ex: /foo?bar=baz)   | string
   |
| H | %HV  | HTTP version (ex: HTTP/1.0)   | string
   |
|   | %ID  | unique-id | string
   |
 diff --git a/include/types/log.h b/include/types/log.h
 index bbfe020..d0fb966 100644
 --- a/include/types/log.h
 +++ b/include/types/log.h
 @@ -96,6 +96,7 @@ enum {
   LOG_FMT_HTTP_METHOD,
   LOG_FMT_HTTP_URI,
   LOG_FMT_HTTP_PATH,
 + LOG_FMT_HTTP_QUERY,
   LOG_FMT_HTTP_VERSION,
   LOG_FMT_HOSTNAME,
   LOG_FMT_UNIQUEID,
 diff --git a/src/log.c b/src/log.c
 index ffd8f10..1112f8a 100644
 --- a/src/log.c
 +++ b/src/log.c
 @@ -111,6 +111,7 @@ static const struct logformat_type
 logformat_keywords[] = {
   { hsl, LOG_FMT_HDRRESPONSLIST, PR_MODE_TCP, LW_RSPHDR, NULL },  /*
 header response list */
   { HM, LOG_FMT_HTTP_METHOD, PR_MODE_HTTP, LW_REQ, NULL },  /* HTTP
 method */
   { HP, LOG_FMT_HTTP_PATH, PR_MODE_HTTP, LW_REQ, NULL },  /* HTTP path */
 + { HQ, LOG_FMT_HTTP_QUERY, PR_MODE_HTTP, LW_REQ, NULL },  /* HTTP query
 */
   { HU, LOG_FMT_HTTP_URI, PR_MODE_HTTP, LW_REQ, NULL },  /* HTTP full
 URI */
   { HV, LOG_FMT_HTTP_VERSION, PR_MODE_HTTP, LW_REQ, NULL },  /* HTTP
 version */
   { lc, LOG_FMT_LOGCNT, PR_MODE_TCP, LW_INIT, NULL }, /* log counter */
 @@ -937,6 +938,7 @@ int build_logline(struct stream *s, char *dst,
 size_t maxsize, struct list *list
   struct chunk chunk;
   char *uri;
   char *spc;
 + char *qmark;
   char *end;
   struct tm tm;
   int t_request;
 @@ -1578,6 +1580,42 @@ int build_logline(struct stream *s, char *dst,
 size_t maxsize, struct list *list
 last_isspace = 0;
 break;

 + case LOG_FMT_HTTP_QUERY: // %HQ
 +   uri = txn-uri ? txn-uri : BADREQ;
 +
 +   if (tmp-options  LOG_OPT_QUOTE)
 + LOGCHAR('');
 +
 +   end = uri + strlen(uri);
 +   // look for the first question mark
 +   while (uri  end  *uri != '?')
 + uri++;
 +
 +   qmark = uri;
 +
 +   // look for first space or question mark after url
 +   while (uri  end  !HTTP_IS_SPHT(*uri))
 + uri++;
 +
 +   if (!txn-uri) {
 + chunk.str = BADREQ;
 + chunk.len = strlen(BADREQ);
 +   } else {
 + chunk.str = qmark;
 + chunk.len = uri - qmark;
 +   }
 +
 +   ret = encode_chunk(tmplog, dst + maxsize, '#', url_encode_map,
 chunk);
 +   if (ret == NULL || *ret != '\0')
 + goto out;
 +
 +   tmplog = ret;
 +   if (tmp-options  LOG_OPT_QUOTE)
 + LOGCHAR('');
 +
 +   last_isspace = 0;
 +   break;
 +
   case LOG_FMT_HTTP_URI: // %HU
 uri = txn-uri ? txn-uri : BADREQ;

 --
 2.1.3

[SPAM] Re: [PATCH] Add log-format variable %HQ, to log HTTP query strings

Re: [PATCH] Add log-format variable %HQ, to log HTTP query strings

Re: [PATCH] Add log-format variable %HQ, to log HTTP query strings

Re: [PATCH] Add log-format variable %HQ, to log HTTP query strings

Re: [PATCH] Add log-format variable %HQ, to log HTTP query strings

5 matches

Site Navigation

Mail list logo

Footer information