Re: Logging request parameters - Filter vs Servlet
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Leon, On 3/30/2011 3:36 PM, Leo Donahue - PLANDEVX wrote: I'm going this route for now: FacesContext.getCurrentInstance().getExternalContext().log(the string buffer of the parcel listbox); FacesContext.getCurrentInstance().getExternalContext().log(the map URL); Might I suggest that you try to cram all that information onto a single line and replace any CR or LF characters with a space or something? If you want to post-process the log, having each even on a single line will be sooo much easier than writing a script just to piece everything back together again. Also, in high-volume situations, you may get logs from one request interleaved with those of another, like this: Mar 30, 2011 12:02:35 PM org.apache.catalina.core.ApplicationContext log INFO: PARCEL = '125-27-089' Mar 30, 2011 12:02:35 PM org.apache.catalina.core.ApplicationContext log INFO: PARCEL = '456-67-890' Mar 30, 2011 12:02:45 PM org.apache.catalina.core.ApplicationContext log INFO: http://planning.maricopa.gov/agsoutput/_ags_map08eb57df58224e1884e17a3e8a59b555.jpg Mar 30, 2011 12:02:45 PM org.apache.catalina.core.ApplicationContext log INFO: http://planning.maricopa.gov/agsoutput/_ags_map738249178234b78b09213aba07f78923.jpg Good luck, - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk2U2bcACgkQ9CaO5/Lv0PAvAwCfdiGr2HewlXGxzB05uvQwnIj6 8SAAn3IeKzFMBlwmPnDejpOs1buN87tS =QUb7 -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Logging request parameters - Filter vs Servlet
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Leo, On 3/29/2011 5:44 PM, Leo Donahue - PLANDEVX wrote: -Original Message- From: Christopher Schultz [mailto:ch...@christopherschultz.net] Subject: Re: Logging request parameters - Filter vs Servlet On 3/29/2011 12:57 PM, Leo Donahue - PLANDEVX wrote: Where do you initialize the Logger (Filter or Servlet)? Which logger? I don't know what I don't know... Log4j I guess. That is what everyone recommends. Hmm... The servlet 2.5 spec says you can use filters for logging, but since I'm not modifying the request or response, is logging from within a filter the right approach to logging request parameters? That depends upon your requirements. What are they? I just want to capture the request parameters on a certain web app and log them (time and what they were) in a separate log file from the standard logs, so that I don't have to hunt them down in the standard Tomcat logs. The security tool that our telecom office uses for auditing our sites makes quite a mess of my standard logs periodically. There is an AccessLogValve that you can use if you know which request parameters you want to log. There is also a RequestDumperValve (and RequestDumperFilter in 7.0) that you can use to dump everything from the request. See the docs for details. Would those work for you? Note that reading request parameters in a Filter may trigger parsing of a POST request body which may not be something you want to happen on every request. An example of things I don't know that I don't know... Heh. I suppose you'll know if things stop working. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk2TRmcACgkQ9CaO5/Lv0PAIBQCfeDY2N9qXHbuijqz5cIOSnkj8 POgAn0ghVWuAkkiEv3cTrOTWsyCRe4IB =c7cf -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: Logging request parameters - Filter vs Servlet
Chris, -Original Message- From: Christopher Schultz [mailto:ch...@christopherschultz.net] Subject: Re: Logging request parameters - Filter vs Servlet I don't know what I don't know... Log4j I guess. That is what everyone recommends. Hmm... ServletContext logging then? There is an AccessLogValve that you can use if you know which request parameters you want to log. There is also a RequestDumperValve (and RequestDumperFilter in 7.0) that you can use to dump everything from the request. See the docs for details. Would those work for you? The AccessLog pattern I'm using: pattern=%h %l %u %t %r %q %s %b The results: 2.3.4.5 - - [29/Mar/2011:16:02:39 -0700] POST /oppositioncase/oppositioncase.faces HTTP/1.1 200 38621 My web app uses the JSF framework, forgot to mention that. http://planning.maricopa.gov/oppositioncase samples: 211-52-002A 211-74-016 211-53-005C Those are the parameters I'm trying to capture, along with the map image url. A. We'd like to know how many requests actually generated a map image. B. We'd like to know whether this app is searching for parcels primarily in the unincorporated areas of the County, or parcels located within a city jurisdiction. That part I can figure out once I know which parcels people are searching. Note that reading request parameters in a Filter may trigger parsing of a POST request body which may not be something you want to happen on every request. I guess my only option then is to log them from the web app. But somewhere I've read that is the wrong/lazy way to do logging.
Re: Logging request parameters - Filter vs Servlet
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Leo, On 3/30/2011 12:28 PM, Leo Donahue - PLANDEVX wrote: -Original Message- From: Christopher Schultz [mailto:ch...@christopherschultz.net] Subject: Re: Logging request parameters - Filter vs Servlet I don't know what I don't know... Log4j I guess. That is what everyone recommends. Hmm... ServletContext logging then? The only logging the ServletContext will do is if you explicitly call ServletContext.log(...) which Tomcat will direct to the appropriate logger (set up using logging.properties or, if you go through the configuration, log4j). There is an AccessLogValve that you can use if you know which request parameters you want to log. There is also a RequestDumperValve (and RequestDumperFilter in 7.0) that you can use to dump everything from the request. See the docs for details. Would those work for you? The AccessLog pattern I'm using: pattern=%h %l %u %t %r %q %s %b The results: 2.3.4.5 - - [29/Mar/2011:16:02:39 -0700] POST /oppositioncase/oppositioncase.faces HTTP/1.1 200 38621 Hmm... sorry to have steered you astray: looking at the documentation, AccessLogValve only allows you to log request headers and attributes, not parameters. That's a shame... My web app uses the JSF framework, forgot to mention that. http://planning.maricopa.gov/oppositioncase samples: 211-52-002A 211-74-016 211-53-005C So you need to log the txtAPN parameter, right? Those are the parameters I'm trying to capture, along with the map image url. A. We'd like to know how many requests actually generated a map image. Can you tell that, just from a parameter value? I would think that the URL itself would more likely tell you if a map was generated. B. We'd like to know whether this app is searching for parcels primarily in the unincorporated areas of the County, or parcels located within a city jurisdiction. That part I can figure out once I know which parcels people are searching. Simply logging the parcels used in searches would allow you to do that, as you've said. The RequestDumperValve logs a /ton/ of information, and probably wouldn't get you what you want. Note that reading request parameters in a Filter may trigger parsing of a POST request body which may not be something you want to happen on every request. I guess my only option then is to log them from the web app. But somewhere I've read that is the wrong/lazy way to do logging. I'm not sure where you read it, but that sounds like a platitude applied as a blanket admonition not to log in your webapp. If it's appropriate for your situation, then feel free to do it. Back to the original filter question: logging using a filter is perfectly acceptable if it's the right solution (see below for questions that might lead you to other options). Writing the filter is trivial: just implement the javax.servlet.Filter interface and be sure to read the javadoc for it before you try: you'll thank yourself, later. You can even use ServletContext.log() to write to the context log if you want. Otherwise, feel free to use your webapp's log4j or other logging facility (you'll have to configure this yourself). Just don't use System.out :) One might argue that blindly logging request parameters is not particularly useful. For instance, a quick look at your interface indicates that you can add several plots of land before performing the search. Those initial add operations may be of little use to you. Instead, you may wish to log them only at a certain point in your workflow. Since most requests go to oppositioncase.faces, you may not be able to map your Filter to a URL pattern that is fine-grained enough. Instead, it might make more sense to log this data when you know there's a reason to log it. Since you're expecting to perform statistical analysis on the data, you might even consider writing it directly to a database instead of to a plain-old log file. Cool webapp, by the way! NB: The ctrl key isn't always the modifier key to use to de-select items from a multi-select list. I think that's a Microsoft Windows convention, but it's CMD-click on Mac and probably something like META-click on *NIX. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk2TeiYACgkQ9CaO5/Lv0PC3BgCcDrR863az0BctJcOI3gGcjD0J kpcAn3WMLt7MseeErBLAADS6Gh+UNTbS =LZdC -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: Logging request parameters - Filter vs Servlet
Chris, Thanks for the feedback.. very much appreciated! -Original Message- From: Christopher Schultz [mailto:ch...@christopherschultz.net] Subject: Re: Logging request parameters - Filter vs Servlet So you need to log the txtAPN parameter, right? Yes. Actually, the contents of the listbox... as you'll see below. Those are the parameters I'm trying to capture, along with the map image url. A. We'd like to know how many requests actually generated a map image. Can you tell that, just from a parameter value? I would think that the URL itself would more likely tell you if a map was generated. No, I couldn't tell that from a parameter value, which is why I knew I had to log from within the webapp somewhere, but I didn't know whether logging should happen in a filter or at a specific place in the model. Or should I log in the fascade (pattern)? I wanted the logs to capture the url so that I knew the model reached a point where it generated an image. http://planning.maricopa.gov/agsoutput/_ags_mapdee1a8d3a28f49a48f44aaf4dc4cc316.jpg Those images are only there for 20 min, so that url is already dead. B. We'd like to know whether this app is searching for parcels primarily in the unincorporated areas of the County, or parcels located within a city jurisdiction. That part I can figure out once I know which parcels people are searching. Simply logging the parcels used in searches would allow you to do that, as you've said. The RequestDumperValve logs a /ton/ of information, and probably wouldn't get you what you want. I'm not sure where you read it, but that sounds like a platitude applied as a blanket admonition not to log in your webapp. If it's appropriate for your situation, then feel free to do it. Back to the original filter question: logging using a filter is perfectly acceptable if it's the right solution (see below for questions that might lead you to other options). Writing the filter is trivial: just implement the javax.servlet.Filter interface and be sure to read the javadoc for it before you try: you'll thank yourself, later. You can even use ServletContext.log() to write to the context log if you want. Otherwise, feel free to use your webapp's log4j or other logging facility (you'll have to configure this yourself). Just don't use System.out :) One might argue that blindly logging request parameters is not particularly useful. For instance, a quick look at your interface indicates that you can add several plots of land before performing the search. Those initial add operations may be of little use to you. You are right. Instead, you may wish to log them only at a certain point in your workflow. Since most requests go to oppositioncase.faces, you may not be able to map your Filter to a URL pattern that is fine-grained enough. Instead, it might make more sense to log this data when you know there's a reason to log it. Since you're expecting to perform statistical analysis on the data, you might even consider writing it directly to a database instead of to a plain-old log file. I'm going this route for now: FacesContext.getCurrentInstance().getExternalContext().log(the string buffer of the parcel listbox); FacesContext.getCurrentInstance().getExternalContext().log(the map URL); Produces: Mar 30, 2011 12:02:35 PM org.apache.catalina.core.ApplicationContext log INFO: PARCEL = '125-27-089' Mar 30, 2011 12:02:45 PM org.apache.catalina.core.ApplicationContext log INFO: http://planning.maricopa.gov/agsoutput/_ags_map08eb57df58224e1884e17a3e8a59b555.jpg Mar 30, 2011 12:03:16 PM org.apache.catalina.core.ApplicationContext log INFO: PARCEL = '125-27-089' OR PARCEL = '125-27-090' OR PARCEL = '125-27-091' Mar 30, 2011 12:03:27 PM org.apache.catalina.core.ApplicationContext log INFO: http://planning.maricopa.gov/agsoutput/_ags_map1a6e940afbd1494794c8d22b36f3a11a.jpg Cool webapp, by the way! Thanks! NB: The ctrl key isn't always the modifier key to use to de-select items from a multi-select list. I think that's a Microsoft Windows convention, but it's CMD-click on Mac and probably something like META-click on *NIX. Sadly, my department wanted that blurb on there. They forget stuff. Leo
Logging request parameters - Filter vs Servlet
Where do you initialize the Logger (Filter or Servlet)? The servlet 2.5 spec says you can use filters for logging, but since I'm not modifying the request or response, is logging from within a filter the right approach to logging request parameters? Using CATALINA_BASE: C:\ApacheTomcat\apache-tomcat-6.0.32 Using CATALINA_HOME: C:\ApacheTomcat\apache-tomcat-6.0.32 Using CATALINA_TMPDIR: C:\ApacheTomcat\apache-tomcat-6.0.32\temp Using JRE_HOME:C:\jdk1.6.0_24 Using CLASSPATH: C:\ApacheTomcat\apache-tomcat-6.0.32\bin\bootstrap.jar Server version: Apache Tomcat/6.0.32 Server built: February 2 2011 2003 Server number: 6.0.32.0 OS Name:Windows XP OS Version: 5.1 Architecture: x86 JVM Version:1.6.0_24-b07 JVM Vendor: Sun Microsystems Inc. Leo
Re: Logging request parameters - Filter vs Servlet
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Leon, On 3/29/2011 12:57 PM, Leo Donahue - PLANDEVX wrote: Where do you initialize the Logger (Filter or Servlet)? Which logger? The servlet 2.5 spec says you can use filters for logging, but since I'm not modifying the request or response, is logging from within a filter the right approach to logging request parameters? That depends upon your requirements. What are they? Note that reading request parameters in a Filter may trigger parsing of a POST request body which may not be something you want to happen on every request. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk2SQ2QACgkQ9CaO5/Lv0PCkogCfcjmdq3Y2nEboa10pIoqx532R Wv0An0lqu05E16lGvZNOYBUedUFHAAF3 =VaL5 -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: Logging request parameters - Filter vs Servlet
Chris, -Original Message- From: Christopher Schultz [mailto:ch...@christopherschultz.net] Subject: Re: Logging request parameters - Filter vs Servlet Leo, On 3/29/2011 12:57 PM, Leo Donahue - PLANDEVX wrote: Where do you initialize the Logger (Filter or Servlet)? Which logger? I don't know what I don't know... Log4j I guess. That is what everyone recommends. The servlet 2.5 spec says you can use filters for logging, but since I'm not modifying the request or response, is logging from within a filter the right approach to logging request parameters? That depends upon your requirements. What are they? I just want to capture the request parameters on a certain web app and log them (time and what they were) in a separate log file from the standard logs, so that I don't have to hunt them down in the standard Tomcat logs. The security tool that our telecom office uses for auditing our sites makes quite a mess of my standard logs periodically. Note that reading request parameters in a Filter may trigger parsing of a POST request body which may not be something you want to happen on every request. An example of things I don't know that I don't know... Leo
