Re: Configuring log format for console output (catalina.out)
Thanks Guido,
I hope that with the rsyslog magic (see
startmsg.regex="^[[:digit:]]{2}-[[:alpha:]]{3}-[[:digit:]]{4}
[[:digit:]]{1,2}:[[:digit:]]{1,2}:[[:digit:]]{1,2}\\.[[:digit:]]{3}" for
the reference), I will be able to group this messages, otherwise I'll jump
into your solution.
Joan
Missatge de Jäkel, Guido del dia dc., 22 de maig 2019 a
les 8:55:
> Dear Joan,
>
> by use of the common scripts, the file catalina.out will contain the
> console output (stdout/tderr) of the *JVM* process and -- if not configured
> in another way -- of the applications. Therefore, you have to deal with the
> features of output formatting of current JVMs. Or -- as me -- you might
> pipe this file descriptors through a tiny script that will prepend a
> timestamp in a format of your choice. This will have the advantage that it
> also handle "console" output of Java application; despite of the fact that
> using stdout/stderr instead of a java logging mechanism is a very bad style.
>
> If you don't need to process thousands of lines per second, a simple shell
> script may do the job:
>
> while read line; do echo "`date -Ins` $line"; done
>
> To avoid "double-stamping", you may add an heuristic check of the incoming
> line; please adjust the RegExpr to your neeeds
>
> while read line; do [[ ! "$line" =~ ^\d\d\d\d ]] && echo -n
> "[`date -Ins`] "; echo $line; done
>
>
> Guido
>
RE: Configuring log format for console output (catalina.out)
Dear Joan, by use of the common scripts, the file catalina.out will contain the console output (stdout/tderr) of the *JVM* process and -- if not configured in another way -- of the applications. Therefore, you have to deal with the features of output formatting of current JVMs. Or -- as me -- you might pipe this file descriptors through a tiny script that will prepend a timestamp in a format of your choice. This will have the advantage that it also handle "console" output of Java application; despite of the fact that using stdout/stderr instead of a java logging mechanism is a very bad style. If you don't need to process thousands of lines per second, a simple shell script may do the job: while read line; do echo "`date -Ins` $line"; done To avoid "double-stamping", you may add an heuristic check of the incoming line; please adjust the RegExpr to your neeeds while read line; do [[ ! "$line" =~ ^\d\d\d\d ]] && echo -n "[`date -Ins`] "; echo $line; done Guido
Re: Configuring log format for console output (catalina.out)
Thanks a lot for your answer Chris > > I am trying to forward the logs generated on a server to a > > centralized log collector (using rsyslog), one of the features I > > would like to use the ability to group messages by using their > > timestamp as the start of the message. By reading the documentation > > at https://tomcat.apache.org/tomcat-9.0-doc/logging.html#Console, I > > can't see the proper way to get a common format (preferably with > > timestamp), or ar tleast save the outputs into three different > > types in different files? > > I think you are being confused by the fact that the default log > pattern contains a newline character. For example, these are the first > two entries in my local catalina.out file: > > Apr 12, 2019 1:55:03 PM org.apache.coyote.AbstractProtocol init > INFO: Initializing ProtocolHandler ["ajp-nio-8215"] > Apr 12, 2019 1:55:03 PM org.apache.tomcat.util.net.NioSelectorPool > getSharedSelector > INFO: Using a shared selector for servlet write/read > Apr 12, 2019 1:55:03 PM org.apache.coyote.AbstractProtocol init > > (Apologies for the word-wrapping) > > The first entry is [Apr 12, 2019 1:55:03 PM] and the text ends with > ["ajp-nio-8215"] > > The second entry has the same timestmp and the text ends with > "AbstractProtocol init" > > Stack traces are always impossible to handle when looking at the > resulting file, since they almost always span multiple lines of text > - -- that is, they have embedded newlines in the message. > > Are you asking about how to make sense of the log file, how to send > log messages to a central server, or how to coalesce log messages by > timestamp? > Rsyslog allows to read multiline messages by matching a regular expression (see https://www.rsyslog.com/doc/v8-stable/configuration/modules/imfile.html#startmsg-regex), this allow the messages to be forwarded to a third party (graylog, logstash, ..) as a a single message. So my intention would be to be able to make all the messages to start with a timestamp (or a defined format) After your message I undertand that this is a single message (I didn't realize it) and all the messages in catalina.out are obeying the same format. Being the next lines a single message, am I right? 19-May-2019 06:25:20.937 INFO [localhost-startStop-1] > org.apache.jasper.servlet.TldScanner.scanJars At least one JAR was scanned > for TLDs yet contained no TLDs. Enable debug logging for this logger for a > comple > te list of JARs that were scanned but no TLDs were found in them. Skipping > unneeded JARs during scanning can improve startup time and JSP compilation > time. > [ERROR] Unable to find config file. Creating new servlet engine config > file: /WEB-INF/server-config.wsdd > [WARN] Web application uses > org.apache.axis2.transport.http.AxisAdminServlet; please update web.xml to > use org.apache.axis2.webapp.AxisAdminServlet instead > [WARN] Unable to load manifest for StAX implementation at jrt:/java.xml/ > [INFO] Clustering has been disabled > Thanks, Joan > >
Re: Configuring log format for console output (catalina.out)
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Joan, On 5/21/19 09:11, Joan wrote: > Hi all, > > I am trying to forward the logs generated on a server to a > centralized log collector (using rsyslog), one of the features I > would like to use the ability to group messages by using their > timestamp as the start of the message. By reading the documentation > at https://tomcat.apache.org/tomcat-9.0-doc/logging.html#Console, I > can't see the proper way to get a common format (preferably with > timestamp), or ar tleast save the outputs into three different > types in different files? I think you are being confused by the fact that the default log pattern contains a newline character. For example, these are the first two entries in my local catalina.out file: Apr 12, 2019 1:55:03 PM org.apache.coyote.AbstractProtocol init INFO: Initializing ProtocolHandler ["ajp-nio-8215"] Apr 12, 2019 1:55:03 PM org.apache.tomcat.util.net.NioSelectorPool getSharedSelector INFO: Using a shared selector for servlet write/read Apr 12, 2019 1:55:03 PM org.apache.coyote.AbstractProtocol init (Apologies for the word-wrapping) The first entry is [Apr 12, 2019 1:55:03 PM] and the text ends with ["ajp-nio-8215"] The second entry has the same timestmp and the text ends with "AbstractProtocol init" Stack traces are always impossible to handle when looking at the resulting file, since they almost always span multiple lines of text - -- that is, they have embedded newlines in the message. Are you asking about how to make sense of the log file, how to send log messages to a central server, or how to coalesce log messages by timestamp? - -chris -BEGIN PGP SIGNATURE- Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/ iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAlzkEO8ACgkQHPApP6U8 pFg8EA/+KsH4JGirKakL+/OxqAz1W7T7MoR2QE9Y1Tk26FL6XI+VITk11gqLegH5 yx3zwmXLS9v2XilppnSvRB9xPd073Uf4v0NPSjCt1vSY7obPHYcRSIuA43QU9Pqb 7pEgxuGn+uUifRRK+DLd6Wcd3IkFo2hfKG/YHFL5McKxqsrEvPsspWU7uLRHoKML BR1MM5iKbsiBEJfncfyZqeGY2Yq4HOarcp//iv5bC20RgakLxlSjiWAyJqC77PKH 3dy9XaV/VyQpjEQe8/ADt1NqhF6qBVj/KcKeA41AAwL7grxX0R23ZeSQsKJeulgf jnXcK8mHWXs4Dybu7hhUbkS47whbfW6M3H4ZhGYutsoOB58shzbVR0MUtEYlOBMg GbpCt5xPW6NZ5LMbEDTMlQW95XBnrLdxe8dFdTo+qgEdSoe3o4mdvGUtrlgAxZlq Ay28U+/maI84StrScDwfC96chakdaGj/4CxyFwoj+5uVeeiBbglubR1gCghqVjM7 MQWDKrTACmcEu4ZbLy1sgNtBb/xjCYRtp05JrB531E9Py7//f6qprIKjAYOBLcxF uWDKP204C3UlC3/mQt6Ewd6QzQM0xCXPm0k+6817wf3xIaN/aST4R0kbR0OjkB6F CfAm8Qf2FVgRHOP4zzKHvm82wjET2FJmcw38do2ki89yml1iKXg= =UolI -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
