Thomas, We had a business requirement of correlating logs (sort of SIEM) from endpoints, edge devices for threat identification. Few of the applications were in Java with log4j log configuration, few in other streams. I have pasted the design we followed to configure and forward the logs to elastic for indexing and analyzing thereafter, look at it. So, we forwarded it to logstash server, which is configured with log4j encoder, i believe thats what you want as well, rather than you trying to explicitly converting logs to json.
http://www.tiikoni.com/tis/view/?id=7a93bfc Regards, Santhosh On Fri, Jul 31, 2015 at 11:03 PM, Thomas Schneider < thomas.schnei...@euskill.com> wrote: > I downloaded the jar from > > http://central.maven.org/maven2/net/logstash/log4j/jsonevent-layout/1.7/jsonevent-layout-1.7.jar > > Put it in /usr/share/cloudstack-management/webapps/client/WEB-INF/lib/ > > then edited /etc/cloudstack/management/log4j-cloud.xml > > From: > > <appender name="FILE" > class="org.apache.log4j.rolling.RollingFileAppender"> > <param name="Append" value="true"/> > <param name="Threshold" value="TRACE"/> > <rollingPolicy > class="org.apache.log4j.rolling.TimeBasedRollingPolicy"> > <param name="FileNamePattern" > > value="/var/log/cloudstack/management/management-server.log.%d{yyyy-MM-dd}.gz"/> > <param name="ActiveFileName" > value="/var/log/cloudstack/management/management-server.log"/> > </rollingPolicy> > <layout class="org.apache.log4j.EnhancedPatternLayout"> > <param name="ConversionPattern" value="%d{ISO8601} %-5p > [%c{1.}] (%t:%x) %m%n"/> > </appender> > > To: > <appender name="FILE" > class="org.apache.log4j.rolling.RollingFileAppender"> > <param name="Append" value="true"/> > <param name="Threshold" value="TRACE"/> > <rollingPolicy > class="org.apache.log4j.rolling.TimeBasedRollingPolicy"> > <param name="FileNamePattern" > > value="/var/log/cloudstack/management/management-server.log.%d{yyyy-MM-dd}.gz"/> > <param name="ActiveFileName" > value="/var/log/cloudstack/management/management-server.log"/> > </rollingPolicy> > <layout class="net.logstash.log4j.JSONEventLayoutV1" /> > </appender> > > but after that I don't have log anymore. > > I also tryed to: > mkdir -p /root/classpath/ > cp jsonevent-layout-1.7.jar /root/classpath > vi /etc/environement > add: CLASSPATH="/root/classpath" > source /etc/environement > > but i have the same result. > > If someone can advice me ? > > > Le 30/07/2015 21:36, Thomas Schneider a écrit : > > Hello, > > > > I would like to setup ELK stack to monitor CloudStack Log. > > I have already setup a central Elastisearch + Logstach + Kibana server > > who receive logs from all my cloudstack management server via > > logstach-forwader and it work pretty well with the standart system's log > > file like /var/log/syslog etc... because they can be easyly parsed by > > logstach's grok filter. > > > > But the main problem I have, is I didn't find a good technique to parse > > cloudstack log file. > > > > However i founded a plugin for log4j who's called log4j-jsonevent-layout > > that can output the cloudstack log in json and the json log are easy to > > parse for logstache, but a dindn't found how to setup this plugin. > > > > So howto setup log4j-jsonevent-layout witch cloudstack ? > > If someone can advise me on this issue. > > > > Regards, > > -- > *Thomas Schneider* > Directeur des Opérations > Euskill SARL > Web: www.euskill.com > Mobile: +33 (0)6 19 26 47 76 > Mail: thomas.schnei...@euskill.com > 5 rue de Phalsbourg > F-67000 Strasbourg >
