Re: Chainsaw log-parsing classes

2018-01-20 Thread Scott Deboy 
Yeah - good idea.

Here's the code Chainsaw uses to grab a log4j XML config's
FileAppender definition, including the PatternLayout, and build what's
needed to process the log messages going in to the target file.

BTW this is also how the log4j2 Advertiser integrates with Chainsaw.

  Map entries =
LogFilePatternLayoutBuilder.getAppenderConfiguration(log4jConfigFile);
  for (Iterator iter =
entries.entrySet().iterator();iter.hasNext();) {
try {
  Map.Entry entry = (Map.Entry)iter.next();
  String name = (String) entry.getKey();
  Map values = (Map) entry.getValue();
  //values: conversion, file
  String conversionPattern =
values.get("conversion").toString();
  File file = new File(values.get("file").toString());
  URL fileURL = file.toURI().toURL();
  String timestampFormat =
LogFilePatternLayoutBuilder.getTimeStampFormat(conversionPattern);
  String receiverPattern =
LogFilePatternLayoutBuilder.getLogFormatFromPatternLayout(conversionPattern);
  VFSLogFilePatternReceiver fileReceiver = new
VFSLogFilePatternReceiver();
  fileReceiver.setName(name);
  fileReceiver.setAutoReconnect(true);
  fileReceiver.setContainer(LogUI.this);
  fileReceiver.setAppendNonMatches(true);
  fileReceiver.setFileURL(fileURL.toURI().toString());
  fileReceiver.setTailing(true);
  fileReceiver.setLogFormat(receiverPattern);
  fileReceiver.setTimestampFormat(timestampFormat);
  fileReceiver.setThreshold(Level.TRACE);
  pluginRegistry.addPlugin(fileReceiver);
  fileReceiver.activateOptions();

Scott

On 1/20/18, Mikael Ståldal  wrote:
> Maybe we could take some code from Chainsaw and implement a
> LogEventParser for Log4j 2?
>
> https://github.com/apache/logging-log4j2/blob/master/log4j-core/src/main/java/org/apache/logging/log4j/core/parser/TextLogEventParser.java
>
>
> On 2018-01-19 17:00, Scott Deboy wrote:
>> Here's info on how you can use the same Receiver Chainsaw uses in your
>> own
>> application - note this is not built in to log4j2:
>>
>> https://stackoverflow.com/questions/47605483/log4j-unstructured-log-parser-in-java-or-scala?rq=1
>>
>> Scott
>>
>> On Jan 19, 2018 7:32 AM, "Joseph Husby"  wrote:
>>
>> Hello!
>>
>>
>>
>> I’m working on a project where we need to take log4j2 output and
>> (offline)
>> load it into a Mongo database.  We can't use a mongo appender because the
>> database is not available where the software is running.  We could use
>> the
>> log4j2's JsonLayout to have an easily parsable log format, but would
>> prefer
>> not to as it makes the logs less human readable (especially timestamps).
>>
>>
>> Obviously Chainsaw has the capability to parse standard log4j output, but
>> is any of that exposed so we could use those classes in our application?
>> We don’t need the GUI component.
>>
>>
>>
>> Thank you for any help you’re able to provide!
>>
>>
>> ~ Joe ~
>>
>> p.s. resending this after subscribing to the list as I don't see my
>> original message in the archive, so assuming it was rejected.  Sorry if
>> I'm
>> double messaging!
>>
>
>
> -
> To unsubscribe, e-mail: log4j-user-unsubscr...@logging.apache.org
> For additional commands, e-mail: log4j-user-h...@logging.apache.org
>
>

-
To unsubscribe, e-mail: log4j-user-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-user-h...@logging.apache.org

Re: Chainsaw log-parsing classes

2018-01-20 Thread Mikael Ståldal 
Maybe we could take some code from Chainsaw and implement a 
LogEventParser for Log4j 2?


https://github.com/apache/logging-log4j2/blob/master/log4j-core/src/main/java/org/apache/logging/log4j/core/parser/TextLogEventParser.java


On 2018-01-19 17:00, Scott Deboy wrote:

Here's info on how you can use the same Receiver Chainsaw uses in your own
application - note this is not built in to log4j2:

https://stackoverflow.com/questions/47605483/log4j-unstructured-log-parser-in-java-or-scala?rq=1

Scott

On Jan 19, 2018 7:32 AM, "Joseph Husby"  wrote:

Hello!



I’m working on a project where we need to take log4j2 output and (offline)
load it into a Mongo database.  We can't use a mongo appender because the
database is not available where the software is running.  We could use the
log4j2's JsonLayout to have an easily parsable log format, but would prefer
not to as it makes the logs less human readable (especially timestamps).


Obviously Chainsaw has the capability to parse standard log4j output, but
is any of that exposed so we could use those classes in our application?
We don’t need the GUI component.



Thank you for any help you’re able to provide!


~ Joe ~

p.s. resending this after subscribing to the list as I don't see my
original message in the archive, so assuming it was rejected.  Sorry if I'm
double messaging!




-
To unsubscribe, e-mail: log4j-user-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-user-h...@logging.apache.org

Re: Chainsaw log-parsing classes

2018-01-20 Thread Mikael Ståldal 
FWIW, Log4j 2's log4j-core has an API to parse the output of JsonLayout, 
XmlLayout and YamlLayout:


https://github.com/apache/logging-log4j2/tree/master/log4j-core/src/main/java/org/apache/logging/log4j/core/parser


On 2018-01-19 16:32, Joseph Husby wrote:

Hello!



I’m working on a project where we need to take log4j2 output and (offline)
load it into a Mongo database.  We can't use a mongo appender because the
database is not available where the software is running.  We could use the
log4j2's JsonLayout to have an easily parsable log format, but would prefer
not to as it makes the logs less human readable (especially timestamps).


Obviously Chainsaw has the capability to parse standard log4j output, but
is any of that exposed so we could use those classes in our application?
We don’t need the GUI component.



Thank you for any help you’re able to provide!


~ Joe ~

p.s. resending this after subscribing to the list as I don't see my
original message in the archive, so assuming it was rejected.  Sorry if I'm
double messaging!




-
To unsubscribe, e-mail: log4j-user-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-user-h...@logging.apache.org

Re: Chainsaw log-parsing classes

2018-01-19 Thread Ralph Goers 
If you application is running somewhere and Mongo is running somewhere else I 
would recommend using the Flume Appender to send the events to a Flume node 
that is running where Mongo is. Then have Flume write the events to Mongo.

Ralph

> On Jan 19, 2018, at 8:32 AM, Joseph Husby  wrote:
> 
> Hello!
> 
> 
> 
> I’m working on a project where we need to take log4j2 output and (offline)
> load it into a Mongo database.  We can't use a mongo appender because the
> database is not available where the software is running.  We could use the
> log4j2's JsonLayout to have an easily parsable log format, but would prefer
> not to as it makes the logs less human readable (especially timestamps).
> 
> 
> Obviously Chainsaw has the capability to parse standard log4j output, but
> is any of that exposed so we could use those classes in our application?
> We don’t need the GUI component.
> 
> 
> 
> Thank you for any help you’re able to provide!
> 
> 
> ~ Joe ~
> 
> p.s. resending this after subscribing to the list as I don't see my
> original message in the archive, so assuming it was rejected.  Sorry if I'm
> double messaging!



-
To unsubscribe, e-mail: log4j-user-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-user-h...@logging.apache.org