I'm trying to replace all '\t' and '\r\n' characters from the Message field of
a log entry. I have attempted to solve a couple different ways, but both
solutions created, what appears to be, a huge memory leak that was unacceptable.
My first attempt was on the client shipper side. Here's my CONF block.
<Input eventlog_2008_default>
Module im_msvistalog
SavePos True
Query <QueryList>\
<Query Id="0">\
<Select Path="Application">*</Select> \
<Select Path="System">*</Select>\
<Select Path="Security">*</Select>\
</Query>\
</QueryList>
Exec if $Message =~ s/(\t|\r\n)/ /g log_debug("Replaced spaces & newlines in
eventlog_2008_default");
</Input>
Every message that was received from this shipper looked correct, but the
memory usage skyrocketed on the client side.
I tried a slightly different method after reading the architecture
documentation. Because each processor block is spun off into its own thread I
thought it could process the regex more efficiently. I created patterndb.xml
file that performed the above regex on the Message field on the receiver side.
The receiver is a server with a lot more horsepower than the client. Here's the
code:
<?xml version="1.0"?>
<patterndb>
<created>2013-12-03 14:14:00</created>
<version>1</version>
<group>
<name>windows_eventlog</name>
<id>1</id>
<matchfield>
<name>SourceModuleType</name>
<type>exact</type>
<value>im_msvistalog</value>
</matchfield>
<pattern>
<id>101</id>
<name>remove_tab_newline</name>
<matchfield>
<name>Message</name>
<type>regexp</type>
<value>\t|\r\n</value>
</matchfield>
<exec>if $Message =~ s/\t|\r\n/ /g log_debug("Removed tabs
and carriage return");</exec>
</pattern>
</group>
</patterndb>
This also worked as expected, but it also had, what appears to be, a huge
memory leak.
I know regex is hugely inefficient for these types of search and replaces at
higher speeds. Is there a better way? Is there something in my config that I
can do to optimize?
------------------------------------------------------------------------------
Sponsored by Intel(R) XDK
Develop, test and display web and hybrid apps with a single code base.
Download it for free now!
http://pubads.g.doubleclick.net/gampad/clk?id=111408631&iu=/4140/ostg.clktrk
_______________________________________________
nxlog-ce-users mailing list
nxlog-ce-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nxlog-ce-users
