Hi,
The regexp replacement operator s/// is leaking memory. This
is a known issue and this is already fixed. The fix will be available in
the next release of the nxlog community edition.
Have you tried using the replace() function? This should be able to do
what you need.
Regards,
Botond
On Wed, 4 Dec 2013 23:32:00 +0000
"Mitzimberg, Justin" <justin.mitzimb...@portofportland.com> wrote:
> I'm trying to replace all '\t' and '\r\n' characters from the Message field
> of a log entry. I have attempted to solve a couple different ways, but both
> solutions created, what appears to be, a huge memory leak that was
> unacceptable.
>
> My first attempt was on the client shipper side. Here's my CONF block.
>
> <Input eventlog_2008_default>
> Module im_msvistalog
> SavePos True
> Query <QueryList>\
> <Query Id="0">\
> <Select Path="Application">*</Select> \
> <Select Path="System">*</Select>\
> <Select Path="Security">*</Select>\
> </Query>\
> </QueryList>
> Exec if $Message =~ s/(\t|\r\n)/ /g log_debug("Replaced spaces & newlines
> in eventlog_2008_default");
> </Input>
>
> Every message that was received from this shipper looked correct, but the
> memory usage skyrocketed on the client side.
>
> I tried a slightly different method after reading the architecture
> documentation. Because each processor block is spun off into its own thread I
> thought it could process the regex more efficiently. I created patterndb.xml
> file that performed the above regex on the Message field on the receiver
> side. The receiver is a server with a lot more horsepower than the client.
> Here's the code:
>
> <?xml version="1.0"?>
> <patterndb>
> <created>2013-12-03 14:14:00</created>
> <version>1</version>
>
> <group>
> <name>windows_eventlog</name>
> <id>1</id>
> <matchfield>
> <name>SourceModuleType</name>
> <type>exact</type>
> <value>im_msvistalog</value>
> </matchfield>
> <pattern>
> <id>101</id>
> <name>remove_tab_newline</name>
> <matchfield>
> <name>Message</name>
> <type>regexp</type>
> <value>\t|\r\n</value>
> </matchfield>
> <exec>if $Message =~ s/\t|\r\n/ /g log_debug("Removed
> tabs and carriage return");</exec>
> </pattern>
> </group>
> </patterndb>
>
> This also worked as expected, but it also had, what appears to be, a huge
> memory leak.
>
> I know regex is hugely inefficient for these types of search and replaces at
> higher speeds. Is there a better way? Is there something in my config that I
> can do to optimize?
>
------------------------------------------------------------------------------
Sponsored by Intel(R) XDK
Develop, test and display web and hybrid apps with a single code base.
Download it for free now!
http://pubads.g.doubleclick.net/gampad/clk?id=111408631&iu=/4140/ostg.clktrk
_______________________________________________
nxlog-ce-users mailing list
nxlog-ce-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nxlog-ce-users
