[
https://bro-tracker.atlassian.net/browse/BIT-1289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Johanna Amann updated BIT-1289:
-------------------------------
Resolution: Works for Me
Status: Closed (was: Open)
This should have been fixed in 2.3 -- the return value of SendEventStreamEvent
was changed to stream->num_fields, which is equivalent to your patch.
There also were a number of other fixes in the input framework, so you should
consider upgrading. Closing the bug, if this does not solve your problem please
reopen it.
> Ascii Stream Reader not freeing up memory
> ------------------------------------------
>
> Key: BIT-1289
> URL: https://bro-tracker.atlassian.net/browse/BIT-1289
> Project: Bro Issue Tracker
> Issue Type: Problem
> Components: Bro
> Affects Versions: 2.2, 2.3
> Environment: CentOS 6.6 distro x86_64
> gcc 4.4.7 20120313 (Red Hat 4.4.7-4)
> ld 2.20.51.0.2-5.36.el6 20100205
> glibc 2.12 1.32.el6_5.4
> libunwind 1.1 2.el6
> libstdc++ 4.4.7 4.el6
> libm (via glibc-devel 2.12 1.07.el6_4.4)
> Reporter: Steve Egbert
> Assignee: Johanna Amann
> Labels: input-framework
> Attachments: bro-ascii-stream-event-leak.patch
>
>
> A triple leak occurred whenever an ASCII file has its modify time changed
> {{DoUpdate}} as well as once during {{DoInit}} initialization which called
> {{DoUpdate}}.
> When tracing the EVENT_STREAM message from reading of an input file using
> Ascii framework, but I spotted a memory leak that impacts both Bro 2.2 (and
> possibly 2.3) as reported by gperftool/tcmalloc {{HEAPCHECK}} output:
> Using local file /usr/bin/bro.
> Leak of 5148576 bytes in 160893 objects allocated from:
> @ 7ab2b5 AsciiFormatter::ParseValue
> @ 7d9404 input::reader::Ascii::DoUpdate
> @ 7d665c input::ReaderBackend::Update
> @ 7d97ba input::reader::Ascii::DoHeartbeat
> @ 7d674a input::ReaderBackend::OnHeartbeat
> @ 7b0e95 threading::HeartbeatMessage::Process
> @ 7b085a threading::MsgThread::Run
> @ 7ad264 threading::BasicThread::launcher
> @ 37340079d1 start_thread
> @ 37338e886d __clone
> @ 0 _init
> Leak of 5148480 bytes in 160890 objects allocated from:
> @ 7d91e2 input::reader::Ascii::DoUpdate
> @ 7d665c input::ReaderBackend::Update
> @ 7d97ba input::reader::Ascii::DoHeartbeat
> @ 7d674a input::ReaderBackend::OnHeartbeat
> @ 7b0e95 threading::HeartbeatMessage::Process
> @ 7b085a threading::MsgThread::Run
> @ 7ad264 threading::BasicThread::launcher
> @ 37340079d1 start_thread
> @ 37338e886d __clone
> @ 0 _init
> Leak of 2062752 bytes in 160854 objects allocated from:
> @ 673735 copy_string
> @ 7ab399 AsciiFormatter::ParseValue
> @ 7d9404 input::reader::Ascii::DoUpdate
> @ 7d665c input::ReaderBackend::Update
> @ 7d97ba input::reader::Ascii::DoHeartbeat
> @ 7d674a input::ReaderBackend::OnHeartbeat
> @ 7b0e95 threading::HeartbeatMessage::Process
> @ 7b085a threading::MsgThread::Run
> @ 7ad264 threading::BasicThread::launcher
> @ 37340079d1 start_thread
> @ 37338e886d __clone
> @ 0 _init
> In short, the leakage was found to occur in the Manager during processing
> incoming EVENT_STREAM messages that was sent by its child thread(s) via
> {{queue_out}} queue. Manager used {{RetrieveOut()}} of getting these
> messages from the {{queue_out}} and eventually calling {{SendEntry()}} for
> final processing and disposal of these messages (note the
> {{delete_vals_ptr_array()}} at the end of the {{SendEntry()}} function). But
> the error was in miscomputing of {[readFields}} for its actual number of
> fields associating with the msg/vals/fields.
> In {{SendEntry()}} (of {{Manager.cc}}), note the out-of-sync between
> EVENT_STREAM's {{readField}} assignment and the original
> {{stream->num_fields}} waaaay back in when the message was first created by
> the {{DoInit}}.
> So, when it comes to counting number of columns/fields, it becomes an issue
> of whether to go by what was in the first line of the text file describing
> the columns or what was given in the data line afterwards.
> Perhaps a better value is to use {{(EventStream *)i->num_fields}} instead,
> but only for EVENT_STREAM? I've demonstrated removal of these leaks using
> this field instead.
> We've already lost the {{columnMap}} details at file read time which would
> have made for a better comparator value.
--
This message was sent by Atlassian JIRA
(v6.4-OD-09-008#64005)
_______________________________________________
bro-dev mailing list
[email protected]
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
