In the meantime (actually nearly a year ago), log parsing was rewritten
and now does de-duplication instantly. This should reduce memory usage a
lot - my experience is that especially large lots have lots of
duplication included.
I also removed some intermediate steps in the chain from logfile to aa-
logprof questions, which means less intermediate variables/arrays that
eat quite some memory.
The new log parsing is currently only in git master (not in 2.13.x). It
will be included in the next major release (probably 3.0).
Processing / asking about the log events one by one (instead of reading
the whole log first) would in theory save even more memory. However,
IMHO it would make the user experience worse because it would also mean
loosing the ability to sort the questions aa-logprof asks by profile and
rule type.
Therefore I'm closing this bug as "mostly fixed" ;-) - if you still see
aa-logprof eating too much memory, feel free to open a new bug.
** Changed in: apparmor
Status: Triaged => Fix Committed
--
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to apparmor in Ubuntu.
https://bugs.launchpad.net/bugs/387657
Title:
aa-logprof: doesn't handle large logs
Status in AppArmor:
Fix Committed
Status in apparmor package in Ubuntu:
Confirmed
Bug description:
Binary package hint: apparmor
Ubuntu 8.04.2
Package: apparmor-utils
Version: 2.1+1075-0ubuntu9.2
My first experience with AppArmor was finding a kernel log file that
was full of 800 MB of AppArmor warnings from Samba and CUPS. I'm not
sure what suddenly enabled AppArmor on this LTS system, but while
fixing the problem I noticed....
aa-logprof, while processing an 800 MB log file, drove the load
average up into the high 40's. I suspect it was trying to load the
whole thing into memory (on a system with 1 GB of RAM). I'd recommend
revising the architecture so that it processes the lines as it sees
them.
I worked around the problem by splitting the file into a dozen 75MB
chunks.
To manage notifications about this bug go to:
https://bugs.launchpad.net/apparmor/+bug/387657/+subscriptions
--
Mailing list: https://launchpad.net/~touch-packages
Post to : touch-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~touch-packages
More help : https://help.launchpad.net/ListHelp
