On Friday 18 December 2009 08:42:51 pm LC Bruzenak wrote: > What is the plan for this tool? As I said, I think it is very nice > feature-wise in general but in practice it isn't living up to > expectations. > I can try to help but will take a while to get python-proficient. Or > is the trouble in the parse library?
The audit parsing library has not been optimized for handling large data sets. I don't think its the entire problem you are seeing, but I'm sure its a contributor to the problem. I was planning to look at performance issues in a future release. The immediate plans are: 1) Get store and forward working for remote logging 2) Get intermingled records fixed in auparse 3) Get sighup corrected for network options 4) Look at performance issues in auparse The biggest obstacle is #2 above. What makes fixing this in auparse so much fun is that there is a state machine right in the middle of auparse (due to the feed input option) that was not in aureport or ausearch, so I couldn't fix it at the same time. > I do not have scientific data yet, but recently I loaded one 100MB > audit file from the store. It took around 3 minute to load. Then I > changed the source and that one took longer. When it was finally > loaded it, the process size was over 2GB. Sure. The audit viewer could be changed to hold only the records that might be displayed and not all of them. It would then need to track what's displayed and start a background thread to gather more info for display as you scroll around. > I can run some better tests and try to get some data if it is helpful. > Are there ways I can try to exercise the parse library outside the GUI > on these same files which might help me know what to look for? The parse library has some test programs in https://fedorahosted.org/audit/browser/trunk/auparse/test that could be adapted for performance testing. But I don't want to change the code in auparse until after we make it handle interlaced records correctly. But you could test the native C library against the python version to see if python itself is adding delay. -Steve PS - I keep a TODO file up to date that will always let you know what the immediate plans are: https://fedorahosted.org/audit/browser/trunk/TODO -- Linux-audit mailing list [email protected] https://www.redhat.com/mailman/listinfo/linux-audit
