> On Oct 12, 2017, at 5:21 PM, Aaron Eppert <aaron.epp...@packetsled.com> wrote: > > I crafted a custom file analysis plugin that attaches to specific MIME types > via file_sniff and fires an appropriate event once processing has been > completed. > > I had to jump through a few hoops to make a file analysis plugin, first, but > those were cleared and everything runs and loads appropriately there (bro -NN > verified.) My test regime is very straight forward, I have several PCAPs > cooked up that contain simple HTTP file GETs (that extract otherwise properly > and do not exhibit missing_bytes) and I am running them via `bro -C -r > <>.pcap`. My issue comes with utter and complete inconsistency with execution > - it is, effectively, a coin flip, with zero changes. > > When I have dumped the buffers being processed, as my file analysis plugin > has a secondary verification to make sure the data passed is appropriate - > which is confusing, as the mime type fires correct, which seems to indicate a > bug somewhere in the data path - the correct execution, clearly has the > proper data in it. The invalid executions, again changing nothing other than > a subsequent execution, shows a buffer of what appears to be completely > random data. > >
That sounds a lot like an uninitialized buffer somewhere. I wonder if you compile bro and your plugin with -fsanitize=address if you will trigger something with that. > I currently cannot supply the file analysis plugin for inspection, but would > very much appreciate insight in how to find the root cause. It very much > seems to be upstream. If I run the analysis portion of the plugin as a free > standing executable outside of Bro against the data transferred via HTTP, > everything works perfect and the structures are filled accordingly. If you are seeing what looks like random data in your plugin you should be able to reproduce this behavior by having a file analysis plugin that just dumps out the buffers to stdout (as hex?). Can you rip out all the custom logic in your plugin leaving something that just dumps the buffers as-is? That should leave you with just the hello world of file analysis plugins. If that shows the problem we should be able to figure out where it is coming from. I don't think file analysis is inherently broken somewhere, otherwise the bro test suite would fail. I think this would have to point to something unique about your plugin. I think you are the first person to build an out of tree file analysis plugin, so there may be an issue with the bro<->plugin interface for file analsys itself. If that is the case, extracting something like the built in md5 analysis plugin to an external plugin and calling it 'mymd5' would show the same problems. > I saw BIT-1832, and there could be similar root causes in there, but I have > not had time to investigate otherwise. The issues I am raising, again, are > command line replay via command line, not even “live” network traffic or > tcpreplay over a NIC/dummy interface. That does sound similar, but I'm not sure if they were seeing different results on the same pcap on different runs. — Justin Azoff _______________________________________________ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
