On Wednesday, March 9, 2016 at 12:47:08 PM UTC-8, decod...@googlemail.com wrote: > > > > > the sample tests (xpcshell-tests) are extremely complicated to adapt > > > > That seems like it would be a problem in any new thing too, right? > > Actually no. I adapted our gtests in less than an hour. > > > > > > and we can't easily use it with AFL. > > > > Just to satisfy my curiosity, what is AFL? > > http://lcamtuf.coredump.cx/afl/ > > > > > > but that still doesn't solve the problem that people have to write the > > > necessary code that we can fuzz then. > > > > OK. This is a problem, certainly, and pretty independent of both the > > "split Gecko" thing and the existence of shells, right? > > Not really no. Because some shells and tests we have are very straightforward > to use and we can figure it out ourselves. xpcshell is not such an example. > > > > > What are the necessary qualities for things you can fuzz? > > > It depends on the type of fuzzing. Let's stick to AFL: > > - Program is easy to start (doesn't need profiles or long initialization) and > can be packaged > - Has AFL persistent mode support (requires support on C++ level) > - Exercises the targeted feature in a similar way compared to how Firefox > would do it > - Optionally has some extra testing features (e.g. gczeal, ion-eager, > extra-checks for the JS shell) that make bug finding easier > - Can be compiled with all sanitizer types (although MSan is not going to > work for some stuff even in shells) > > > That's just a dump out of my head, might be missing some stuff.
More qualities that benefit fuzzing: - Fast - Deterministic (very useful for feedback driven fuzzing) - Automate-able, easily run from a script from the command line - Distribute-able in parallel and size by side. So ideally statically linked and multiple instances can run at the same time in the same environment. - Lots of assertions - Targeted, only meant to test one area of code (very useful for feedback driven fuzzing) - Cross platform, unless functionality is the exactly the same and then in that case target Linux - Accepts data from a file or stdin, having to depend on web servers etc, does complicate things but sometimes is required - Simple build system, now I feel like I'm just stating the obvious :) _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
