Re: Split Gecko in standalone fuzzing-friendly programs.

2016-03-10 Thread Kurt Roeckx
On 2016-03-09 22:17, Boris Zbarsky wrote: On 3/9/16 3:47 PM, decoder...@googlemail.com wrote: Actually no. I adapted our gtests in less than an hour. Does this have to do with the set of things they're testing, or the style the tests are written in? I think the point is that some tests make

Re: Split Gecko in standalone fuzzing-friendly programs.

2016-03-09 Thread Andrew McCreight
On Wed, Mar 9, 2016 at 12:17 PM, Boris Zbarsky wrote: > Just to satisfy my curiosity, what is AFL? > AFL is American Fuzzy Lop, a fuzzer that uses a combination of compiled-in code coverage and genetic algorithms. http://lcamtuf.coredump.cx/afl/ It has found a ton of errors in

Re: Split Gecko in standalone fuzzing-friendly programs.

2016-03-09 Thread twsmith
On Wednesday, March 9, 2016 at 12:47:08 PM UTC-8, decod...@googlemail.com wrote: > > > > > the sample tests (xpcshell-tests) are extremely complicated to adapt > > > > That seems like it would be a problem in any new thing too, right? > > Actually no. I adapted our gtests in less than an hour.

Re: Split Gecko in standalone fuzzing-friendly programs.

2016-03-09 Thread David Rajchenbach-Teller
On 09/03/16 22:17, Bobby Holley wrote: > I think splitting Gecko into multiple repos is an anti-goal, because we > derive enormous productivity benefits from using a mono-repo. > > If you're serious about proposing that, I would suggest starting a new > thread to avoid derailing this one. I'm

Re: Split Gecko in standalone fuzzing-friendly programs.

2016-03-09 Thread Bobby Holley
On Wed, Mar 9, 2016 at 12:58 PM, David Rajchenbach-Teller < dtel...@mozilla.com> wrote: > I'm assuming that, for the sake of this discussion, Gecko == > mozilla-central minus stuff that can be built and distributed on its > own. In particular, most of toolkit/ is covered by this definition. > >

Re: Split Gecko in standalone fuzzing-friendly programs.

2016-03-09 Thread Boris Zbarsky
On 3/9/16 3:47 PM, decoder...@googlemail.com wrote: Actually no. I adapted our gtests in less than an hour. Does this have to do with the set of things they're testing, or the style the tests are written in? - Program is easy to start (doesn't need profiles or long initialization) and can

Re: Split Gecko in standalone fuzzing-friendly programs.

2016-03-09 Thread David Rajchenbach-Teller
I'm assuming that, for the sake of this discussion, Gecko == mozilla-central minus stuff that can be built and distributed on its own. In particular, most of toolkit/ is covered by this definition. Just looking at things I use/work on, I believe that we could relatively easily split in distinct

Re: Split Gecko in standalone fuzzing-friendly programs.

2016-03-09 Thread decoder . oh
> > > the sample tests (xpcshell-tests) are extremely complicated to adapt > > That seems like it would be a problem in any new thing too, right? Actually no. I adapted our gtests in less than an hour. > > > and we can't easily use it with AFL. > > Just to satisfy my curiosity, what is AFL?

Re: Split Gecko in standalone fuzzing-friendly programs.

2016-03-09 Thread Boris Zbarsky
On 3/9/16 3:06 PM, decoder...@googlemail.com wrote: Not at all. xpcshell is not very useful for fuzzing. It is slow OK, fair. the sample tests (xpcshell-tests) are extremely complicated to adapt That seems like it would be a problem in any new thing too, right? and we can't easily use it

Re: Split Gecko in standalone fuzzing-friendly programs.

2016-03-09 Thread decoder . oh
> To what extent are we fuzzing things like our url parser and other necko > bits via our existing shell (xpcshell) that exposes all that stuff? > > -Boris Not at all. xpcshell is not very useful for fuzzing. It is slow, the sample tests (xpcshell-tests) are extremely complicated to adapt and

Re: Split Gecko in standalone fuzzing-friendly programs.

2016-03-09 Thread Boris Zbarsky
On 3/9/16 12:54 PM, twsm...@mozilla.com wrote: I think Nicolas is right on the mark! JS shell is a good example of using a shell for fuzzing and I think media and graphics could also get a good deal out of a shell. To what extent are we fuzzing things like our url parser and other necko

Re: Split Gecko in standalone fuzzing-friendly programs.

2016-03-09 Thread Bobby Holley
On Wed, Mar 9, 2016 at 10:45 AM, wrote: > This seems to be the general response whenever this topic is brought up, > with good reason. I don't think modularizing Gecko just for the sake of it > makes much sense. What I think the take away from this should be is: when >

Re: Split Gecko in standalone fuzzing-friendly programs.

2016-03-09 Thread twsmith
This seems to be the general response whenever this topic is brought up, with good reason. I don't think modularizing Gecko just for the sake of it makes much sense. What I think the take away from this should be is: when developing a new component or maintaining an old one keep in mind that

Re: Split Gecko in standalone fuzzing-friendly programs.

2016-03-09 Thread Bobby Holley
Can you elaborate on which Gecko components you're hoping to fuzz separately? A lot of the core is pretty heavily-intertwined, so I'm pretty skeptical that we'd ever be able to separate out DOM, style, and layout from each other (for example). There are basically two barriers: (1) These

Re: Split Gecko in standalone fuzzing-friendly programs.

2016-03-09 Thread twsmith
On Wednesday, March 9, 2016 at 9:38:55 AM UTC-8, Nicolas B. Pierron wrote: > This discussion is a follow-up discussion to some emails sent privately by > accident. > > If you have not followed, I will quote David Bryant: > > Improving release quality is one of the three fundamental goals