Re: [libvirt] Should Libvirt apply for OSS-Fuzz?
On Fri, Dec 02, 2016 at 11:08:14AM +0100, Martin Kletzander wrote: > On Fri, Dec 02, 2016 at 09:16:26AM +, Daniel P. Berrange wrote: > > On Fri, Dec 02, 2016 at 10:14:22AM +0100, Martin Kletzander wrote: > > > On Fri, Dec 02, 2016 at 08:44:48AM +0100, Michal Privoznik wrote: > > > > Google announced OSS-Fuzz project [1]. It's aim is to test projects with > > > > significant user base and/or critical projects to the global > > > > infrastructure. I like to think that libvirt falls in both categories > > > > :-) > > > > You can find a list of already accepted projects here [2]. Once accepted > > > > to the project we would have to provide some scripts that build libvirt > > > > and run some tests. > > > > > > > > > > I was thinking about that too. And danpb would like that as well, I > > > guess, since he came up with the fuzzing idea for GSoC. > > > > > > > One of the disadvantages is that we have to provide a docker(!) image > > > > where the scripts would run from. > > > > > > > > > > But it's not like the whole libvirt has to be installed and running > > > there, right? It's unit-test fuzzing, it will just link against > > > libvirt.la and run random APIs (mostly public ones, I guess). > > > > You have to write test harnesses for the fuzzer, so it'll fuzz whatever > > APIs you call from your test harnesses. > > > > Yeah, we choose what to do. I was elaborating on what we are going to > choose. So I take it as you like the idea? I'm totally supportive of fuzzing of libvirt in general. I'm ambivalent on where we actually run the fuzzers we create. Whether its on Google or our Jenkins CI, or both doesn't matter as long as we have something running somewhere. Since our Jenkins CI is already maxing out CPU capacity quite often, it makes sense to try Google for now - we can easily move later if needed, since their framework is built around opensource fuzzing libraries/tools. Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://entangle-photo.org -o-http://search.cpan.org/~danberr/ :| -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] Should Libvirt apply for OSS-Fuzz?
On Fri, Dec 02, 2016 at 09:16:26AM +, Daniel P. Berrange wrote: On Fri, Dec 02, 2016 at 10:14:22AM +0100, Martin Kletzander wrote: On Fri, Dec 02, 2016 at 08:44:48AM +0100, Michal Privoznik wrote: > Google announced OSS-Fuzz project [1]. It's aim is to test projects with > significant user base and/or critical projects to the global > infrastructure. I like to think that libvirt falls in both categories :-) > You can find a list of already accepted projects here [2]. Once accepted > to the project we would have to provide some scripts that build libvirt > and run some tests. > I was thinking about that too. And danpb would like that as well, I guess, since he came up with the fuzzing idea for GSoC. > One of the disadvantages is that we have to provide a docker(!) image > where the scripts would run from. > But it's not like the whole libvirt has to be installed and running there, right? It's unit-test fuzzing, it will just link against libvirt.la and run random APIs (mostly public ones, I guess). You have to write test harnesses for the fuzzer, so it'll fuzz whatever APIs you call from your test harnesses. Yeah, we choose what to do. I was elaborating on what we are going to choose. So I take it as you like the idea? Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://entangle-photo.org -o-http://search.cpan.org/~danberr/ :| signature.asc Description: Digital signature -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] Should Libvirt apply for OSS-Fuzz?
On Fri, Dec 02, 2016 at 10:14:22AM +0100, Martin Kletzander wrote: > On Fri, Dec 02, 2016 at 08:44:48AM +0100, Michal Privoznik wrote: > > Google announced OSS-Fuzz project [1]. It's aim is to test projects with > > significant user base and/or critical projects to the global > > infrastructure. I like to think that libvirt falls in both categories :-) > > You can find a list of already accepted projects here [2]. Once accepted > > to the project we would have to provide some scripts that build libvirt > > and run some tests. > > > > I was thinking about that too. And danpb would like that as well, I > guess, since he came up with the fuzzing idea for GSoC. > > > One of the disadvantages is that we have to provide a docker(!) image > > where the scripts would run from. > > > > But it's not like the whole libvirt has to be installed and running > there, right? It's unit-test fuzzing, it will just link against > libvirt.la and run random APIs (mostly public ones, I guess). You have to write test harnesses for the fuzzer, so it'll fuzz whatever APIs you call from your test harnesses. Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://entangle-photo.org -o-http://search.cpan.org/~danberr/ :| -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] Should Libvirt apply for OSS-Fuzz?
On Fri, Dec 02, 2016 at 08:44:48AM +0100, Michal Privoznik wrote: > Google announced OSS-Fuzz project [1]. It's aim is to test projects with > significant user base and/or critical projects to the global > infrastructure. I like to think that libvirt falls in both categories :-) > You can find a list of already accepted projects here [2]. Once accepted > to the project we would have to provide some scripts that build libvirt > and run some tests. > > One of the disadvantages is that we have to provide a docker(!) image > where the scripts would run from. > > What are your thoughts on this? Should we apply? I think we'd certainly benefit from some fuzzing of libvirt / libvirtd. I was a little unclear on exactly how they're doing the fuzzing from those docs, but looking at some of the project's they're already got it seems they rely on providing some custom test programs that call the APIs. eg https://github.com/google/oss-fuzz/blob/master/projects/libyaml/libyaml_fuzzer.cc https://github.com/google/oss-fuzz/blob/master/projects/libxml2/libxml2_xml_read_memory_fuzzer.cc https://github.com/google/oss-fuzz/blob/master/projects/libxml2/libxml2_xml_regexp_compile_fuzzer.cc OpenSSL is a little different as it seems they've natively integrated support for libfuzz in their code base, so google merely turns that on at configure stage https://github.com/google/oss-fuzz/blob/master/projects/openssl/build.sh So someone is going to need to write the test harness to actually exercise libvirt via the fuzzer still, which is work we've long had an outstanding todo for - its even a GSoC suggestion https://wiki.libvirt.org/page/Google_Summer_of_Code_2017#QEMU_command_line_generator_XML_fuzzing Once that's done we could leverage google's infra to actually run it. The XML parser is only half the story though - it'd be nice to get fuzzing on the main public APIs too (and thus the remote RPC protocol handling and QEMU driver code). Writing fuzzer test harnesses for all the public APIs is an even bigger bit of work Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://entangle-photo.org -o-http://search.cpan.org/~danberr/ :| -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] Should Libvirt apply for OSS-Fuzz?
On Fri, Dec 02, 2016 at 08:44:48AM +0100, Michal Privoznik wrote: Google announced OSS-Fuzz project [1]. It's aim is to test projects with significant user base and/or critical projects to the global infrastructure. I like to think that libvirt falls in both categories :-) You can find a list of already accepted projects here [2]. Once accepted to the project we would have to provide some scripts that build libvirt and run some tests. I was thinking about that too. And danpb would like that as well, I guess, since he came up with the fuzzing idea for GSoC. One of the disadvantages is that we have to provide a docker(!) image where the scripts would run from. But it's not like the whole libvirt has to be installed and running there, right? It's unit-test fuzzing, it will just link against libvirt.la and run random APIs (mostly public ones, I guess). What are your thoughts on this? Should we apply? I was already reading up on it when you sent the mail, so I'd say yes. Michal 1: https://opensource.googleblog.com/2016/12/announcing-oss-fuzz-continuous-fuzzing.html 2: https://github.com/google/oss-fuzz -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list signature.asc Description: Digital signature -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
