El dimecres, 6 de març de 2019, a les 20:20:03 CET, alcinos va escriure: > Hello, > > As some of you might have heard, we have been rewriting significantly some > key components of Kdenlive over the past couple years, in an effort to > improve stability, testability, and separation of concerns between the view > and the model. > > As part of this effort, on top of traditional unit-tests (none existed > before the rewrite), I have been looking at fuzzing as a way to push the > code to its limit. For those who are not familiar with it, fuzzing is a > testing technique that essentially consists in generating a lot of random > inputs and feeding them to a target program in an attempt to generate > unintended behaviours (memoryleak, crash,...). This technique has an > excellent track record of finding bugs and vulnerabilites, even in well > tested softwares (see eg. https://llvm.org/docs/LibFuzzer.html#trophies) > > In the case of Kdenlive, we generate a lot of operations that the user > could execute on the timeline, and simulate them in the model, to check for > robustness over unexpected action sequences. > Note that this kind of testing doesn't explicitly test for correctness (it > doesn't check whether a clip is correctly moved after a move operation for > example), but try to get the software to crash, if there is a way to make > that happen. Also, we have some internal checking mechanisms that > periodically ensure that the model is in a sane state, and the invariants > are respected. So if things go wrong with respect to that, it will also be > caught by the fuzzer. > > This approach has already led to the discovery of several bugs in kdenlive, > although the parts covered by the fuzzer are still fairly limited. > > Are any other projects in KDE using that technique? In order for it to be > useful, it pretty much needs to be run continuously (I've been running it a > bit on my laptop, but it's not very practical). Some companies offer such > service for free for OSS, under some conditions: see for eg: > https://github.com/google/oss-fuzz and https://fuzzbuzz.io/pricing, > although I'm not sure whether Kdenlive would meet the requirements for > either of those. Is there any way we could do this inside the KDE > infrastructure instead?
We (KDE Security team, but here basically just me) are running kcodecs and kimageformats in oss-fuzz. I can help you with the setup if you want. Cheers, Albert > Ideally, this are the requirements for such a > service: > > - Continuous running > - Git hook to get the latest code > - Automatic bug deduplication and reporting > - (optional) Coverage report to ensure the fuzzing is effective. > > Google open-sourced most of its tooling for fuzzing at scale on a cluster, > so there is no need to start from scratch. > > What are your thoughts? > > Best regards, > Nicolas Carion >
