Re: [Development] Submitting Qt to oss-fuzz
Hi everybody, the Qt project is now registered in oss-fuzz's list of projects: https://github.com/google/oss-fuzz/tree/master/projects What I have so far: - fuzzing Qt with libFuzzer locally - registered Qt project for oss-fuzz What I don't have so far: - setting up build/run on Google's servers - finding out what kind of dashboard I'll get there I'll send updates as soon as I know more... Cheers, Robert -- Robert Löhning, Software Engineer - The Qt Company GmbH The Qt Company GmbH, Rudower Chaussee 13, D-12489 Berlin Geschäftsführer: Mika Pälsi, Juha Varelius, Mika Harjuaho Sitz der Gesellschaft: Berlin, Registergericht: Amtsgericht Charlottenburg, HRB 144331 B Am 31.08.2018 um 11:24 schrieb Robert Löhning: > Hi everybody, > > yes I did. So far I have been working on getting Qt into a better shape > for fuzzing at all, resulting in [1]. > > This prepares a Qt build for being fuzz tested with clang's libFuzzer, > the tool that Google also uses in oss-fuzz. The fuzzer I used for > testing my setup already found a crash. > > What I have so far: > - fuzzing Qt with libFuzzer locally, using [1] > - AFAICS collected all the needed "OK"s to enter Qt [2] > > What I don't have so far: > - create the pull request for [2], wanted to do this now-ish > - Everything that comes after registering the project like: > - setting up build/run on Google's servers > - finding out what kind of dashboard I'll get there > > I would appreciate if I might use the scripts you posted, Albert, or if > we could work on this together. > > Cheers, > Robert > > [1] https://codereview.qt-project.org/236937/ > [2] https://github.com/google/oss-fuzz/compare/master...rlohning:master > > Am 30.08.2018 um 21:27 schrieb Lars Knoll: >> Hi Albert, >> >> Nice! Robert has been working on exactly the same thing lately. I >> think it would be good if you guys coordinated the effort :) >> >> It would be ideal, if we could somehow get those mails forwarded to >> the security mailing list. I wonder whether we could do that with a >> special mail account that forwards to the security mailing list. >> >> Cheers, >> Lars >> >>> On 30 Aug 2018, at 20:42, Albert Astals Cid via Development >>> wrote: >>> >>> oss-fuzz is an online fuzzing service run by Google. >>> >>> They test daily the code base and run fuzzying over it, maintaining a >>> list of open and closed bugs. >>> >>> As example you can see one of the poppler issues i fixed at >>> https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=9382 >>> >>> Everything is done automatically by a bot, except my "This is fixed >>> by" comment, but that's just there for historical reasons, it's not >>> really needed. >>> >>> Found bugs are sent to a list of trusted address and kept private for >>> 90 days, then if not fixed then they become public. >>> >>> Fixed bugs become public 30 days after being fixed. >>> >>> I have made a qimage fuzzer that uses libpng test files as seed corpus. >>> >>> You can find it at https://github.com/albert-astals-cid-kdab/oss-fuzz >>> >>> Adding support for Qt is "relatively simple" see >>> https://github.com/albert-astals-cid-kdab/oss-fuzz/commit/2df60c7af6619b8a6a44b1cd679bf356e6e6ed3f >>> >>> >>> >>> I made a local test run of the undefined sanitizer and it found >>> https://paste.kde.org/prkox41mx >>> in a few seconds, so "it works" >>> >>> If you want to test it locally you can do >>> python infra/helper.py build_fuzzers --sanitizer undefined qt >>> python infra/helper.py run_fuzzer qt qimage_fuzzer >>> for the undefined sanitizer and >>> python infra/helper.py build_fuzzers --sanitizer address qt >>> python infra/helper.py run_fuzzer qt qimage_fuzzer >>> >>> Unfortunately I have not been able to compile with the memory >>> sanitizer enabled yet. >>> >>> The most important thing before submitting this upstream is changing >>> the list of trusted addresses the private bugs get sent to. >>> >>> To have something written i've used my email address but i guess at >>> least i should add eirik.aavitsl...@qt.io (listed as QImage >>> maintainer) there too? Anyone else? >>> >>> I am not sure how the email address thing works, but i think they >>> need to be "google account" activated, whatever that means, so we >>> can't use secur...@qt-project.org. On poppler i'm using my @gmail.com >>> address and not my @kde.org address since it was just easier. >>> >>> Comments? >>> >>> Cheers, >>> Albert >>> >>> -- >>> Albert Astals Cid | albert.astals@kdab.com | Software Engineer >>> Klarälvdalens Datakonsult AB, a KDAB Group company >>> Tel: Sweden (HQ) +46-563-540090, USA +1-866-777-KDAB(5322) >>> KDAB - The Qt, C++ and OpenGL Experts >>> >>> ___ >>> Development mailing list >>> Development@qt-project.org >>> http://lists.qt-project.org/mailman/listinfo/development >> > > ___ > Development mailing list >
Re: [Development] Submitting Qt to oss-fuzz
Hello Robert, On 05.09.2018 13:08, Robert Löhning wrote: > if (x == 0) > foo() > else > bar() > > Then all values of x will be considered different. For fuzzing, though, > only the two cases x == 0 and x != 0 will matter. > > Do I miss something here? yes I think it would consider all comparisons different; from what I understand it gives the fuzzer a better understanding of which part of the input needs to be altered to follow a different code path. Anyhow the libFuzzer documentation at https://llvm.org/docs/LibFuzzer.html says: "This may slow down the fuzzing but is very likely to improve the results." Regards, Peter -- Peter Hartmann // Titurelstrasse 2 // 89125 Munich // Germany pe...@hartmann.tk www.peter.hartmann.tk ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Submitting Qt to oss-fuzz
Am 05.09.2018 um 12:40 schrieb Peter Hartmann: > Hello Robert, > > On 04.09.2018 14:03, Robert Löhning wrote: >> I saw your post and it helped me get started. Thanks for that! > > cool, thanks for the feedback! > >> >> By the way: In step 1 you use >> -fsanitize-coverage=edge >> whereas I followed LLVM's example [1] and used >> -fsanitize-coverage=trace-pc-guard >> >> Unfortunately the documentation seems to be a bit scarce. Could you >> please tell me briefly about the advantages of either one? >> >> I guess I should include "edge" to [2], shouldn't I? Instead of >> hacking the mkspec you could then just use >> /configure -sanitize address -coverage edge > > The "-fsanitize-coverage=edge" just comes from an older clang version > when there was no trace-pc-guard. Nowadays "trace-pc-guard" without > other args implies "edge", so the latter can be ignored when configuring > with address sanitizer. > > IIUC edge just provided information on whether a piece of code was > executed or not, while trace-pc-guard provides a callback which allows > for more fine-grained coverage information; the callback itself is then > implemented in libFuzzer. > > What might be interesting to look at is "-fsanitize-coverage=trace-cmp" > though... Definitely interesting, but will it be useful for fuzzing? If I get this right, it will record all different values at comparisons. So, if we take a simple example: if (x == 0) foo() else bar() Then all values of x will be considered different. For fuzzing, though, only the two cases x == 0 and x != 0 will matter. Do I miss something here? Cheers, Robert ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Submitting Qt to oss-fuzz
Hello Robert, On 04.09.2018 14:03, Robert Löhning wrote: > I saw your post and it helped me get started. Thanks for that! cool, thanks for the feedback! > > By the way: In step 1 you use > -fsanitize-coverage=edge > whereas I followed LLVM's example [1] and used > -fsanitize-coverage=trace-pc-guard > > Unfortunately the documentation seems to be a bit scarce. Could you > please tell me briefly about the advantages of either one? > > I guess I should include "edge" to [2], shouldn't I? Instead of > hacking the mkspec you could then just use > /configure -sanitize address -coverage edge The "-fsanitize-coverage=edge" just comes from an older clang version when there was no trace-pc-guard. Nowadays "trace-pc-guard" without other args implies "edge", so the latter can be ignored when configuring with address sanitizer. IIUC edge just provided information on whether a piece of code was executed or not, while trace-pc-guard provides a callback which allows for more fine-grained coverage information; the callback itself is then implemented in libFuzzer. What might be interesting to look at is "-fsanitize-coverage=trace-cmp" though... Regards, Peter -- Peter Hartmann // Titurelstrasse 2 // 89125 Munich // Germany pe...@hartmann.tk www.peter.hartmann.tk ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Submitting Qt to oss-fuzz
Peter Hartmann (4 September 2018 11:17) > sounds like a good initiative, I was asking about the same thing 2 > years ago ([1]) but then somehow didn't follow up on this. > > Back then I also wrote some simple fuzzing test cases ([2]) that found > some crashes and memory corruptions ([3]), I would be happy to > contribute them if they are deemed useful. > > [1] > http://lists.qt-project.org/pipermail/development/2016-December/028016.html > [2] https://github.com/peter-ha/qt-fuzzing > [3] > https://www.peter.hartmann.tk/single-post/2016/11/29/Fuzzing-Qt-with-libFuzzer I see a blank page at [3], but "View source" shows me the content; once de-HTML-ified, it became readable. The issues you describe sound like having fixes would be useful, yes. Eddy. ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Submitting Qt to oss-fuzz
Hello, sounds like a good initiative, I was asking about the same thing 2 years ago ([1]) but then somehow didn't follow up on this. Back then I also wrote some simple fuzzing test cases ([2]) that found some crashes and memory corruptions ([3]), I would be happy to contribute them if they are deemed useful. Regards, Peter [1] http://lists.qt-project.org/pipermail/development/2016-December/028016.html [2] https://github.com/peter-ha/qt-fuzzing [3] https://www.peter.hartmann.tk/single-post/2016/11/29/Fuzzing-Qt-with-libFuzzer On 31.08.2018 11:24, Robert Löhning wrote: > Hi everybody, > > yes I did. So far I have been working on getting Qt into a better > shape for fuzzing at all, resulting in [1]. > > This prepares a Qt build for being fuzz tested with clang's libFuzzer, > the tool that Google also uses in oss-fuzz. The fuzzer I used for > testing my setup already found a crash. > > What I have so far: > - fuzzing Qt with libFuzzer locally, using [1] > - AFAICS collected all the needed "OK"s to enter Qt [2] > > What I don't have so far: > - create the pull request for [2], wanted to do this now-ish > - Everything that comes after registering the project like: > - setting up build/run on Google's servers > - finding out what kind of dashboard I'll get there > > I would appreciate if I might use the scripts you posted, Albert, or > if we could work on this together. > > Cheers, > Robert > > [1] https://codereview.qt-project.org/236937/ > [2] https://github.com/google/oss-fuzz/compare/master...rlohning:master > > Am 30.08.2018 um 21:27 schrieb Lars Knoll: >> Hi Albert, >> >> Nice! Robert has been working on exactly the same thing lately. I >> think it would be good if you guys coordinated the effort :) >> >> It would be ideal, if we could somehow get those mails forwarded to >> the security mailing list. I wonder whether we could do that with a >> special mail account that forwards to the security mailing list. >> >> Cheers, >> Lars >> >>> On 30 Aug 2018, at 20:42, Albert Astals Cid via Development >>> wrote: >>> >>> oss-fuzz is an online fuzzing service run by Google. >>> >>> They test daily the code base and run fuzzying over it, maintaining >>> a list of open and closed bugs. >>> >>> As example you can see one of the poppler issues i fixed at >>> https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=9382 >>> >>> Everything is done automatically by a bot, except my "This is fixed >>> by" comment, but that's just there for historical reasons, it's not >>> really needed. >>> >>> Found bugs are sent to a list of trusted address and kept private >>> for 90 days, then if not fixed then they become public. >>> >>> Fixed bugs become public 30 days after being fixed. >>> >>> I have made a qimage fuzzer that uses libpng test files as seed corpus. >>> >>> You can find it at https://github.com/albert-astals-cid-kdab/oss-fuzz >>> >>> Adding support for Qt is "relatively simple" see >>> https://github.com/albert-astals-cid-kdab/oss-fuzz/commit/2df60c7af6619b8a6a44b1cd679bf356e6e6ed3f >>> >>> >>> I made a local test run of the undefined sanitizer and it found >>> https://paste.kde.org/prkox41mx >>> in a few seconds, so "it works" >>> >>> If you want to test it locally you can do >>> python infra/helper.py build_fuzzers --sanitizer undefined qt >>> python infra/helper.py run_fuzzer qt qimage_fuzzer >>> for the undefined sanitizer and >>> python infra/helper.py build_fuzzers --sanitizer address qt >>> python infra/helper.py run_fuzzer qt qimage_fuzzer >>> >>> Unfortunately I have not been able to compile with the memory >>> sanitizer enabled yet. >>> >>> The most important thing before submitting this upstream is changing >>> the list of trusted addresses the private bugs get sent to. >>> >>> To have something written i've used my email address but i guess at >>> least i should add eirik.aavitsl...@qt.io (listed as QImage >>> maintainer) there too? Anyone else? >>> >>> I am not sure how the email address thing works, but i think they >>> need to be "google account" activated, whatever that means, so we >>> can't use secur...@qt-project.org. On poppler i'm using my >>> @gmail.com address and not my @kde.org address since it was just >>> easier. >>> >>> Comments? >>> >>> Cheers, >>> Albert >>> >>> -- >>> Albert Astals Cid | albert.astals@kdab.com | Software Engineer >>> Klarälvdalens Datakonsult AB, a KDAB Group company >>> Tel: Sweden (HQ) +46-563-540090, USA +1-866-777-KDAB(5322) >>> KDAB - The Qt, C++ and OpenGL Experts >>> >>> ___ >>> Development mailing list >>> Development@qt-project.org >>> http://lists.qt-project.org/mailman/listinfo/development >> > > ___ > Development mailing list > Development@qt-project.org > http://lists.qt-project.org/mailman/listinfo/development -- Peter Hartmann // Titurelstrasse 2 // 89125 Munich // Germany pe...@hartmann.tk www.peter.hartmann.tk
Re: [Development] Submitting Qt to oss-fuzz
El divendres, 31 d’agost de 2018, a les 11:24:58 CEST, Robert Löhning va escriure: > Hi everybody, > > yes I did. So far I have been working on getting Qt into a better shape > for fuzzing at all, resulting in [1]. > > This prepares a Qt build for being fuzz tested with clang's libFuzzer, > the tool that Google also uses in oss-fuzz. The fuzzer I used for > testing my setup already found a crash. > > What I have so far: > - fuzzing Qt with libFuzzer locally, using [1] > - AFAICS collected all the needed "OK"s to enter Qt [2] > > What I don't have so far: > - create the pull request for [2], wanted to do this now-ish > - Everything that comes after registering the project like: >- setting up build/run on Google's servers >- finding out what kind of dashboard I'll get there > > I would appreciate if I might use the scripts you posted, Albert, or if > we could work on this together. Feel free to merge my github branch into your github branch :) Cheers, Albert > > Cheers, > Robert > > [1] https://codereview.qt-project.org/236937/ > [2] https://github.com/google/oss-fuzz/compare/master...rlohning:master > > Am 30.08.2018 um 21:27 schrieb Lars Knoll: > > Hi Albert, > > > > Nice! Robert has been working on exactly the same thing lately. I think it > > would be good if you guys coordinated the effort :) > > > > It would be ideal, if we could somehow get those mails forwarded to the > > security mailing list. I wonder whether we could do that with a special > > mail account that forwards to the security mailing list. > > > > Cheers, > > Lars > > > >> On 30 Aug 2018, at 20:42, Albert Astals Cid via Development > >> wrote: > >> > >> oss-fuzz is an online fuzzing service run by Google. > >> > >> They test daily the code base and run fuzzying over it, maintaining a > >> list of open and closed bugs. > >> > >> As example you can see one of the poppler issues i fixed at > >> > >> https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=9382 > >> > >> Everything is done automatically by a bot, except my "This is fixed by" > >> comment, but that's just there for historical reasons, it's not really > >> needed. > >> > >> Found bugs are sent to a list of trusted address and kept private for 90 > >> days, then if not fixed then they become public. > >> > >> Fixed bugs become public 30 days after being fixed. > >> > >> I have made a qimage fuzzer that uses libpng test files as seed corpus. > >> > >> You can find it at https://github.com/albert-astals-cid-kdab/oss-fuzz > >> > >> Adding support for Qt is "relatively simple" see > >> https://github.com/albert-astals-cid-kdab/oss-fuzz/commit/2df60c7af6619b8 > >> a6a44b1cd679bf356e6e6ed3f > >> > >> I made a local test run of the undefined sanitizer and it found > >> https://paste.kde.org/prkox41mx > >> in a few seconds, so "it works" > >> > >> If you want to test it locally you can do > >> > >> python infra/helper.py build_fuzzers --sanitizer undefined qt > >> python infra/helper.py run_fuzzer qt qimage_fuzzer > >> > >> for the undefined sanitizer and > >> > >> python infra/helper.py build_fuzzers --sanitizer address qt > >> python infra/helper.py run_fuzzer qt qimage_fuzzer > >> > >> Unfortunately I have not been able to compile with the memory sanitizer > >> enabled yet. > >> > >> The most important thing before submitting this upstream is changing the > >> list of trusted addresses the private bugs get sent to. > >> > >> To have something written i've used my email address but i guess at least > >> i should add eirik.aavitsl...@qt.io (listed as QImage maintainer) there > >> too? Anyone else? > >> > >> I am not sure how the email address thing works, but i think they need to > >> be "google account" activated, whatever that means, so we can't use > >> secur...@qt-project.org. On poppler i'm using my @gmail.com address and > >> not my @kde.org address since it was just easier. > >> > >> Comments? > >> > >> Cheers, > >> > >> Albert > >> > >> ___ > >> Development mailing list > >> Development@qt-project.org > >> http://lists.qt-project.org/mailman/listinfo/development -- Albert Astals Cid | albert.astals@kdab.com | Software Engineer Klarälvdalens Datakonsult AB, a KDAB Group company Tel: Sweden (HQ) +46-563-540090, USA +1-866-777-KDAB(5322) KDAB - The Qt, C++ and OpenGL Experts ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Submitting Qt to oss-fuzz
El divendres, 31 d’agost de 2018, a les 11:31:16 CEST, Robert Löhning va escriure: > Am 30.08.2018 um 21:30 schrieb Albert Astals Cid via Development: > > El dijous, 30 d’agost de 2018, a les 8:59:40 CEST, André Pönitz va escriure: > >> On Thu, Aug 30, 2018 at 08:42:11PM +0200, Albert Astals Cid via > >> > >> Development wrote: > >>> I made a local test run of the undefined sanitizer and it found > >>> https://paste.kde.org/prkox41mx in a few seconds, so "it works" > >>> > >>> If you want to test it locally you can do python infra/helper.py > >>> build_fuzzers --sanitizer undefined qt python infra/helper.py > >>> run_fuzzer qt qimage_fuzzer for the undefined sanitizer and > >>> python infra/helper.py build_fuzzers --sanitizer address qt > >>> python infra/helper.py run_fuzzer qt qimage_fuzzer > >>> > >>> Unfortunately I have not been able to compile with the memory > >>> sanitizer enabled yet. > >>> > >>> The most important thing before submitting this upstream is > >>> changing the list of trusted addresses the private bugs get sent > >>> to. > >>> > >>> To have something written i've used my email address but i guess > >>> at least i should add eirik.aavitsl...@qt.io (listed as QImage > >>> maintainer) there too? Anyone else? I am not sure how the email > >>> address thing works, but i think they need to be "google account" > >>> activated, whatever that means, so we can't use > >>> secur...@qt-project.org. > >> > >> That would be the natural choice. > >> > >>> On poppler i'm using my @gmail.com address and not my @kde.org address > >>> since it was just easier. > >>> > >>> Comments? > >> > >> We are not taking about an innovative approach to coerce people > >> into using Google services, right? > > > > Maybe :D > > > > Not really sure how it works, we can try submitting it with security@qt- > > project.org and see what happens, but first i'd like confirmation from > > them > > that they'll look at the errors and confirmation from "the project" that > > it's a good idea to do this. > > Hi, > > I was planning to do it the other way round: I registered a GMail > address for this sole purpose and will manually forward what comes in > there to the security list whenever needed. Of course I'd then try to > automate this as far as possible. That works for me if it works for the project :) Cheers, Albert > > Cheers, > Robert > > > Cheers, > > > >Albert > >> > >> Andre' > > ___ > Development mailing list > Development@qt-project.org > http://lists.qt-project.org/mailman/listinfo/development -- Albert Astals Cid | albert.astals@kdab.com | Software Engineer Klarälvdalens Datakonsult AB, a KDAB Group company Tel: Sweden (HQ) +46-563-540090, USA +1-866-777-KDAB(5322) KDAB - The Qt, C++ and OpenGL Experts ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Submitting Qt to oss-fuzz
Am 30.08.2018 um 21:30 schrieb Albert Astals Cid via Development: El dijous, 30 d’agost de 2018, a les 8:59:40 CEST, André Pönitz va escriure: On Thu, Aug 30, 2018 at 08:42:11PM +0200, Albert Astals Cid via Development wrote: I made a local test run of the undefined sanitizer and it found https://paste.kde.org/prkox41mx in a few seconds, so "it works" If you want to test it locally you can do python infra/helper.py build_fuzzers --sanitizer undefined qt python infra/helper.py run_fuzzer qt qimage_fuzzer for the undefined sanitizer and python infra/helper.py build_fuzzers --sanitizer address qt python infra/helper.py run_fuzzer qt qimage_fuzzer Unfortunately I have not been able to compile with the memory sanitizer enabled yet. The most important thing before submitting this upstream is changing the list of trusted addresses the private bugs get sent to. To have something written i've used my email address but i guess at least i should add eirik.aavitsl...@qt.io (listed as QImage maintainer) there too? Anyone else? I am not sure how the email address thing works, but i think they need to be "google account" activated, whatever that means, so we can't use secur...@qt-project.org. That would be the natural choice. On poppler i'm using my @gmail.com address and not my @kde.org address since it was just easier. Comments? We are not taking about an innovative approach to coerce people into using Google services, right? Maybe :D Not really sure how it works, we can try submitting it with security@qt- project.org and see what happens, but first i'd like confirmation from them that they'll look at the errors and confirmation from "the project" that it's a good idea to do this. Hi, I was planning to do it the other way round: I registered a GMail address for this sole purpose and will manually forward what comes in there to the security list whenever needed. Of course I'd then try to automate this as far as possible. Cheers, Robert Cheers, Albert Andre' ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Submitting Qt to oss-fuzz
Hi everybody, yes I did. So far I have been working on getting Qt into a better shape for fuzzing at all, resulting in [1]. This prepares a Qt build for being fuzz tested with clang's libFuzzer, the tool that Google also uses in oss-fuzz. The fuzzer I used for testing my setup already found a crash. What I have so far: - fuzzing Qt with libFuzzer locally, using [1] - AFAICS collected all the needed "OK"s to enter Qt [2] What I don't have so far: - create the pull request for [2], wanted to do this now-ish - Everything that comes after registering the project like: - setting up build/run on Google's servers - finding out what kind of dashboard I'll get there I would appreciate if I might use the scripts you posted, Albert, or if we could work on this together. Cheers, Robert [1] https://codereview.qt-project.org/236937/ [2] https://github.com/google/oss-fuzz/compare/master...rlohning:master Am 30.08.2018 um 21:27 schrieb Lars Knoll: Hi Albert, Nice! Robert has been working on exactly the same thing lately. I think it would be good if you guys coordinated the effort :) It would be ideal, if we could somehow get those mails forwarded to the security mailing list. I wonder whether we could do that with a special mail account that forwards to the security mailing list. Cheers, Lars On 30 Aug 2018, at 20:42, Albert Astals Cid via Development wrote: oss-fuzz is an online fuzzing service run by Google. They test daily the code base and run fuzzying over it, maintaining a list of open and closed bugs. As example you can see one of the poppler issues i fixed at https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=9382 Everything is done automatically by a bot, except my "This is fixed by" comment, but that's just there for historical reasons, it's not really needed. Found bugs are sent to a list of trusted address and kept private for 90 days, then if not fixed then they become public. Fixed bugs become public 30 days after being fixed. I have made a qimage fuzzer that uses libpng test files as seed corpus. You can find it at https://github.com/albert-astals-cid-kdab/oss-fuzz Adding support for Qt is "relatively simple" see https://github.com/albert-astals-cid-kdab/oss-fuzz/commit/2df60c7af6619b8a6a44b1cd679bf356e6e6ed3f I made a local test run of the undefined sanitizer and it found https://paste.kde.org/prkox41mx in a few seconds, so "it works" If you want to test it locally you can do python infra/helper.py build_fuzzers --sanitizer undefined qt python infra/helper.py run_fuzzer qt qimage_fuzzer for the undefined sanitizer and python infra/helper.py build_fuzzers --sanitizer address qt python infra/helper.py run_fuzzer qt qimage_fuzzer Unfortunately I have not been able to compile with the memory sanitizer enabled yet. The most important thing before submitting this upstream is changing the list of trusted addresses the private bugs get sent to. To have something written i've used my email address but i guess at least i should add eirik.aavitsl...@qt.io (listed as QImage maintainer) there too? Anyone else? I am not sure how the email address thing works, but i think they need to be "google account" activated, whatever that means, so we can't use secur...@qt-project.org. On poppler i'm using my @gmail.com address and not my @kde.org address since it was just easier. Comments? Cheers, Albert -- Albert Astals Cid | albert.astals@kdab.com | Software Engineer Klarälvdalens Datakonsult AB, a KDAB Group company Tel: Sweden (HQ) +46-563-540090, USA +1-866-777-KDAB(5322) KDAB - The Qt, C++ and OpenGL Experts ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Submitting Qt to oss-fuzz
El divendres, 31 d’agost de 2018, a les 10:27:08 CEST, Edward Welbourne va escriure: >> By "fixed" do they mean "we have told them we've fixed it" or "we've >> released all currently releasing branches of Qt with fixes" ? Albert Astals Cid (31 August 2018 10:52) > Fixed means "the daily bot has run again and it has found that what > was wrong before is now fine" OK, so that'll be shortly after we release an update to whatever branch they're testing. I suppose we have some say in which version they test, so we could start with LTS and work our way closer to the bleeding edge as we get all our old horrors out of the way - and maybe one day get to test live on dev. >> So it would be better to run this *ourselves*, if we can, so that the >> Qt community has more control over how and when the results get to be >> published. > This is scarily close to the security by obscurity argument ;) > > "what if we have an horrible bug, we fix it, it becomes public in 30 > days and we've not been able yet to put out a release?" > > My answer to that is, you had an horrible bug, it's fixed, that is a > great thing, so just put and advisory out with the patch if we can't > get a release out. Yet we have a security group, whose business is to manage the timing of advisories and co-ordinate those with releases. I'm not saying we should try to hide our dirty laundry; just that we should let our security team actually have a chance to have some control over the things they're there to control. >> So it *can* be used locally, without giving Google yet more power ... >> Good to know. > But you lose the daily bot runs and the free hardware. I am not sure, > but i think the bot part is not actually free software, though i may > be wrong. Also when i run it, it stops at the first found issue, i > guess there may be a parameter to have it continue since the bot will > find N issues in a given day. Indeed, running it ourselves would be One More Thing that the poor infrastructure team would have to take care of, and One More System to maintain; all the more so if we have to implement our own replacement for some non-free parts. So the question is whether the impedance mismatch - between Google's disclosure time-line (optimised for Chromium-style software that doesn't care about old versions or backwards-compatibility) and our security team's processes - is a big enough issue that it's worth going to all that effort ourselves ... I'm not saying "let's not do this" only "let's just think about this for a moment, first" - in particular, about how it'll interact with our existing security and release processes, Eddy. ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Submitting Qt to oss-fuzz
El divendres, 31 d’agost de 2018, a les 10:27:08 CEST, Edward Welbourne va escriure: > Albert Astals Cid (30 August 2018 20:42) wrote: > > oss-fuzz is an online fuzzing service run by Google. > > Sounds useful. > > > They test daily the code base and run fuzzying over it, maintaining a > > list of open and closed bugs. > > > > Found bugs are sent to a list of trusted address and kept private for > > 90 days, then if not fixed then they become public. > > > > Fixed bugs become public 30 days after being fixed. > > By "fixed" do they mean "we have told them we've fixed it" or "we've > released all currently releasing branches of Qt with fixes" ? Fixed means "the daily bot has run again and it has found that what was wrong before is now fine" > I'm > guessing it's closer to the former than the latter. So we have a month > from fixing it, or perhaps from releasing *one* branch with a fix, > within which to also release all our other live branches. That sounds > like it may stress our release processes. So we have a quarter year in > which to find a fix, then we need to orchestrate releases across all > branches within a month; and this happens for each and every issue > found. That schedule is fine for Chromium, which doesn't support old > versions or care about backwards-compatibility, but may be a poor fit > for our more conservative processes. > > So it would be better to run this *ourselves*, if we can, so that the Qt > community has more control over how and when the results get to be > published. This is scarily close to the security by obscurity argument ;) "what if we have an horrible bug, we fix it, it becomes public in 30 days and we've not been able yet to put out a release?" My answer to that is, you had an horrible bug, it's fixed, that is a great thing, so just put and advisory out with the patch if we can't get a release out. > > > If you want to test it locally you can do > > > > python infra/helper.py build_fuzzers --sanitizer undefined qt > > python infra/helper.py run_fuzzer qt qimage_fuzzer > > > > for the undefined sanitizer and > > > > python infra/helper.py build_fuzzers --sanitizer address qt > > python infra/helper.py run_fuzzer qt qimage_fuzzer > > So it *can* be used locally, without giving Google yet more power ... > Good to know. But you lose the daily bot runs and the free hardware. I am not sure, but i think the bot part is not actually free software, though i may be wrong. Also when i run it, it stops at the first found issue, i guess there may be a parameter to have it continue since the bot will find N issues in a given day. Cheers, Albert > > Eddy. -- Albert Astals Cid | albert.astals@kdab.com | Software Engineer Klarälvdalens Datakonsult AB, a KDAB Group company Tel: Sweden (HQ) +46-563-540090, USA +1-866-777-KDAB(5322) KDAB - The Qt, C++ and OpenGL Experts ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Submitting Qt to oss-fuzz
Albert Astals Cid (30 August 2018 20:42) wrote: > oss-fuzz is an online fuzzing service run by Google. Sounds useful. > They test daily the code base and run fuzzying over it, maintaining a > list of open and closed bugs. > > Found bugs are sent to a list of trusted address and kept private for > 90 days, then if not fixed then they become public. > > Fixed bugs become public 30 days after being fixed. By "fixed" do they mean "we have told them we've fixed it" or "we've released all currently releasing branches of Qt with fixes" ? I'm guessing it's closer to the former than the latter. So we have a month from fixing it, or perhaps from releasing *one* branch with a fix, within which to also release all our other live branches. That sounds like it may stress our release processes. So we have a quarter year in which to find a fix, then we need to orchestrate releases across all branches within a month; and this happens for each and every issue found. That schedule is fine for Chromium, which doesn't support old versions or care about backwards-compatibility, but may be a poor fit for our more conservative processes. So it would be better to run this *ourselves*, if we can, so that the Qt community has more control over how and when the results get to be published. > If you want to test it locally you can do > python infra/helper.py build_fuzzers --sanitizer undefined qt > python infra/helper.py run_fuzzer qt qimage_fuzzer > for the undefined sanitizer and > python infra/helper.py build_fuzzers --sanitizer address qt > python infra/helper.py run_fuzzer qt qimage_fuzzer So it *can* be used locally, without giving Google yet more power ... Good to know. Eddy. ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Submitting Qt to oss-fuzz
El dijous, 30 d’agost de 2018, a les 20:42:11 CEST, Albert Astals Cid via Development va escriure: > Unfortunately I have not been able to compile with the memory sanitizer > enabled yet. Done :) https://github.com/albert-astals-cid-kdab/oss-fuzz/commit/b4f28e7dc5e4b936166cda1be36c3bde6b62c53d Found https://paste.kde.org/ptyi8utax in under a minute. Cheers, Albert > > Cheers, > Albert -- Albert Astals Cid | albert.astals@kdab.com | Software Engineer Klarälvdalens Datakonsult AB, a KDAB Group company Tel: Sweden (HQ) +46-563-540090, USA +1-866-777-KDAB(5322) KDAB - The Qt, C++ and OpenGL Experts ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Submitting Qt to oss-fuzz
El dijous, 30 d’agost de 2018, a les 8:59:40 CEST, André Pönitz va escriure: > On Thu, Aug 30, 2018 at 08:42:11PM +0200, Albert Astals Cid via > > Development wrote: > > I made a local test run of the undefined sanitizer and it found > > https://paste.kde.org/prkox41mx in a few seconds, so "it works" > > > > If you want to test it locally you can do python infra/helper.py > > build_fuzzers --sanitizer undefined qt python infra/helper.py > > run_fuzzer qt qimage_fuzzer for the undefined sanitizer and > > python infra/helper.py build_fuzzers --sanitizer address qt > > python infra/helper.py run_fuzzer qt qimage_fuzzer > > > > Unfortunately I have not been able to compile with the memory > > sanitizer enabled yet. > > > > The most important thing before submitting this upstream is > > changing the list of trusted addresses the private bugs get sent > > to. > > > > To have something written i've used my email address but i guess > > at least i should add eirik.aavitsl...@qt.io (listed as QImage > > maintainer) there too? Anyone else? I am not sure how the email > > address thing works, but i think they need to be "google account" > > activated, whatever that means, so we can't use > > secur...@qt-project.org. > > That would be the natural choice. > > > On poppler i'm using my @gmail.com address and not my @kde.org address > > since it was just easier. > > > > Comments? > > We are not taking about an innovative approach to coerce people > into using Google services, right? Maybe :D Not really sure how it works, we can try submitting it with security@qt- project.org and see what happens, but first i'd like confirmation from them that they'll look at the errors and confirmation from "the project" that it's a good idea to do this. Cheers, Albert > > Andre' -- Albert Astals Cid | albert.astals@kdab.com | Software Engineer Klarälvdalens Datakonsult AB, a KDAB Group company Tel: Sweden (HQ) +46-563-540090, USA +1-866-777-KDAB(5322) KDAB - The Qt, C++ and OpenGL Experts ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Submitting Qt to oss-fuzz
Hi Albert, Nice! Robert has been working on exactly the same thing lately. I think it would be good if you guys coordinated the effort :) It would be ideal, if we could somehow get those mails forwarded to the security mailing list. I wonder whether we could do that with a special mail account that forwards to the security mailing list. Cheers, Lars > On 30 Aug 2018, at 20:42, Albert Astals Cid via Development > wrote: > > oss-fuzz is an online fuzzing service run by Google. > > They test daily the code base and run fuzzying over it, maintaining a list of > open and closed bugs. > > As example you can see one of the poppler issues i fixed at >https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=9382 > > Everything is done automatically by a bot, except my "This is fixed by" > comment, but that's just there for historical reasons, it's not really needed. > > Found bugs are sent to a list of trusted address and kept private for 90 > days, then if not fixed then they become public. > > Fixed bugs become public 30 days after being fixed. > > I have made a qimage fuzzer that uses libpng test files as seed corpus. > > You can find it at https://github.com/albert-astals-cid-kdab/oss-fuzz > > Adding support for Qt is "relatively simple" see > https://github.com/albert-astals-cid-kdab/oss-fuzz/commit/2df60c7af6619b8a6a44b1cd679bf356e6e6ed3f > > I made a local test run of the undefined sanitizer and it found > https://paste.kde.org/prkox41mx > in a few seconds, so "it works" > > If you want to test it locally you can do >python infra/helper.py build_fuzzers --sanitizer undefined qt >python infra/helper.py run_fuzzer qt qimage_fuzzer > for the undefined sanitizer and >python infra/helper.py build_fuzzers --sanitizer address qt >python infra/helper.py run_fuzzer qt qimage_fuzzer > > Unfortunately I have not been able to compile with the memory sanitizer > enabled yet. > > The most important thing before submitting this upstream is changing the list > of trusted addresses the private bugs get sent to. > > To have something written i've used my email address but i guess at least i > should add eirik.aavitsl...@qt.io (listed as QImage maintainer) there too? > Anyone else? > > I am not sure how the email address thing works, but i think they need to be > "google account" activated, whatever that means, so we can't use > secur...@qt-project.org. On poppler i'm using my @gmail.com address and not > my @kde.org address since it was just easier. > > Comments? > > Cheers, > Albert > > -- > Albert Astals Cid | albert.astals@kdab.com | Software Engineer > Klarälvdalens Datakonsult AB, a KDAB Group company > Tel: Sweden (HQ) +46-563-540090, USA +1-866-777-KDAB(5322) > KDAB - The Qt, C++ and OpenGL Experts > > ___ > Development mailing list > Development@qt-project.org > http://lists.qt-project.org/mailman/listinfo/development ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Submitting Qt to oss-fuzz
On Thu, Aug 30, 2018 at 08:42:11PM +0200, Albert Astals Cid via Development wrote: > I made a local test run of the undefined sanitizer and it found > https://paste.kde.org/prkox41mx in a few seconds, so "it works" > > If you want to test it locally you can do python infra/helper.py > build_fuzzers --sanitizer undefined qt python infra/helper.py > run_fuzzer qt qimage_fuzzer for the undefined sanitizer and > python infra/helper.py build_fuzzers --sanitizer address qt > python infra/helper.py run_fuzzer qt qimage_fuzzer > > Unfortunately I have not been able to compile with the memory > sanitizer enabled yet. > > The most important thing before submitting this upstream is > changing the list of trusted addresses the private bugs get sent > to. > > To have something written i've used my email address but i guess > at least i should add eirik.aavitsl...@qt.io (listed as QImage > maintainer) there too? Anyone else? I am not sure how the email > address thing works, but i think they need to be "google account" > activated, whatever that means, so we can't use > secur...@qt-project.org. That would be the natural choice. > On poppler i'm using my @gmail.com address and not my @kde.org address since > it was > just easier. > > Comments? We are not taking about an innovative approach to coerce people into using Google services, right? Andre' ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
