Would the undeclared runtime exceptions be "fixable" for the fuzzing tool if the methods declared their runtime exceptions being thrown? Or the javadocs? As in, this tool is looking for exceptional conditions that don't appear to be intentional?
I've briefly looked at OSS-Fuzz, and it certainly looks helpful. I think codec, compress, collections (probably), io, net, jelly, jexl, pool, and probably a few others would be interesting to see fuzzed (for different reasons; some for handling XML or other languages, others for concurrency issues, serializability, etc). On Wed, 14 Apr 2021 at 01:42, Fabian Meumertzheim <meumertzh...@code-intelligence.com> wrote: > > As I am not familiar with the structure of your mailing lists and also > can't give a meaningful estimate of the ratio of normal bugs to > security issues we will find, I will only provide the following > general points of information on OSS-Fuzz: > > * By design, fuzzing produces little to no false positives. Unless a > library maintains non-trivial global state that is not accounted for > in the fuzz target, all OSS-Fuzz findings should at least be fully > reproducible reports of normal bugs, including stack traces and > minimized reproducing inputs. > > * Deduplicating certain kinds of issues may be difficult to do > automatically, so some root causes may lead to multiple reports filed. > Since OSS-Fuzz automatically detects when a bug has been fixed > upstream, acting on one of the reports would still close all the > others without additional triage work. > > * OSS-Fuzz considers bugs and security issues fixed once it has > determined that they do no longer reproduce in the upstream project, > which happens roughly a day after the fix has been made in the git/svn > repository. After the first few bugs, most new bugs are usually found > in unreleased commits, where this wouldn't be an issue at all. In case > OSS-Fuzz finds a security issue that reproduces in a release, just > keep in mind that the "patch gap" between the fixing commit and the > point release should be kept as short as possible. > > * From my experience with OSS-Fuzz projects in other languages as well > as the first onboarded Java projects (e.g. various Jackson parsers), > you should expect to see a number of purely functional bugs (e.g. > undeclared runtime exceptions) over the first days before the fuzzer > gets to the more interesting bugs (e.g. OutOfMemoryError and infinite > loops). While it would be possible to just not report purely > functional bugs, I would not recommend to do this as it hurts the > overall fuzzer performance. > > * Depending on the project, we may be able to define additional, > domain-specific classes of security issues that the fuzzer could check > for. As an example, I'm pretty confident that OSS-Fuzz would have > found CVE-2021-29425 (Possible limited path traversal vulnerability in > Apache Commons IO) with a proper fuzz target. > > I'll monitor this thread in case you have any more questions on > OSS-Fuzz or its JVM fuzzer Jazzer. > > > Fabian > > On Wed, Apr 14, 2021 at 4:59 AM Bruno P. Kinoshita <ki...@apache.org> wrote: > > > > +1 for oss fuzz. Fabian also got in contact a few days earlier, and asked > > me about using it with Commons Imaging. I told him it had to be discussed > > here first, but that I thought it could be useful (we are parsing several > > image file formats, probably a few things could be improved). > > > > > > As for the mailing list, for me it depends on the amount of messages, and > > false-positives. i.e. if we get 50 e-mails in security@commons in one week, > > and turns out only 1 is actually a security issue, and the others are > > either normal bugs and no bugs, then eventually I think I'd just create a > > filter to move all the security@commons to a folder and have a look someday. > > > > > > I think we don't have any idea how many e-mails we might get enabling it > > for one or for a few components. So I'd be OK with > > > > - sending e-mails to security@commons initially, but if it spams the list > > with non-security related e-mails, then move to a separate mailing list; OR > > - create the new mailing list (probably private too? until we filter the > > issues?) and use it for a few weeks/months. If the traffic is low, or most > > issues are really security related, then move to security@commons if others > > agree > > > > Either way would be OK for me. > > > > Cheers > > Bruno > > > > > > On Wednesday, 14 April 2021, 4:49:31 am NZST, Stefan Bodewig > > <bode...@apache.org> wrote: > > > > Hi all > > > > I want to pick up (and finish) the discussion that started in > > Compress[1]. > > > > Short Recap: > > ============ > > > > OSS Fuzz[2] runs fuzz testing for open source projects by invoking > > methods of our code with random data looking for unexpected outcomes > > (undeclared exceptions or worse code that never returns because it is > > stuck in an infinite loop for example). > > > > For Compress Fabian (who started [1]) has already identified and > > reported several issues, one of which would have become a CVE if the > > code in question had been part of any release of Compress. In the past > > other people have run different fuzzers and found "interesting" results > > in Compress as well. > > > > Compress may be especially vulnerable as it basically tries to make > > sense out of a bunch of user supplied bytes - but the same is probably > > true for codec or imaging for example. > > > > Fabian has offered to set up OSS Fuzz for Compress. Given that the > > issues OSS Fuzz detects may or may not be security sensitive, I don't > > feel it would be a good idea to have the tool send reports to a public > > mailing list. Therefore I propose to create another subscription > > moderated list just for these kinds of reports. I'm afraid it could be > > too noisy for security@commons. > > > > Proposal > > ======== > > > > Unless anybody objects until then I will create such a list (I believe > > there is a self-service thingy for that, otherwise I'll ask the infra > > folks) on the coming Sunday. I'd add myself as a moderator but we will > > need more moderators. Also I'll gladly accept ideas for the name of the > > list. > > > > If there are objections against yet another mailing list I'll ask Fabian > > to set things up using a private mail alias. If you want to receive the > > messages as well, please tell me. > > > > Cheers > > > > Stefan > > > > [1] > > https://lists.apache.org/thread.html/rb34ea7d9272b8e600437ea705b13aba1bcc2f23ceb55880bce27e479%40%3Cdev.commons.apache.org%3E > > > > [2] https://google.github.io/oss-fuzz/ > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org > > For additional commands, e-mail: dev-h...@commons.apache.org > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org > For additional commands, e-mail: dev-h...@commons.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org For additional commands, e-mail: dev-h...@commons.apache.org
