On Wed, Apr 14, 2021, 17:14 Matt Sicker <boa...@gmail.com> wrote:
> Would the undeclared runtime exceptions be "fixable" for the fuzzing
> tool if the methods declared their runtime exceptions being thrown? Or
> the javadocs? As in, this tool is looking for exceptional conditions
> that don't appear to be intentional?
>
It is, but you have to tell it what is intentional and how to invoke your
API in interesting ways by supplying a fuzz target. As an example, the
following fuzz target found the infinite loop in Compress:
public static void fuzzerTestOneInput(byte[] input) {
try {
new TarFile(input).close();
} catch (IOException ignored) {}
}
OSS-Fuzz integration entails writing these fuzz targets, which can be
straightforward (as for compress) or a bit more involved (e.g. finding the
right properties to check for collections). Think of these fuzz targets as
property-based unit tests for which you don't have to supply the test data
yourself.
> I've briefly looked at OSS-Fuzz, and it certainly looks helpful. I
> think codec, compress, collections (probably), io, net, jelly, jexl,
> pool, and probably a few others would be interesting to see fuzzed
> (for different reasons; some for handling XML or other languages,
> others for concurrency issues, serializability, etc).
>
> On Wed, 14 Apr 2021 at 01:42, Fabian Meumertzheim
> <meumertzh...@code-intelligence.com> wrote:
> >
> > As I am not familiar with the structure of your mailing lists and also
> > can't give a meaningful estimate of the ratio of normal bugs to
> > security issues we will find, I will only provide the following
> > general points of information on OSS-Fuzz:
> >
> > * By design, fuzzing produces little to no false positives. Unless a
> > library maintains non-trivial global state that is not accounted for
> > in the fuzz target, all OSS-Fuzz findings should at least be fully
> > reproducible reports of normal bugs, including stack traces and
> > minimized reproducing inputs.
> >
> > * Deduplicating certain kinds of issues may be difficult to do
> > automatically, so some root causes may lead to multiple reports filed.
> > Since OSS-Fuzz automatically detects when a bug has been fixed
> > upstream, acting on one of the reports would still close all the
> > others without additional triage work.
> >
> > * OSS-Fuzz considers bugs and security issues fixed once it has
> > determined that they do no longer reproduce in the upstream project,
> > which happens roughly a day after the fix has been made in the git/svn
> > repository. After the first few bugs, most new bugs are usually found
> > in unreleased commits, where this wouldn't be an issue at all. In case
> > OSS-Fuzz finds a security issue that reproduces in a release, just
> > keep in mind that the "patch gap" between the fixing commit and the
> > point release should be kept as short as possible.
> >
> > * From my experience with OSS-Fuzz projects in other languages as well
> > as the first onboarded Java projects (e.g. various Jackson parsers),
> > you should expect to see a number of purely functional bugs (e.g.
> > undeclared runtime exceptions) over the first days before the fuzzer
> > gets to the more interesting bugs (e.g. OutOfMemoryError and infinite
> > loops). While it would be possible to just not report purely
> > functional bugs, I would not recommend to do this as it hurts the
> > overall fuzzer performance.
> >
> > * Depending on the project, we may be able to define additional,
> > domain-specific classes of security issues that the fuzzer could check
> > for. As an example, I'm pretty confident that OSS-Fuzz would have
> > found CVE-2021-29425 (Possible limited path traversal vulnerability in
> > Apache Commons IO) with a proper fuzz target.
> >
> > I'll monitor this thread in case you have any more questions on
> > OSS-Fuzz or its JVM fuzzer Jazzer.
> >
> >
> > Fabian
> >
> > On Wed, Apr 14, 2021 at 4:59 AM Bruno P. Kinoshita <ki...@apache.org>
> wrote:
> > >
> > > +1 for oss fuzz. Fabian also got in contact a few days earlier, and
> asked me about using it with Commons Imaging. I told him it had to be
> discussed here first, but that I thought it could be useful (we are parsing
> several image file formats, probably a few things could be improved).
> > >
> > >
> > > As for the mailing list, for me it depends on the amount of messages,
> and false-positives. i.e. if we get 50 e-mails in security@commons in one
> week, and turns out only 1 is actually a security issue, and the others are
> either normal bugs and no bugs, then eventually I think I'd just create a
> filter to move all the security@commons to a folder and have a look
> someday.
> > >
> > >
> > > I think we don't have any idea how many e-mails we might get enabling
> it for one or for a few components. So I'd be OK with
> > >
> > > - sending e-mails to security@commons initially, but if it spams the
> list with non-security related e-mails, then move to a separate mailing
> list; OR
> > > - create the new mailing list (probably private too? until we filter
> the issues?) and use it for a few weeks/months. If the traffic is low, or
> most issues are really security related, then move to security@commons if
> others agree
> > >
> > > Either way would be OK for me.
> > >
> > > Cheers
> > > Bruno
> > >
> > >
> > > On Wednesday, 14 April 2021, 4:49:31 am NZST, Stefan Bodewig <
> bode...@apache.org> wrote:
> > >
> > > Hi all
> > >
> > > I want to pick up (and finish) the discussion that started in
> > > Compress[1].
> > >
> > > Short Recap:
> > > ============
> > >
> > > OSS Fuzz[2] runs fuzz testing for open source projects by invoking
> > > methods of our code with random data looking for unexpected outcomes
> > > (undeclared exceptions or worse code that never returns because it is
> > > stuck in an infinite loop for example).
> > >
> > > For Compress Fabian (who started [1]) has already identified and
> > > reported several issues, one of which would have become a CVE if the
> > > code in question had been part of any release of Compress. In the past
> > > other people have run different fuzzers and found "interesting" results
> > > in Compress as well.
> > >
> > > Compress may be especially vulnerable as it basically tries to make
> > > sense out of a bunch of user supplied bytes - but the same is probably
> > > true for codec or imaging for example.
> > >
> > > Fabian has offered to set up OSS Fuzz for Compress. Given that the
> > > issues OSS Fuzz detects may or may not be security sensitive, I don't
> > > feel it would be a good idea to have the tool send reports to a public
> > > mailing list. Therefore I propose to create another subscription
> > > moderated list just for these kinds of reports. I'm afraid it could be
> > > too noisy for security@commons.
> > >
> > > Proposal
> > > ========
> > >
> > > Unless anybody objects until then I will create such a list (I believe
> > > there is a self-service thingy for that, otherwise I'll ask the infra
> > > folks) on the coming Sunday. I'd add myself as a moderator but we will
> > > need more moderators. Also I'll gladly accept ideas for the name of the
> > > list.
> > >
> > > If there are objections against yet another mailing list I'll ask
> Fabian
> > > to set things up using a private mail alias. If you want to receive the
> > > messages as well, please tell me.
> > >
> > > Cheers
> > >
> > > Stefan
> > >
> > > [1]
> https://lists.apache.org/thread.html/rb34ea7d9272b8e600437ea705b13aba1bcc2f23ceb55880bce27e479%40%3Cdev.commons.apache.org%3E
> > >
> > > [2] https://google.github.io/oss-fuzz/
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> > > For additional commands, e-mail: dev-h...@commons.apache.org
> > >
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> > For additional commands, e-mail: dev-h...@commons.apache.org
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> For additional commands, e-mail: dev-h...@commons.apache.org
>
>
