tballison commented on code in PR #3130: URL: https://github.com/apache/tika/pull/3130#discussion_r3936821230
########## tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-image-module/src/test/java/org/apache/tika/detect/image/RawTiffDetectorFuzzTest.java: ########## @@ -0,0 +1,136 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.tika.detect.image; + +import static org.junit.jupiter.api.Assertions.fail; + +import java.util.Locale; +import java.util.Random; + +import org.junit.jupiter.api.Test; + +/** + * Randomized boundary test for {@link RawTiffDetector}'s directory walk. + * <p> + * The detector runs ahead of every parser, and {@code Detector.detect} declares + * only {@link java.io.IOException}: anything else it throws aborts detection for + * the document and the remaining detectors never run. So the invariant is simply + * that no throwable escapes, whatever the directories say. + * <p> + * Inputs are well-formed TIFF and BigTIFF headers whose offsets, counts and + * entry values are drawn from the arithmetic boundaries -- 0, the 32 and 64 bit + * maxima, the prefix limit, and their neighbours -- since that is where the + * offset handling goes wrong rather than in random bytes. The seed is random per + * run and reported on failure. + */ +public class RawTiffDetectorFuzzTest { + + private static final int TRIALS = 20000; Review Comment: Correcting myself: I answered the cost question, which was the wrong one. You were right that 20,000 is not justified -- just not for the runtime reason. I measured what the trials actually buy. **Bug-catching power.** Running this generator against the pre-fix detector over 2,000 independent seeds, the `ArrayIndexOutOfBoundsException` shows up at median trial 6, worst case trial 82. 100 trials catches it every time. 20,000 is ~250x more than the regression needs. **Coverage saturates.** Branch coverage of `RawTiffDetector` (JaCoCo) against trial count: ``` N=50 110/202 N=5000 152/202 N=200 125/202 N=20000 153/202 N=1000 146/202 N=200000 156/202 ``` 5,000 -> 20,000 buys one branch out of 202. **And the trials were not where the gap was.** `Prefix` sat at 5/16 branches at *every* N, including 200,000: the fuzz test only called the in-memory `detect(byte[], int)`, where `in == null`, so lines 172-188 -- the entire chunked-read body of `Prefix.ensure`, the method this PR changes -- were never executed at any trial count. More trials could never have reached them. Routing the same inputs through `detect(TikaInputStream, ...)` as well takes `Prefix` from 5/16 to 14/16 branches, and 2,000 trials through both entry points reaches 149/202 overall -- as much as 20,000 through one, in ~0.3s. So: `TRIALS` is now 2,000 (matching `TikaEvalTokenizerFuzzTest`, as you suggested) and each input goes through both entry points. Better test, and the number is picked from measurement rather than from a spare zero. Thanks for the nudge. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
