I threw together a quick jmh test, and there is no value in the changes to Hash234.
For the array mismatch, the results are kind of interesting. My observation, stepping through some compression uses, is that the comparison length is typically 100-200 bytes in length, but the actual match length is typically fairly short. This is obviously going to be highly dependent on data, and I was using raw image data for observation. Content like xml or json might have longer matches. So I set up a benchmark which is always comparing 128 bytes and the mismatch occurs after various "lengths": Benchmark (length) Mode Cnt Score Error Units ArrayMismatchBenchmark.legacyMismatch_nomatch 0 avgt 5 3.198 ± 0.168 ns/op ArrayMismatchBenchmark.legacyMismatch_nomatch 1 avgt 5 5.607 ± 0.048 ns/op ArrayMismatchBenchmark.legacyMismatch_nomatch 2 avgt 5 5.852 ± 0.053 ns/op ArrayMismatchBenchmark.legacyMismatch_nomatch 7 avgt 5 12.703 ± 0.350 ns/op ArrayMismatchBenchmark.legacyMismatch_nomatch 13 avgt 5 18.275 ± 0.228 ns/op ArrayMismatchBenchmark.legacyMismatch_nomatch 57 avgt 5 42.313 ± 0.450 ns/op ArrayMismatchBenchmark.legacyMismatch_nomatch 97 avgt 5 89.410 ± 2.927 ns/op ArrayMismatchBenchmark.arraysMismatch_nomatch 0 avgt 5 4.629 ± 0.035 ns/op ArrayMismatchBenchmark.arraysMismatch_nomatch 1 avgt 5 9.515 ± 0.096 ns/op ArrayMismatchBenchmark.arraysMismatch_nomatch 2 avgt 5 9.526 ± 0.132 ns/op ArrayMismatchBenchmark.arraysMismatch_nomatch 7 avgt 5 9.581 ± 0.395 ns/op ArrayMismatchBenchmark.arraysMismatch_nomatch 13 avgt 5 9.781 ± 0.133 ns/op ArrayMismatchBenchmark.arraysMismatch_nomatch 57 avgt 5 9.846 ± 0.182 ns/op ArrayMismatchBenchmark.arraysMismatch_nomatch 97 avgt 5 10.809 ± 0.307 ns/op ArrayMismatchBenchmark.intMismatch_nomatch 0 avgt 5 3.417 ± 0.018 ns/op ArrayMismatchBenchmark.intMismatch_nomatch 1 avgt 5 3.412 ± 0.011 ns/op ArrayMismatchBenchmark.intMismatch_nomatch 2 avgt 5 3.414 ± 0.032 ns/op ArrayMismatchBenchmark.intMismatch_nomatch 7 avgt 5 5.401 ± 0.207 ns/op ArrayMismatchBenchmark.intMismatch_nomatch 13 avgt 5 8.311 ± 0.070 ns/op ArrayMismatchBenchmark.intMismatch_nomatch 57 avgt 5 20.536 ± 0.556 ns/op ArrayMismatchBenchmark.intMismatch_nomatch 97 avgt 5 30.969 ± 0.318 ns/op ArrayMismatchBenchmark.longMismatch_nomatch 0 avgt 5 4.399 ± 0.082 ns/op ArrayMismatchBenchmark.longMismatch_nomatch 1 avgt 5 4.390 ± 0.068 ns/op ArrayMismatchBenchmark.longMismatch_nomatch 2 avgt 5 4.398 ± 0.033 ns/op ArrayMismatchBenchmark.longMismatch_nomatch 7 avgt 5 4.403 ± 0.110 ns/op ArrayMismatchBenchmark.longMismatch_nomatch 13 avgt 5 6.564 ± 0.398 ns/op ArrayMismatchBenchmark.longMismatch_nomatch 57 avgt 5 11.548 ± 0.331 ns/op ArrayMismatchBenchmark.longMismatch_nomatch 97 avgt 5 16.335 ± 0.119 ns/op I labeled the current behavior as "legacy". The Arrays.mismatch is significantly slower when the mismatch occurs early in the array and significantly faster when the mismatch occurs later. Comparing an int (4 bytes) at a time is a clear winner if the mismatch occurs in those 4 bytes, which appeared to be 90+% of the calls I observed. Comparing a long (8 bytes) at a time is faster than the current behavior unless it is the first byte which does not match, but slower than comparing ints if the mismatch occurs in the first 4 bytes. I wrote this test using jdk 9 VarHandle to read the ints and longs from the byte[], but the same thing can be achieved using sun.misc.Unsafe. I will add that as a case in the benchmark, but it is expected to be similar to VarHandle (maybe slightly faster). Brett On Mon, Jan 11, 2021 at 10:04 AM Lasse Collin <lasse.col...@tukaani.org> wrote: > > On 2021-01-09 Brett Okken wrote: > > This would seem to be a potential candidate for a multi-release > > jar[1], if you can figure out a reasonable way to get a build system > > to generate one. > > I suppose it can be done. The build system uses Apache Ant. From some > sources I've understood that there are more modern alternatives but I > haven't had any interest or energy to learn more as Ant seems to still > work OK. > > > The 4 uses I found of comparing byte[] could be refactored to call a > > new utility class to do the comparison. The "regular" implementation > > could be java 7 compatible, and the jdk 9 version would be in the > > META_INF folder. > > Even for the java 7 compatible version, it might be worth exploring > > how much improvement would come from using Unsafe to read int or long > > values from the byte[] and compare those. > > > > For Hash234, I would think the whole class could be handled for the > > All these sound like worth checking out. > > On 2021-01-09 Brett Okken wrote: > > Here is a class which is compatible with jdk 7. It will use a > > MethodHandle to invoke Arrays.mismatch if that is found at runtime. If > > that is not found, it will see if it can find Unsafe to read 4 bytes > > at a time and compare as ints. If that cannot be found/loaded/invoked, > > it falls back to iterating over bytes and comparing one by one. > > > > For jdk 9, the mismatch method could instead be implemented as: > > return Arrays.mismatch(a, aFromIndex, aFromIndex + length, b, > > bFromIndex, bFromIndex + length); > > Thanks! There are several XZ Utils related tasks I hope to get done (of > which not all have been mentioned on xz-devel), so I won't think much > about XZ for Java in the near future, I'm sorry. > > I assume that multi-release method has no performance overhead since > the runtime will load the best .class file and that's it. How the other > methods like using an utility class or looking for available methods at > runtime compare to pure multi-release method in terms of performance? > Perhaps this is a stupid question but I have so little Java experience > that I don't have a clue about this. > > If you have time and interest, it would be valuable to know which > tricks provide the largest performance improvements. However, I repeat > that I cannot spend much time on this in the near future even though I > think it would be good to have such improvements in XZ for Java. > > Thanks! > > -- > Lasse Collin | IRC: Larhzu @ IRCnet & Freenode >
