Hi Claes, You’re fast! :) Thanks for the bug report and working on a fix.
Best, Vitaly On Tue, Jan 19, 2021 at 5:00 PM Claes Redestad <claes.redes...@oracle.com> wrote: > Filed: https://bugs.openjdk.java.net/browse/JDK-8260010 > > I have a potential fix ready, just trying to distill a minimal test case. > > /Claes > > On 2021-01-19 21:46, Claes Redestad wrote: > > Hi, > > > > yes, this seems like a bug introduced by JDK-8243469 where the > > previously thread-safe UTF8ZipCoder is now using a thread-unsafe > > decoder to calculate the normalized hash. I'll file a bug and take a > > look. > > > > /Claes > > > > On 2021-01-19 20:19, Vitaly Davidovich wrote: > >> Hi all, > >> > >> I observed the following stacktrace when inspecting a JarFile's contents > >> using a parallel stream (i.e. FJ pool): > >> > >> Exception in thread "main" java.lang.IllegalStateException: > >> java.lang.IllegalStateException: Current state = CODING_END, new state = > >> FLUSHED > >> > >> at > >> > java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native > > >> > >> Method) > >> > >> at > >> > java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:64) > > >> > >> > >> at > >> > java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > > >> > >> > >> at > >> > java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500) > > >> > >> > >> at > >> > java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:481) > >> > >> > >> at > >> > java.base/java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:600) > > >> > >> > >> at > >> > java.base/java.util.concurrent.ForkJoinTask.reportException(ForkJoinTask.java:678) > > >> > >> > >> at > >> > java.base/java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:737) > >> > >> > >> at > >> > java.base/java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:159) > > >> > >> > >> at > >> > java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(ForEachOps.java:173) > > >> > >> > >> at > >> > java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233) > > >> > >> > >> at > >> > java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497) > > >> > >> > >> at > >> > java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:575) > > >> > >> > >> at > >> <stripped>.ClassVersions.conflicts(ClassVersions.java:703) > >> > >> at <stripped>.ClassVersions.check(ClassVersions.java:743) > >> > >> at <stripped>.ClassVersions.main(ClassVersions.java:839) > >> > >> Caused by: java.lang.IllegalStateException: Current state = > >> CODING_END, new > >> state = FLUSHED > >> > >> at > >> > java.base/java.nio.charset.CharsetDecoder.throwIllegalStateException(CharsetDecoder.java:998) > > >> > >> > >> at > >> java.base/java.nio.charset.CharsetDecoder.flush(CharsetDecoder.java:681) > >> > >> at > >> > java.base/java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:810) > >> > >> > >> at > >> java.base/java.util.zip.ZipCoder.normalizedHashDecode(ZipCoder.java:136) > >> > >> at > >> > java.base/java.util.zip.ZipCoder$UTF8ZipCoder.normalizedHash(ZipCoder.java:228) > > >> > >> > >> at > >> java.base/java.util.zip.ZipFile$Source.initCEN(ZipFile.java:1527) > >> > >> at > >> java.base/java.util.zip.ZipFile$Source.<init>(ZipFile.java:1249) > >> > >> at > >> java.base/java.util.zip.ZipFile$Source.get(ZipFile.java:1211) > >> > >> at > >> > java.base/java.util.zip.ZipFile$CleanableResource.<init>(ZipFile.java:701) > >> > >> > >> at > >> java.base/java.util.zip.ZipFile.<init>(ZipFile.java:240) > >> > >> at > >> java.base/java.util.zip.ZipFile.<init>(ZipFile.java:171) > >> > >> at > >> java.base/java.util.jar.JarFile.<init>(JarFile.java:347) > >> > >> at > >> java.base/java.util.jar.JarFile.<init>(JarFile.java:318) > >> > >> at > >> java.base/java.util.jar.JarFile.<init>(JarFile.java:284) > >> > >> at > >> <stripped>.ClassVersions.lookupJar(ClassVersions.java:665) > >> > >> at > >> > java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195) > > >> > >> > >> at > >> > java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:177) > > >> > >> > >> at > >> > java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195) > > >> > >> > >> at > >> > java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:177) > > >> > >> > >> at > >> > java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948) > > >> > >> > >> at > >> > java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) > > >> > >> > >> at > >> > java.base/java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:290) > > >> > >> > >> at > >> > java.base/java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:746) > > >> > >> > >> at > >> > java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290) > >> > >> > >> at > >> > java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1016) > > >> > >> > >> at > >> java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1665) > >> > >> at > >> > java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1598) > > >> > >> > >> at > >> > java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183) > > >> > >> > >> > >> (Note: I stripped off some package names to protect the innocent). > >> > >> > >> While investigating this a bit, I noticed some code that looks a bit > >> suspect. Namely: > >> > >> - ZipCoder has a singleton UTF8ZipCoder > >> < > https://github.com/openjdk/jdk/blob/05bcd67e6501f3824020edd1ab036c87888fa229/src/java.base/share/classes/java/util/zip/ZipCoder.java#L48 > > > >> instance it uses for UTF8 decoding. > >> - Source.get() creates a new Source > >> < > https://github.com/openjdk/jdk/blob/05bcd67e6501f3824020edd1ab036c87888fa229/src/java.base/share/classes/java/util/zip/ZipFile.java#L1225 > > > >> instance outside of any synchronization > >> > >> The links above are to tip, but I think this code is the same in Java > >> 15. > >> > >> It seems possible that multiple threads can end up in Source.get(), > >> new up > >> a Source, and proceed to use the decoder of UTF8ZipCoder racily. Per my > >> understanding (and CharsetDecoder javadoc), CharsetDecoders are not > >> threadsafe. > >> > >> Any thoughts? Did I miss anything that would suggest the stacktrace > >> above > >> is due to something else entirely? > >> > >> Thanks > -- Sent from my phone
