calcite-1.35.0 with JDK8(1.8.0_391) on MacOS with M1 Pro chip: can reproduce
RemapperTest > calcite35WithCheck() FAILED
java.lang.IllegalArgumentException: Invalid start label (must be
visited first)
at
org.objectweb.asm.util.CheckMethodAdapter.checkLabel(CheckMethodAdapter.java:1453)
at
org.objectweb.asm.util.CheckMethodAdapter.visitLocalVariableAnnotation(CheckMethodAdapter.java:995)
at
org.objectweb.asm.MethodVisitor.visitLocalVariableAnnotation(MethodVisitor.java:757)
at
org.objectweb.asm.commons.MethodRemapper.visitLocalVariableAnnotation(MethodRemapper.java:257)
at org.objectweb.asm.ClassReader.readCode(ClassReader.java:2614)
at org.objectweb.asm.ClassReader.readMethod(ClassReader.java:1515)
at org.objectweb.asm.ClassReader.accept(ClassReader.java:745)
at org.objectweb.asm.ClassReader.accept(ClassReader.java:425)
at remapper.bug.RemapperTest.runTest(RemapperTest.java:53)
at remapper.bug.RemapperTest.calcite35WithCheck(RemapperTest.java:32)
https://github.com/MasseGuillaume/asm-remapper-bug/blob/main/SqlFunctions-1.35.0.class
If you kept the .class for your tests, can you try to run javap -p -c -v
On Wed, Dec 20, 2023 at 1:11 PM Julian Hyde <jhyde.apa...@gmail.com> wrote:
> The version of Guava that you use at compile time is important. If it’s
> earlier than 20 (-ish) the byte codes will be different because of the
> presence/absence of certain varargs methods.
>
>
> > On Dec 19, 2023, at 7:28 PM, Benchao Li <libenc...@apache.org> wrote:
> >
> > I tried to reproduce this using Guillaume's
> > project(https://github.com/MasseGuillaume/asm-remapper-bug), and the
> > findings are below:
> >
> > - compiled latest main branch
> > (50a20824c4536450dcae963b5e757cf4bbc7e406) with JDK8(1.8.0_391) on
> > MacOS with M2 chip: reproduced
> > - compiled latest main branch
> > (50a20824c4536450dcae963b5e757cf4bbc7e406) with JDK8(1.8.0_391) on
> > Linux with x64 architecture: reproduced
> > - compiled latest main branch
> > (50a20824c4536450dcae963b5e757cf4bbc7e406) with JDK11(1.11.0_21) on
> > MacOS with M2 chip: cannot reproduce
> > - compiled latest main branch
> > (50a20824c4536450dcae963b5e757cf4bbc7e406) with JDK17(1.17.0_7) on
> > MacOS with M2 chip: cannot reproduce
> >
> > It seems unrelated to platforms, and I assume it should not either,
> > since Java's bytecode should be platform independent.
> >
> > @Guillaume You said above that with 1.8.0_371-b11 on macos x64, you
> > cannot reproduce the problem, can you confirm that? Per my testing,
> > JDK8's output could all reproduce the problem
> >
> > Guillaume Masse <masse.guilla...@narrative.io.invalid> 于2023年12月18日周一
> 23:13写道：
> >>
> >> If I build 1.36.0 and 1.35.0 with the same java version/build javap can
> >> read the classfiles. Since the release process is manual it's really
> hard
> >> to know what causes the problem. My solution is to build and release
> each
> >> Calcite version myself and host it on our own S3 repository.
> >>
> >> On Mon, Dec 18, 2023, 8:56 AM Ruben Q L <rube...@gmail.com> wrote:
> >>
> >>> Hello,
> >>>
> >>> My project also recently upgraded to Calcite 1.36, and we are facing
> the
> >>> exact same issue when trying to shade with ASM.
> >>>
> >>> @Guillame , I can see that
> https://gitlab.ow2.org/asm/asm/-/issues/318008
> >>> has
> >>> been closed, but I can't really understand why, since the explanation
> >>> mentions SqlFunctions-1.35.0, but the issue is with 1.36, do you have
> more
> >>> info on that?
> >>>
> >>> Best,
> >>> Ruben
> >>>
> >>>
> >>> On Fri, Nov 24, 2023 at 10:09 PM Guillaume Masse
> >>> <masse.guilla...@narrative.io.invalid> wrote:
> >>>
> >>>> We run spark in AWS EMR and it overrides the classpath, so we don't
> have
> >>>> control over it:
> >>>>
> >>>> Sparks will pull guava 14.0.1:
> >>>>
> >>>>
> >>>
> https://github.com/apache/spark/blob/master/dev/deps/spark-deps-hadoop-3-hive-2.3#L68
> >>>>
> >>>> That's why we need to shade guava.
> >>>>
> >>>>
> >>>> On Fri, Nov 24, 2023 at 4:31 PM Julian Hyde <jh...@apache.org> wrote:
> >>>>
> >>>>> The version of Guava we compile against is relevant. In 1.35 we
> >>>>> compiled against Guava 19.0; in 1.36 we compiled against Guava
> >>>>> 32.1.3-jre. The effect would be the same if we compiled against any
> >>>>> version of Guava 20.0 or higher, due to the addition of
> >>>>> Preconditions.checkArgument methods that in earlier Guava versions
> >>>>> would be handled by varargs method.
> >>>>>
> >>>>> See https://issues.apache.org/jira/browse/CALCITE-5477 (which was
> >>>>> fixed in 1.35) and
> https://issues.apache.org/jira/browse/CALCITE-5763
> >>>>> (which was fixed in 1.36 and essentially reverted 5477).
> >>>>>
> >>>>> Julian
> >>>>>
> >>>>> On Fri, Nov 24, 2023 at 10:20 AM Guillaume Masse
> >>>>> <masse.guilla...@narrative.io.invalid> wrote:
> >>>>>>
> >>>>>> Hi Benchao Li,
> >>>>>>
> >>>>>> thanks for giving me more details like the java version,
> >>>>>>
> >>>>>> I compiled with 1.8.0_371-b11 on macos x64 (emulated) and I was
> still
> >>>>> able
> >>>>>> to transform the classfile. The bug must come from something else.
> >>> Was
> >>>>> this
> >>>>>> compiled on your personal machine? Was it on windows/linux/mac?
> >>> What's
> >>>>> the
> >>>>>> processor architecture?
> >>>>>>
> >>>>>> 1.35.0 also pases the transformation and the same piece of code is
> >>>> there.
> >>>>>> How was 1.35.0 released?
> >>>>>>
> >>>>>> Thanks!
> >>>>>>
> >>>>>>
> >>>>>> On Thu, Nov 23, 2023 at 9:59 PM Benchao Li <libenc...@apache.org>
> >>>> wrote:
> >>>>>>
> >>>>>>> Guillaume,
> >>>>>>>
> >>>>>>> The binary release are always compiled with JDK8, this is a
> >>> required
> >>>>>>> procedure[1].
> >>>>>>>
> >>>>>>> For 1.36.0, the JDK version I'm using is:
> >>>>>>> $ java -version
> >>>>>>> java version "1.8.0_371"
> >>>>>>> Java(TM) SE Runtime Environment (build 1.8.0_371-b11)
> >>>>>>> Java HotSpot(TM) 64-Bit Server VM (build 25.371-b11, mixed mode)
> >>>>>>>
> >>>>>>> [1]
> >>>>>
> https://calcite.apache.org/docs/howto.html#making-a-release-candidate
> >>>>>>>
> >>>>>>> Guillaume Masse <masse.guilla...@narrative.io.invalid>
> >>>> 于2023年11月24日周五
> >>>>>>> 07:22写道：
> >>>>>>>>
> >>>>>>>> Hi all,
> >>>>>>>>
> >>>>>>>> We recently upgraded to calcite 1.36.0 and it broke our
> >>> deployment
> >>>>>>> process.
> >>>>>>>> We are using java asm <https://asm.ow2.io/> to shade (rename
> >>>>> packages
> >>>>>>> from
> >>>>>>>> the bytecode) since there are multiple dependencies clash when
> >>>> using
> >>>>> with
> >>>>>>>> Apache Spark. For example, Apache Spark is using a much older
> >>>>> version of
> >>>>>>>> guava. If I build calcite locally with Java Correto 17.0.7-amzn I
> >>>> can
> >>>>>>>> transform calcite's bytecode without any problem. Where can I
> >>> find
> >>>>> the
> >>>>>>> java
> >>>>>>>> version used to compiled calcite? Is there a process in place to
> >>>>> have a
> >>>>>>>> consistent java version?
> >>>>>>>>
> >>>>>>>> https://gitlab.ow2.org/asm/asm/-/issues/318008
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> Guillaume Massé
> >>>>>>>> [Gee-OHM]
> >>>>>>>> (马赛卫)
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>>
> >>>>>>> Best,
> >>>>>>> Benchao Li
> >>>>>>>
> >>>>>
> >>>>
> >>>
> >
> >
> >
> > --
> >
> > Best,
> > Benchao Li
>
>
