[jira] [Commented] (BEAM-9252) Problem shading Beam pipeline with Beam 2.20.0-SNAPSHOT
[
https://issues.apache.org/jira/browse/BEAM-9252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17045606#comment-17045606
]
Tomo Suzuki commented on BEAM-9252:
---
[~iemejia] Thank you for confirming.
> Problem shading Beam pipeline with Beam 2.20.0-SNAPSHOT
> ---
>
> Key: BEAM-9252
> URL: https://issues.apache.org/jira/browse/BEAM-9252
> Project: Beam
> Issue Type: Bug
> Components: build-system
>Affects Versions: 2.20.0
>Reporter: Ismaël Mejía
>Assignee: Luke Cwik
>Priority: Critical
> Fix For: 2.20.0
>
> Attachments: ArrayIndexOutOfBoundsException.png,
> image-2020-02-21-13-22-01-341.png
>
> Time Spent: 3.5h
> Remaining Estimate: 0h
>
> I was checking today a pipeline against the latest 2.20.0-SNAPSHOT and I
> found that it works perfectly with version 2.19.0, but it is failing with a
> shade related exception that refers to grpc 1.26.0:
> {{[ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-shade-plugin:3.2.1:shade (default) on project
> EventsToIOs: Error creating shaded jar: Problem shading JAR
> /home/ismael/.m2/repository/org/apache/beam/beam-vendor-grpc-1_26_0/0.1/beam-vendor-grpc-1_26_0-0.1.jar
> entry org/apache/beam/vendor/grpc/v1p26p0/org/jboss/modules/Main.class:
> org.apache.maven.plugin.MojoExecutionException: Error in ASM processing class
> org/apache/beam/vendor/grpc/v1p26p0/org/jboss/modules/Main.class: 65536 ->
> [Help 1]}}
> {{There is also a warning that is not present in the build against 2.19.0}}
> {{[WARNING] Discovered module-info.class. Shading will break its strong
> encapsulation.}}
>
> I wonder if we are not doing something wrong during our vendoring, can
> someone take a look please.
> This is relatively easy to reproduce with the beam-samples repo, just clone
> it and run:
> {noformat}
> git clone https://github.com/jbonofre/beam-samples
> mvn clean verify -Pbeam-release-repo -Dbeam.version=2.20.0-SNAPSHOT
> {noformat}
> Available logs of the latest run:
> [https://github.com/jbonofre/beam-samples/runs/427537544?check_suite_focus=true]
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (BEAM-9252) Problem shading Beam pipeline with Beam 2.20.0-SNAPSHOT
[
https://issues.apache.org/jira/browse/BEAM-9252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17045314#comment-17045314
]
Ismaël Mejía commented on BEAM-9252:
The fix for this is working now, and users can shade again their pipelines with
2.20.0.
The tests on beam-samples are passing, for ref
https://github.com/jbonofre/beam-samples/actions/runs/45201474
> Problem shading Beam pipeline with Beam 2.20.0-SNAPSHOT
> ---
>
> Key: BEAM-9252
> URL: https://issues.apache.org/jira/browse/BEAM-9252
> Project: Beam
> Issue Type: Bug
> Components: build-system
>Affects Versions: 2.20.0
>Reporter: Ismaël Mejía
>Assignee: Luke Cwik
>Priority: Critical
> Fix For: 2.20.0
>
> Attachments: ArrayIndexOutOfBoundsException.png,
> image-2020-02-21-13-22-01-341.png
>
> Time Spent: 3.5h
> Remaining Estimate: 0h
>
> I was checking today a pipeline against the latest 2.20.0-SNAPSHOT and I
> found that it works perfectly with version 2.19.0, but it is failing with a
> shade related exception that refers to grpc 1.26.0:
> {{[ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-shade-plugin:3.2.1:shade (default) on project
> EventsToIOs: Error creating shaded jar: Problem shading JAR
> /home/ismael/.m2/repository/org/apache/beam/beam-vendor-grpc-1_26_0/0.1/beam-vendor-grpc-1_26_0-0.1.jar
> entry org/apache/beam/vendor/grpc/v1p26p0/org/jboss/modules/Main.class:
> org.apache.maven.plugin.MojoExecutionException: Error in ASM processing class
> org/apache/beam/vendor/grpc/v1p26p0/org/jboss/modules/Main.class: 65536 ->
> [Help 1]}}
> {{There is also a warning that is not present in the build against 2.19.0}}
> {{[WARNING] Discovered module-info.class. Shading will break its strong
> encapsulation.}}
>
> I wonder if we are not doing something wrong during our vendoring, can
> someone take a look please.
> This is relatively easy to reproduce with the beam-samples repo, just clone
> it and run:
> {noformat}
> git clone https://github.com/jbonofre/beam-samples
> mvn clean verify -Pbeam-release-repo -Dbeam.version=2.20.0-SNAPSHOT
> {noformat}
> Available logs of the latest run:
> [https://github.com/jbonofre/beam-samples/runs/427537544?check_suite_focus=true]
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (BEAM-9252) Problem shading Beam pipeline with Beam 2.20.0-SNAPSHOT
[
https://issues.apache.org/jira/browse/BEAM-9252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17045132#comment-17045132
]
Rui Wang commented on BEAM-9252:
Ack.
[~j...@nanthrax.net] could you validate if your project can shade after fixes?
> Problem shading Beam pipeline with Beam 2.20.0-SNAPSHOT
> ---
>
> Key: BEAM-9252
> URL: https://issues.apache.org/jira/browse/BEAM-9252
> Project: Beam
> Issue Type: Bug
> Components: build-system
>Affects Versions: 2.20.0
>Reporter: Ismaël Mejía
>Assignee: Luke Cwik
>Priority: Critical
> Fix For: 2.20.0
>
> Attachments: ArrayIndexOutOfBoundsException.png,
> image-2020-02-21-13-22-01-341.png
>
> Time Spent: 3.5h
> Remaining Estimate: 0h
>
> I was checking today a pipeline against the latest 2.20.0-SNAPSHOT and I
> found that it works perfectly with version 2.19.0, but it is failing with a
> shade related exception that refers to grpc 1.26.0:
> {{[ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-shade-plugin:3.2.1:shade (default) on project
> EventsToIOs: Error creating shaded jar: Problem shading JAR
> /home/ismael/.m2/repository/org/apache/beam/beam-vendor-grpc-1_26_0/0.1/beam-vendor-grpc-1_26_0-0.1.jar
> entry org/apache/beam/vendor/grpc/v1p26p0/org/jboss/modules/Main.class:
> org.apache.maven.plugin.MojoExecutionException: Error in ASM processing class
> org/apache/beam/vendor/grpc/v1p26p0/org/jboss/modules/Main.class: 65536 ->
> [Help 1]}}
> {{There is also a warning that is not present in the build against 2.19.0}}
> {{[WARNING] Discovered module-info.class. Shading will break its strong
> encapsulation.}}
>
> I wonder if we are not doing something wrong during our vendoring, can
> someone take a look please.
> This is relatively easy to reproduce with the beam-samples repo, just clone
> it and run:
> {noformat}
> git clone https://github.com/jbonofre/beam-samples
> mvn clean verify -Pbeam-release-repo -Dbeam.version=2.20.0-SNAPSHOT
> {noformat}
> Available logs of the latest run:
> [https://github.com/jbonofre/beam-samples/runs/427537544?check_suite_focus=true]
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (BEAM-9252) Problem shading Beam pipeline with Beam 2.20.0-SNAPSHOT
[
https://issues.apache.org/jira/browse/BEAM-9252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17045109#comment-17045109
]
Luke Cwik commented on BEAM-9252:
-
This task still requires validation that JB's project can shade.
> Problem shading Beam pipeline with Beam 2.20.0-SNAPSHOT
> ---
>
> Key: BEAM-9252
> URL: https://issues.apache.org/jira/browse/BEAM-9252
> Project: Beam
> Issue Type: Bug
> Components: build-system
>Affects Versions: 2.20.0
>Reporter: Ismaël Mejía
>Assignee: Luke Cwik
>Priority: Critical
> Fix For: 2.20.0
>
> Attachments: ArrayIndexOutOfBoundsException.png,
> image-2020-02-21-13-22-01-341.png
>
> Time Spent: 3.5h
> Remaining Estimate: 0h
>
> I was checking today a pipeline against the latest 2.20.0-SNAPSHOT and I
> found that it works perfectly with version 2.19.0, but it is failing with a
> shade related exception that refers to grpc 1.26.0:
> {{[ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-shade-plugin:3.2.1:shade (default) on project
> EventsToIOs: Error creating shaded jar: Problem shading JAR
> /home/ismael/.m2/repository/org/apache/beam/beam-vendor-grpc-1_26_0/0.1/beam-vendor-grpc-1_26_0-0.1.jar
> entry org/apache/beam/vendor/grpc/v1p26p0/org/jboss/modules/Main.class:
> org.apache.maven.plugin.MojoExecutionException: Error in ASM processing class
> org/apache/beam/vendor/grpc/v1p26p0/org/jboss/modules/Main.class: 65536 ->
> [Help 1]}}
> {{There is also a warning that is not present in the build against 2.19.0}}
> {{[WARNING] Discovered module-info.class. Shading will break its strong
> encapsulation.}}
>
> I wonder if we are not doing something wrong during our vendoring, can
> someone take a look please.
> This is relatively easy to reproduce with the beam-samples repo, just clone
> it and run:
> {noformat}
> git clone https://github.com/jbonofre/beam-samples
> mvn clean verify -Pbeam-release-repo -Dbeam.version=2.20.0-SNAPSHOT
> {noformat}
> Available logs of the latest run:
> [https://github.com/jbonofre/beam-samples/runs/427537544?check_suite_focus=true]
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (BEAM-9252) Problem shading Beam pipeline with Beam 2.20.0-SNAPSHOT
[
https://issues.apache.org/jira/browse/BEAM-9252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17043966#comment-17043966
]
Rui Wang commented on BEAM-9252:
Just FYI, I am going to cut 2.20.0 branch on 02/26/2020.
> Problem shading Beam pipeline with Beam 2.20.0-SNAPSHOT
> ---
>
> Key: BEAM-9252
> URL: https://issues.apache.org/jira/browse/BEAM-9252
> Project: Beam
> Issue Type: Bug
> Components: build-system
>Affects Versions: 2.20.0
>Reporter: Ismaël Mejía
>Assignee: Luke Cwik
>Priority: Critical
> Fix For: 2.20.0
>
> Attachments: ArrayIndexOutOfBoundsException.png,
> image-2020-02-21-13-22-01-341.png
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> I was checking today a pipeline against the latest 2.20.0-SNAPSHOT and I
> found that it works perfectly with version 2.19.0, but it is failing with a
> shade related exception that refers to grpc 1.26.0:
> {{[ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-shade-plugin:3.2.1:shade (default) on project
> EventsToIOs: Error creating shaded jar: Problem shading JAR
> /home/ismael/.m2/repository/org/apache/beam/beam-vendor-grpc-1_26_0/0.1/beam-vendor-grpc-1_26_0-0.1.jar
> entry org/apache/beam/vendor/grpc/v1p26p0/org/jboss/modules/Main.class:
> org.apache.maven.plugin.MojoExecutionException: Error in ASM processing class
> org/apache/beam/vendor/grpc/v1p26p0/org/jboss/modules/Main.class: 65536 ->
> [Help 1]}}
> {{There is also a warning that is not present in the build against 2.19.0}}
> {{[WARNING] Discovered module-info.class. Shading will break its strong
> encapsulation.}}
>
> I wonder if we are not doing something wrong during our vendoring, can
> someone take a look please.
> This is relatively easy to reproduce with the beam-samples repo, just clone
> it and run:
> {noformat}
> git clone https://github.com/jbonofre/beam-samples
> mvn clean verify -Pbeam-release-repo -Dbeam.version=2.20.0-SNAPSHOT
> {noformat}
> Available logs of the latest run:
> [https://github.com/jbonofre/beam-samples/runs/427537544?check_suite_focus=true]
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (BEAM-9252) Problem shading Beam pipeline with Beam 2.20.0-SNAPSHOT
[
https://issues.apache.org/jira/browse/BEAM-9252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17042305#comment-17042305
]
Luke Cwik commented on BEAM-9252:
-
Vote thread:
https://lists.apache.org/thread.html/r5d762577f9ad3a71b91c749a382603019e7b0a15ef875a459c921229%40%3Cdev.beam.apache.org%3E
> Problem shading Beam pipeline with Beam 2.20.0-SNAPSHOT
> ---
>
> Key: BEAM-9252
> URL: https://issues.apache.org/jira/browse/BEAM-9252
> Project: Beam
> Issue Type: Bug
> Components: build-system
>Affects Versions: 2.20.0
>Reporter: Ismaël Mejía
>Assignee: Luke Cwik
>Priority: Critical
> Fix For: 2.20.0
>
> Attachments: ArrayIndexOutOfBoundsException.png,
> image-2020-02-21-13-22-01-341.png
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> I was checking today a pipeline against the latest 2.20.0-SNAPSHOT and I
> found that it works perfectly with version 2.19.0, but it is failing with a
> shade related exception that refers to grpc 1.26.0:
> {{[ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-shade-plugin:3.2.1:shade (default) on project
> EventsToIOs: Error creating shaded jar: Problem shading JAR
> /home/ismael/.m2/repository/org/apache/beam/beam-vendor-grpc-1_26_0/0.1/beam-vendor-grpc-1_26_0-0.1.jar
> entry org/apache/beam/vendor/grpc/v1p26p0/org/jboss/modules/Main.class:
> org.apache.maven.plugin.MojoExecutionException: Error in ASM processing class
> org/apache/beam/vendor/grpc/v1p26p0/org/jboss/modules/Main.class: 65536 ->
> [Help 1]}}
> {{There is also a warning that is not present in the build against 2.19.0}}
> {{[WARNING] Discovered module-info.class. Shading will break its strong
> encapsulation.}}
>
> I wonder if we are not doing something wrong during our vendoring, can
> someone take a look please.
> This is relatively easy to reproduce with the beam-samples repo, just clone
> it and run:
> {noformat}
> git clone https://github.com/jbonofre/beam-samples
> mvn clean verify -Pbeam-release-repo -Dbeam.version=2.20.0-SNAPSHOT
> {noformat}
> Available logs of the latest run:
> [https://github.com/jbonofre/beam-samples/runs/427537544?check_suite_focus=true]
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (BEAM-9252) Problem shading Beam pipeline with Beam 2.20.0-SNAPSHOT
[
https://issues.apache.org/jira/browse/BEAM-9252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17042276#comment-17042276
]
Luke Cwik commented on BEAM-9252:
-
Now we just need someone to perform the vendored grpc 1.26.0 0.2 release and
then the Beam version bump to 0.2
> Problem shading Beam pipeline with Beam 2.20.0-SNAPSHOT
> ---
>
> Key: BEAM-9252
> URL: https://issues.apache.org/jira/browse/BEAM-9252
> Project: Beam
> Issue Type: Bug
> Components: build-system
>Affects Versions: 2.20.0
>Reporter: Ismaël Mejía
>Assignee: Tomo Suzuki
>Priority: Critical
> Fix For: 2.20.0
>
> Attachments: ArrayIndexOutOfBoundsException.png,
> image-2020-02-21-13-22-01-341.png
>
> Time Spent: 2h
> Remaining Estimate: 0h
>
> I was checking today a pipeline against the latest 2.20.0-SNAPSHOT and I
> found that it works perfectly with version 2.19.0, but it is failing with a
> shade related exception that refers to grpc 1.26.0:
> {{[ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-shade-plugin:3.2.1:shade (default) on project
> EventsToIOs: Error creating shaded jar: Problem shading JAR
> /home/ismael/.m2/repository/org/apache/beam/beam-vendor-grpc-1_26_0/0.1/beam-vendor-grpc-1_26_0-0.1.jar
> entry org/apache/beam/vendor/grpc/v1p26p0/org/jboss/modules/Main.class:
> org.apache.maven.plugin.MojoExecutionException: Error in ASM processing class
> org/apache/beam/vendor/grpc/v1p26p0/org/jboss/modules/Main.class: 65536 ->
> [Help 1]}}
> {{There is also a warning that is not present in the build against 2.19.0}}
> {{[WARNING] Discovered module-info.class. Shading will break its strong
> encapsulation.}}
>
> I wonder if we are not doing something wrong during our vendoring, can
> someone take a look please.
> This is relatively easy to reproduce with the beam-samples repo, just clone
> it and run:
> {noformat}
> git clone https://github.com/jbonofre/beam-samples
> mvn clean verify -Pbeam-release-repo -Dbeam.version=2.20.0-SNAPSHOT
> {noformat}
> Available logs of the latest run:
> [https://github.com/jbonofre/beam-samples/runs/427537544?check_suite_focus=true]
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (BEAM-9252) Problem shading Beam pipeline with Beam 2.20.0-SNAPSHOT
[
https://issues.apache.org/jira/browse/BEAM-9252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17042133#comment-17042133
]
Tomo Suzuki commented on BEAM-9252:
---
For module-info.class, Beam used to have verification mechanism to detect
unexpected module-info.class:
This change disabled that
https://github.com/apache/beam/pull/10463/files#diff-23833058cbf2c1172b90e7764032aa59R959
.
> Problem shading Beam pipeline with Beam 2.20.0-SNAPSHOT
> ---
>
> Key: BEAM-9252
> URL: https://issues.apache.org/jira/browse/BEAM-9252
> Project: Beam
> Issue Type: Bug
> Components: build-system
>Affects Versions: 2.20.0
>Reporter: Ismaël Mejía
>Assignee: Tomo Suzuki
>Priority: Critical
> Fix For: 2.20.0
>
> Attachments: ArrayIndexOutOfBoundsException.png,
> image-2020-02-21-13-22-01-341.png
>
>
> I was checking today a pipeline against the latest 2.20.0-SNAPSHOT and I
> found that it works perfectly with version 2.19.0, but it is failing with a
> shade related exception that refers to grpc 1.26.0:
> {{[ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-shade-plugin:3.2.1:shade (default) on project
> EventsToIOs: Error creating shaded jar: Problem shading JAR
> /home/ismael/.m2/repository/org/apache/beam/beam-vendor-grpc-1_26_0/0.1/beam-vendor-grpc-1_26_0-0.1.jar
> entry org/apache/beam/vendor/grpc/v1p26p0/org/jboss/modules/Main.class:
> org.apache.maven.plugin.MojoExecutionException: Error in ASM processing class
> org/apache/beam/vendor/grpc/v1p26p0/org/jboss/modules/Main.class: 65536 ->
> [Help 1]}}
> {{There is also a warning that is not present in the build against 2.19.0}}
> {{[WARNING] Discovered module-info.class. Shading will break its strong
> encapsulation.}}
>
> I wonder if we are not doing something wrong during our vendoring, can
> someone take a look please.
> This is relatively easy to reproduce with the beam-samples repo, just clone
> it and run:
> {noformat}
> git clone https://github.com/jbonofre/beam-samples
> mvn clean verify -Pbeam-release-repo -Dbeam.version=2.20.0-SNAPSHOT
> {noformat}
> Available logs of the latest run:
> [https://github.com/jbonofre/beam-samples/runs/427537544?check_suite_focus=true]
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (BEAM-9252) Problem shading Beam pipeline with Beam 2.20.0-SNAPSHOT
[
https://issues.apache.org/jira/browse/BEAM-9252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17042106#comment-17042106
]
Tomo Suzuki commented on BEAM-9252:
---
The jboss's Main class seems incorrect.
{noformat}
suztomo@suxtomo24:~/jbonofre-beam-samples$ javap -verbose -cp
/usr/local/google/home/suztomo/.m2/repository/org/apache/beam/beam-vendor-grpc-1_26_0/0.1/beam-vendor-grpc-1_26_0-0.1.jar
org/apache/beam/vendor/grpc/v1p26p0/org/jboss/modules/Main
...
LocalVariableTable:
Start Length Slot Name Signature
562 6497415 cnt I
588 2515 e Ljava/lang/IndexOutOfBoundsException;
52 56114 arg Ljava/lang/String;
36 6550012 i I
39 6549713 argsLength I
717 313 environmentLoader
Lorg/apache/beam/vendor/grpc/v1p26p0/org/jboss/modules/ModuleLoader;
749 6478712 loader
Lorg/apache/beam/vendor/grpc/v1p26p0/org/jboss/modules/ModuleLoader;
759 6477714 moduleIdentifier
Lorg/apache/beam/vendor/grpc/v1p26p0/org/jboss/modules/ModuleIdentifier;
0 012 loader
Lorg/apache/beam/vendor/grpc/v1p26p0/org/jboss/modules/ModuleLoader;
0 014 moduleIdentifier
Lorg/apache/beam/vendor/grpc/v1p26p0/org/jboss/modules/ModuleIdentifier;
0 020 i I
945 6459120 old Ljava/lang/ClassLoader;
0104517 reader Ljava/io/BufferedReader;
864 18118 name Ljava/lang/String;
872 17319 line Ljava/lang/String;
0 017 ignored Ljava/io/IOException;
1070 023 ignored Ljava/io/IOException;
0 015 classLoader
Lorg/apache/beam/vendor/grpc/v1p26p0/org/jboss/modules/ModuleClassLoader;
0 016 stream Ljava/io/InputStream;
0 016 e
Lorg/apache/beam/vendor/grpc/v1p26p0/org/jboss/modules/ModuleNotFoundException;
0114916 e
Ljava/lang/reflect/InvocationTargetException;
0 0 0 args [Ljava/lang/String;
3 65533 1 argsLen I
5 65531 2 deps Ljava/lang/String;
9 65527 3 moduleArgs [Ljava/lang/String;
12 65524 4 modulePath Ljava/lang/String;
15 65521 5 configPath Ljava/lang/String;
18 65518 6 classpath Ljava/lang/S
{noformat}
{noformat}
suztomo@suxtomo24:~/jbonofre-beam-samples$ java -cp
/usr/local/google/home/suztomo/.m2/repository/org/apache/beam/beam-vendor-grpc-1_26_0/0.1/beam-vendor-grpc-1_26_0-0.1.jar
org/apache/beam/vendor/grpc/v1p26p0/org/jboss/modules/Main
Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" java.lang.ClassFormatError: Invalid length 64974 in
LocalVariableTable in class file
org/apache/beam/vendor/grpc/v1p26p0/org/jboss/modules/Main
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:756)
at
java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:468)
at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:495)
{noformat}
> Problem shading Beam pipeline with Beam 2.20.0-SNAPSHOT
> ---
>
> Key: BEAM-9252
> URL: https://issues.apache.org/jira/browse/BEAM-9252
> Project: Beam
> Issue Type: Bug
> Components: build-system
>Affects Versions: 2.20.0
>Reporter: Ismaël Mejía
>Priority: Critical
> Fix For: 2.20.0
>
> Attachments: ArrayIndexOutOfBoundsException.png,
> image-2020-02-21-13-22-01-341.png
>
>
> I was checking today a pipeline against the latest 2.20.0-SNAPSHOT and I
> found that it works perfectly with version 2.19.0, but it is failing with a
> shade related exception that refers to grpc 1.26.0:
> {{[ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-shade-plugin:3.2.1:shade (default) on project
> EventsToIOs: Error creating shaded jar: Problem shading JAR
>
[jira] [Commented] (BEAM-9252) Problem shading Beam pipeline with Beam 2.20.0-SNAPSHOT
[
https://issues.apache.org/jira/browse/BEAM-9252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17042088#comment-17042088
]
Tomo Suzuki commented on BEAM-9252:
---
Looking into this.
!ArrayIndexOutOfBoundsException.png.png!
> Problem shading Beam pipeline with Beam 2.20.0-SNAPSHOT
> ---
>
> Key: BEAM-9252
> URL: https://issues.apache.org/jira/browse/BEAM-9252
> Project: Beam
> Issue Type: Bug
> Components: build-system
>Affects Versions: 2.20.0
>Reporter: Ismaël Mejía
>Priority: Critical
> Fix For: 2.20.0
>
> Attachments: ArrayIndexOutOfBoundsException.png
>
>
> I was checking today a pipeline against the latest 2.20.0-SNAPSHOT and I
> found that it works perfectly with version 2.19.0, but it is failing with a
> shade related exception that refers to grpc 1.26.0:
> {{[ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-shade-plugin:3.2.1:shade (default) on project
> EventsToIOs: Error creating shaded jar: Problem shading JAR
> /home/ismael/.m2/repository/org/apache/beam/beam-vendor-grpc-1_26_0/0.1/beam-vendor-grpc-1_26_0-0.1.jar
> entry org/apache/beam/vendor/grpc/v1p26p0/org/jboss/modules/Main.class:
> org.apache.maven.plugin.MojoExecutionException: Error in ASM processing class
> org/apache/beam/vendor/grpc/v1p26p0/org/jboss/modules/Main.class: 65536 ->
> [Help 1]}}
> {{There is also a warning that is not present in the build against 2.19.0}}
> {{[WARNING] Discovered module-info.class. Shading will break its strong
> encapsulation.}}
>
> I wonder if we are not doing something wrong during our vendoring, can
> someone take a look please.
> This is relatively easy to reproduce with the beam-samples repo, just clone
> it and run:
> {noformat}
> git clone https://github.com/jbonofre/beam-samples
> mvn clean verify -Pbeam-release-repo -Dbeam.version=2.20.0-SNAPSHOT
> {noformat}
> Available logs of the latest run:
> [https://github.com/jbonofre/beam-samples/runs/427537544?check_suite_focus=true]
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (BEAM-9252) Problem shading Beam pipeline with Beam 2.20.0-SNAPSHOT
[
https://issues.apache.org/jira/browse/BEAM-9252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17041986#comment-17041986
]
Ismaël Mejía commented on BEAM-9252:
I am trying to finish BEAM-9329 which has some API changes that I would like to
get into 2.20.0 (before the API gets out). I can take a look just after that,
but if [~suztomo] or someone else has some extra cycles before that it should
be better.
> Problem shading Beam pipeline with Beam 2.20.0-SNAPSHOT
> ---
>
> Key: BEAM-9252
> URL: https://issues.apache.org/jira/browse/BEAM-9252
> Project: Beam
> Issue Type: Bug
> Components: build-system
>Affects Versions: 2.20.0
>Reporter: Ismaël Mejía
>Priority: Critical
> Fix For: 2.20.0
>
>
> I was checking today a pipeline against the latest 2.20.0-SNAPSHOT and I
> found that it works perfectly with version 2.19.0, but it is failing with a
> shade related exception that refers to grpc 1.26.0:
> {{[ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-shade-plugin:3.2.1:shade (default) on project
> EventsToIOs: Error creating shaded jar: Problem shading JAR
> /home/ismael/.m2/repository/org/apache/beam/beam-vendor-grpc-1_26_0/0.1/beam-vendor-grpc-1_26_0-0.1.jar
> entry org/apache/beam/vendor/grpc/v1p26p0/org/jboss/modules/Main.class:
> org.apache.maven.plugin.MojoExecutionException: Error in ASM processing class
> org/apache/beam/vendor/grpc/v1p26p0/org/jboss/modules/Main.class: 65536 ->
> [Help 1]}}
> {{There is also a warning that is not present in the build against 2.19.0}}
> {{[WARNING] Discovered module-info.class. Shading will break its strong
> encapsulation.}}
>
> I wonder if we are not doing something wrong during our vendoring, can
> someone take a look please.
> This is relatively easy to reproduce with the beam-samples repo, just clone
> it and run:
> {noformat}
> git clone https://github.com/jbonofre/beam-samples
> mvn clean verify -Pbeam-release-repo -Dbeam.version=2.20.0-SNAPSHOT
> {noformat}
> Available logs of the latest run:
> [https://github.com/jbonofre/beam-samples/runs/427537544?check_suite_focus=true]
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (BEAM-9252) Problem shading Beam pipeline with Beam 2.20.0-SNAPSHOT
[
https://issues.apache.org/jira/browse/BEAM-9252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17041374#comment-17041374
]
Kenneth Knowles commented on BEAM-9252:
---
[~suztomo] or [~iemejia] would you have time to look at the module-info.class
and Main.class issues? I think BEAM-9288 is separate.
> Problem shading Beam pipeline with Beam 2.20.0-SNAPSHOT
> ---
>
> Key: BEAM-9252
> URL: https://issues.apache.org/jira/browse/BEAM-9252
> Project: Beam
> Issue Type: Bug
> Components: build-system
>Affects Versions: 2.20.0
>Reporter: Ismaël Mejía
>Priority: Critical
> Fix For: 2.20.0
>
>
> I was checking today a pipeline against the latest 2.20.0-SNAPSHOT and I
> found that it works perfectly with version 2.19.0, but it is failing with a
> shade related exception that refers to grpc 1.26.0:
> {{[ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-shade-plugin:3.2.1:shade (default) on project
> EventsToIOs: Error creating shaded jar: Problem shading JAR
> /home/ismael/.m2/repository/org/apache/beam/beam-vendor-grpc-1_26_0/0.1/beam-vendor-grpc-1_26_0-0.1.jar
> entry org/apache/beam/vendor/grpc/v1p26p0/org/jboss/modules/Main.class:
> org.apache.maven.plugin.MojoExecutionException: Error in ASM processing class
> org/apache/beam/vendor/grpc/v1p26p0/org/jboss/modules/Main.class: 65536 ->
> [Help 1]}}
> {{There is also a warning that is not present in the build against 2.19.0}}
> {{[WARNING] Discovered module-info.class. Shading will break its strong
> encapsulation.}}
>
> I wonder if we are not doing something wrong during our vendoring, can
> someone take a look please.
> This is relatively easy to reproduce with the beam-samples repo, just clone
> it and run:
> {noformat}
> git clone https://github.com/jbonofre/beam-samples
> mvn clean verify -Pbeam-release-repo -Dbeam.version=2.20.0-SNAPSHOT
> {noformat}
> Available logs of the latest run:
> [https://github.com/jbonofre/beam-samples/runs/427537544?check_suite_focus=true]
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (BEAM-9252) Problem shading Beam pipeline with Beam 2.20.0-SNAPSHOT
[
https://issues.apache.org/jira/browse/BEAM-9252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17037396#comment-17037396
]
Ahmet Altay commented on BEAM-9252:
---
Would this be resolved when https://issues.apache.org/jira/browse/BEAM-9288 is
fixed?
> Problem shading Beam pipeline with Beam 2.20.0-SNAPSHOT
> ---
>
> Key: BEAM-9252
> URL: https://issues.apache.org/jira/browse/BEAM-9252
> Project: Beam
> Issue Type: Bug
> Components: build-system
>Affects Versions: 2.20.0
>Reporter: Ismaël Mejía
>Priority: Critical
> Fix For: 2.20.0
>
>
> I was checking today a pipeline against the latest 2.20.0-SNAPSHOT and I
> found that it works perfectly with version 2.19.0, but it is failing with a
> shade related exception that refers to grpc 1.26.0:
> {{[ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-shade-plugin:3.2.1:shade (default) on project
> EventsToIOs: Error creating shaded jar: Problem shading JAR
> /home/ismael/.m2/repository/org/apache/beam/beam-vendor-grpc-1_26_0/0.1/beam-vendor-grpc-1_26_0-0.1.jar
> entry org/apache/beam/vendor/grpc/v1p26p0/org/jboss/modules/Main.class:
> org.apache.maven.plugin.MojoExecutionException: Error in ASM processing class
> org/apache/beam/vendor/grpc/v1p26p0/org/jboss/modules/Main.class: 65536 ->
> [Help 1]}}
> {{There is also a warning that is not present in the build against 2.19.0}}
> {{[WARNING] Discovered module-info.class. Shading will break its strong
> encapsulation.}}
>
> I wonder if we are not doing something wrong during our vendoring, can
> someone take a look please.
> This is relatively easy to reproduce with the beam-samples repo, just clone
> it and run:
> {noformat}
> git clone https://github.com/jbonofre/beam-samples
> mvn clean verify -Pbeam-release-repo -Dbeam.version=2.20.0-SNAPSHOT
> {noformat}
> Available logs of the latest run:
> [https://github.com/jbonofre/beam-samples/runs/427537544?check_suite_focus=true]
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (BEAM-9252) Problem shading Beam pipeline with Beam 2.20.0-SNAPSHOT
[
https://issues.apache.org/jira/browse/BEAM-9252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17037153#comment-17037153
]
Kenneth Knowles commented on BEAM-9252:
---
Per https://beam.apache.org/contribute/jira-priorities/ I think this is
probably P1 and definitely needs an owner as you mentioned on list. Definite
release blocker.
> Problem shading Beam pipeline with Beam 2.20.0-SNAPSHOT
> ---
>
> Key: BEAM-9252
> URL: https://issues.apache.org/jira/browse/BEAM-9252
> Project: Beam
> Issue Type: Bug
> Components: build-system
>Affects Versions: 2.20.0
>Reporter: Ismaël Mejía
>Priority: Blocker
> Fix For: 2.20.0
>
>
> I was checking today a pipeline against the latest 2.20.0-SNAPSHOT and I
> found that it works perfectly with version 2.19.0, but it is failing with a
> shade related exception that refers to grpc 1.26.0:
> {{[ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-shade-plugin:3.2.1:shade (default) on project
> EventsToIOs: Error creating shaded jar: Problem shading JAR
> /home/ismael/.m2/repository/org/apache/beam/beam-vendor-grpc-1_26_0/0.1/beam-vendor-grpc-1_26_0-0.1.jar
> entry org/apache/beam/vendor/grpc/v1p26p0/org/jboss/modules/Main.class:
> org.apache.maven.plugin.MojoExecutionException: Error in ASM processing class
> org/apache/beam/vendor/grpc/v1p26p0/org/jboss/modules/Main.class: 65536 ->
> [Help 1]}}
> {{There is also a warning that is not present in the build against 2.19.0}}
> {{[WARNING] Discovered module-info.class. Shading will break its strong
> encapsulation.}}
>
> I wonder if we are not doing something wrong during our vendoring, can
> someone take a look please.
> This is relatively easy to reproduce with the beam-samples repo, just clone
> it and run:
> {noformat}
> git clone https://github.com/jbonofre/beam-samples
> mvn clean verify -Pbeam-release-repo -Dbeam.version=2.20.0-SNAPSHOT
> {noformat}
> Available logs of the latest run:
> [https://github.com/jbonofre/beam-samples/runs/427537544?check_suite_focus=true]
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (BEAM-9252) Problem shading Beam pipeline with Beam 2.20.0-SNAPSHOT
[
https://issues.apache.org/jira/browse/BEAM-9252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17034013#comment-17034013
]
Luke Cwik commented on BEAM-9252:
-
It looks like conscrypt may have been shaded incorrectly and is causing
conflicts with other versions of conscrypt on the classpath due to JNI. See
BEAM-9288 for additional details.
> Problem shading Beam pipeline with Beam 2.20.0-SNAPSHOT
> ---
>
> Key: BEAM-9252
> URL: https://issues.apache.org/jira/browse/BEAM-9252
> Project: Beam
> Issue Type: Bug
> Components: build-system
>Affects Versions: 2.20.0
>Reporter: Ismaël Mejía
>Priority: Blocker
> Fix For: 2.20.0
>
>
> I was checking today a pipeline against the latest 2.20.0-SNAPSHOT and I
> found that it works perfectly with version 2.19.0, but it is failing with a
> shade related exception that refers to grpc 1.26.0:
> {{[ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-shade-plugin:3.2.1:shade (default) on project
> EventsToIOs: Error creating shaded jar: Problem shading JAR
> /home/ismael/.m2/repository/org/apache/beam/beam-vendor-grpc-1_26_0/0.1/beam-vendor-grpc-1_26_0-0.1.jar
> entry org/apache/beam/vendor/grpc/v1p26p0/org/jboss/modules/Main.class:
> org.apache.maven.plugin.MojoExecutionException: Error in ASM processing class
> org/apache/beam/vendor/grpc/v1p26p0/org/jboss/modules/Main.class: 65536 ->
> [Help 1]}}
> {{There is also a warning that is not present in the build against 2.19.0}}
> {{[WARNING] Discovered module-info.class. Shading will break its strong
> encapsulation.}}
>
> I wonder if we are not doing something wrong during our vendoring, can
> someone take a look please.
> This is relatively easy to reproduce with the beam-samples repo, just clone
> it and run:
> {noformat}
> git clone https://github.com/jbonofre/beam-samples
> mvn clean verify -Pbeam-release-repo -Dbeam.version=2.20.0-SNAPSHOT
> {noformat}
> Available logs of the latest run:
> [https://github.com/jbonofre/beam-samples/runs/427537544?check_suite_focus=true]
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (BEAM-9252) Problem shading Beam pipeline with Beam 2.20.0-SNAPSHOT
[
https://issues.apache.org/jira/browse/BEAM-9252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17032096#comment-17032096
]
Kenneth Knowles commented on BEAM-9252:
---
Within that repo getJarName is not reference but getVersionString is references
at
https://github.com/jboss-modules/jboss-modules/blob/4d9d040b248523ad21b2d6a9c7cc061fb7db1565/src/main/java/org/jboss/modules/log/JDKModuleLogger.java#L183
Since they are public, you would have to search the other class files, which
actually should be quite easy since the string will be present.
> Problem shading Beam pipeline with Beam 2.20.0-SNAPSHOT
> ---
>
> Key: BEAM-9252
> URL: https://issues.apache.org/jira/browse/BEAM-9252
> Project: Beam
> Issue Type: Bug
> Components: build-system
>Affects Versions: 2.20.0
>Reporter: Ismaël Mejía
>Priority: Blocker
> Fix For: 2.20.0
>
>
> I was checking today a pipeline against the latest 2.20.0-SNAPSHOT and I
> found that it works perfectly with version 2.19.0, but it is failing with a
> shade related exception that refers to grpc 1.26.0:
> {{[ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-shade-plugin:3.2.1:shade (default) on project
> EventsToIOs: Error creating shaded jar: Problem shading JAR
> /home/ismael/.m2/repository/org/apache/beam/beam-vendor-grpc-1_26_0/0.1/beam-vendor-grpc-1_26_0-0.1.jar
> entry org/apache/beam/vendor/grpc/v1p26p0/org/jboss/modules/Main.class:
> org.apache.maven.plugin.MojoExecutionException: Error in ASM processing class
> org/apache/beam/vendor/grpc/v1p26p0/org/jboss/modules/Main.class: 65536 ->
> [Help 1]}}
> {{There is also a warning that is not present in the build against 2.19.0}}
> {{[WARNING] Discovered module-info.class. Shading will break its strong
> encapsulation.}}
>
> I wonder if we are not doing something wrong during our vendoring, can
> someone take a look please.
> This is relatively easy to reproduce with the beam-samples repo, just clone
> it and run:
> {noformat}
> git clone https://github.com/jbonofre/beam-samples
> mvn clean verify -Pbeam-release-repo -Dbeam.version=2.20.0-SNAPSHOT
> {noformat}
> Available logs of the latest run:
> [https://github.com/jbonofre/beam-samples/runs/427537544?check_suite_focus=true]
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (BEAM-9252) Problem shading Beam pipeline with Beam 2.20.0-SNAPSHOT
[
https://issues.apache.org/jira/browse/BEAM-9252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17030906#comment-17030906
]
Luke Cwik commented on BEAM-9252:
-
We might be able to remove Main.class as well since it doesn't have many public
methods but I do worry that either *getJarName* or *getVersionString* are
referenced elsewhere.
> Problem shading Beam pipeline with Beam 2.20.0-SNAPSHOT
> ---
>
> Key: BEAM-9252
> URL: https://issues.apache.org/jira/browse/BEAM-9252
> Project: Beam
> Issue Type: Bug
> Components: build-system
>Affects Versions: 2.20.0
>Reporter: Ismaël Mejía
>Priority: Blocker
> Fix For: 2.20.0
>
>
> I was checking today a pipeline against the latest 2.20.0-SNAPSHOT and I
> found that it works perfectly with version 2.19.0, but it is failing with a
> shade related exception that refers to grpc 1.26.0:
> {{[ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-shade-plugin:3.2.1:shade (default) on project
> EventsToIOs: Error creating shaded jar: Problem shading JAR
> /home/ismael/.m2/repository/org/apache/beam/beam-vendor-grpc-1_26_0/0.1/beam-vendor-grpc-1_26_0-0.1.jar
> entry org/apache/beam/vendor/grpc/v1p26p0/org/jboss/modules/Main.class:
> org.apache.maven.plugin.MojoExecutionException: Error in ASM processing class
> org/apache/beam/vendor/grpc/v1p26p0/org/jboss/modules/Main.class: 65536 ->
> [Help 1]}}
> {{There is also a warning that is not present in the build against 2.19.0}}
> {{[WARNING] Discovered module-info.class. Shading will break its strong
> encapsulation.}}
>
> I wonder if we are not doing something wrong during our vendoring, can
> someone take a look please.
> This is relatively easy to reproduce with the beam-samples repo, just clone
> it and run:
> {noformat}
> git clone https://github.com/jbonofre/beam-samples
> mvn clean verify -Pbeam-release-repo -Dbeam.version=2.20.0-SNAPSHOT
> {noformat}
> Available logs of the latest run:
> [https://github.com/jbonofre/beam-samples/runs/427537544?check_suite_focus=true]
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (BEAM-9252) Problem shading Beam pipeline with Beam 2.20.0-SNAPSHOT
[
https://issues.apache.org/jira/browse/BEAM-9252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17030905#comment-17030905
]
Luke Cwik commented on BEAM-9252:
-
Yes to removing module-info.class but it would be good to figure out the Main
class issue if possible before performing a 0.2 release.
A cursory look at they bytecode doesn't provide any details as to why
Main.class is failing. I tried updating ASM to 7.3.1 in the maven-shade-plugin
within the beam-samples project to see if the issue had been fixed but that
didn't work.
> Problem shading Beam pipeline with Beam 2.20.0-SNAPSHOT
> ---
>
> Key: BEAM-9252
> URL: https://issues.apache.org/jira/browse/BEAM-9252
> Project: Beam
> Issue Type: Bug
> Components: build-system
>Affects Versions: 2.20.0
>Reporter: Ismaël Mejía
>Priority: Blocker
> Fix For: 2.20.0
>
>
> I was checking today a pipeline against the latest 2.20.0-SNAPSHOT and I
> found that it works perfectly with version 2.19.0, but it is failing with a
> shade related exception that refers to grpc 1.26.0:
> {{[ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-shade-plugin:3.2.1:shade (default) on project
> EventsToIOs: Error creating shaded jar: Problem shading JAR
> /home/ismael/.m2/repository/org/apache/beam/beam-vendor-grpc-1_26_0/0.1/beam-vendor-grpc-1_26_0-0.1.jar
> entry org/apache/beam/vendor/grpc/v1p26p0/org/jboss/modules/Main.class:
> org.apache.maven.plugin.MojoExecutionException: Error in ASM processing class
> org/apache/beam/vendor/grpc/v1p26p0/org/jboss/modules/Main.class: 65536 ->
> [Help 1]}}
> {{There is also a warning that is not present in the build against 2.19.0}}
> {{[WARNING] Discovered module-info.class. Shading will break its strong
> encapsulation.}}
>
> I wonder if we are not doing something wrong during our vendoring, can
> someone take a look please.
> This is relatively easy to reproduce with the beam-samples repo, just clone
> it and run:
> {noformat}
> git clone https://github.com/jbonofre/beam-samples
> mvn clean verify -Pbeam-release-repo -Dbeam.version=2.20.0-SNAPSHOT
> {noformat}
> Available logs of the latest run:
> [https://github.com/jbonofre/beam-samples/runs/427537544?check_suite_focus=true]
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (BEAM-9252) Problem shading Beam pipeline with Beam 2.20.0-SNAPSHOT
[
https://issues.apache.org/jira/browse/BEAM-9252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17030902#comment-17030902
]
Tomo Suzuki commented on BEAM-9252:
---
Shall we remove module-info.class from the vendored artifact?
{noformat}
suztomo@suxtomo24:~/beam-samples$ jar tf
/usr/local/google/home/suztomo/.m2/repository/org/apache/beam/beam-vendor-grpc-1_26_0/0.1/beam-vendor-grpc-1_26_0-0.1.jar
|grep module-info
module-info.class
{noformat}
[~lcwik] I haven't figured out why maven-shade-plugin is complaining jboss's
Main class. (Is your decompile result explaining that?)
> Problem shading Beam pipeline with Beam 2.20.0-SNAPSHOT
> ---
>
> Key: BEAM-9252
> URL: https://issues.apache.org/jira/browse/BEAM-9252
> Project: Beam
> Issue Type: Bug
> Components: build-system
>Affects Versions: 2.20.0
>Reporter: Ismaël Mejía
>Priority: Blocker
> Fix For: 2.20.0
>
>
> I was checking today a pipeline against the latest 2.20.0-SNAPSHOT and I
> found that it works perfectly with version 2.19.0, but it is failing with a
> shade related exception that refers to grpc 1.26.0:
> {{[ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-shade-plugin:3.2.1:shade (default) on project
> EventsToIOs: Error creating shaded jar: Problem shading JAR
> /home/ismael/.m2/repository/org/apache/beam/beam-vendor-grpc-1_26_0/0.1/beam-vendor-grpc-1_26_0-0.1.jar
> entry org/apache/beam/vendor/grpc/v1p26p0/org/jboss/modules/Main.class:
> org.apache.maven.plugin.MojoExecutionException: Error in ASM processing class
> org/apache/beam/vendor/grpc/v1p26p0/org/jboss/modules/Main.class: 65536 ->
> [Help 1]}}
> {{There is also a warning that is not present in the build against 2.19.0}}
> {{[WARNING] Discovered module-info.class. Shading will break its strong
> encapsulation.}}
>
> I wonder if we are not doing something wrong during our vendoring, can
> someone take a look please.
> This is relatively easy to reproduce with the beam-samples repo, just clone
> it and run:
> {noformat}
> git clone https://github.com/jbonofre/beam-samples
> mvn clean verify -Pbeam-release-repo -Dbeam.version=2.20.0-SNAPSHOT
> {noformat}
> Available logs of the latest run:
> [https://github.com/jbonofre/beam-samples/runs/427537544?check_suite_focus=true]
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (BEAM-9252) Problem shading Beam pipeline with Beam 2.20.0-SNAPSHOT
[
https://issues.apache.org/jira/browse/BEAM-9252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17030897#comment-17030897
]
Luke Cwik commented on BEAM-9252:
-
ASM and javac both have bugs so I'm not surprised that shading exposed yet
another one.
The *module-info.class* is invalid since it contains:
{code:java}
module com.google.gson {
requires java.base;
requires transitive java.sql;
exports com.google.gson;
exports com.google.gson.annotations;
exports com.google.gson.reflect;
exports com.google.gson.stream;
}{code}
The decompiled *Main.class* looks like:
{code:java}
package org.apache.beam.vendor.grpc.v1p26p0.org.jboss.modules;
import java.io.File;
import java.security.AccessController;
import java.security.PrivilegedAction;
public final class Main {
static {
long start = StartTimeHolder.START_TIME;
}
private static final String[] NO_STRINGS = new String[0];
private static void usage() {
System.out.println("Usage: java [-jvmoptions...] -jar " + getJarName() +
".jar [-options...] [args...]");
System.out.println(" java [-jvmoptions...] -jar " + getJarName() +
".jar [-options...] -jar [args...]");
System.out.println(" java [-jvmoptions...] -jar " + getJarName() +
".jar [-options...] -cp [args...]");
System.out.println(" java [-jvmoptions...] -jar " + getJarName() +
".jar [-options...] -class [args...]");
System.out.println("where is a valid module specification
string");
System.out.println("and options include:");
System.out.println("-help Display this message");
System.out.println("-modulepath ");
System.out.println("-mp ");
System.out.println(" A list of directories, separated by
'" + File.pathSeparator + "', where modules may be located");
System.out.println(" If not specified, the value of the
\"module.path\" system property is used");
System.out.println("-classSpecify that the final argument is
a");
System.out.println(" class to load from the class path;
not compatible with -jar");
System.out.println("-cp,-classpath ");
System.out.println(" A search path for class files;
implies -class");
System.out.println("-dep,-dependencies
[,,...]");
System.out.println(" A list of module dependencies to add
to the class path;");
System.out.println(" requires -class or -cp");
System.out.println("-jar Specify that the final argument is
the name of a");
System.out.println(" JAR file to run as a module; not
compatible with -class");
System.out.println("-config ");
System.out.println(" The location of the module
configuration. Either -mp or -config");
System.out.println(" may be specified, but not both");
System.out.println("-logmodule ");
System.out.println(" The module to use to load the system
logmanager");
System.out.println("-version Print version and exit\n");
}
public static void main(String[] args) throws Throwable {
// Byte code:
// 0: aload_0
// 1: arraylength
// 2: istore_1
// 3: aconst_null
// 4: astore_2
// 5: getstatic
org/apache/beam/vendor/grpc/v1p26p0/org/jboss/modules/Main.NO_STRINGS :
[Ljava/lang/String;
// 8: astore_3
// 9: aconst_null
// 10: astore #4
// 12: aconst_null
// 13: astore #5
// 15: aconst_null
// 16: astore #6
// 18: iconst_0
// 19: istore #7
// 21: iconst_0
// 22: istore #8
// 24: iconst_0
// 25: istore #9
// 27: aconst_null
// 28: astore #10
// 30: aconst_null
// 31: astore #11
// 33: iconst_0
// 34: istore #12
// 36: iload_1
// 37: istore #13
// 39: iload #12
// 41: iload #13
// 43: if_icmpge -> 649
// 46: aload_0
// 47: iload #12
// 49: aaload
// 50: astore #14
// 52: aload #14
// 54: iconst_0
// 55: invokevirtual charAt : (I)C
// 58: bipush #45
// 60: if_icmpne -> 580
// 63: ldc '-version'
// 65: aload #14
// 67: invokevirtual equals : (Ljava/lang/Object;)Z
// 70: ifeq -> 101
// 73: getstatic java/lang/System.out : Ljava/io/PrintStream;
// 76: new java/lang/StringBuilder
// 79: dup
// 80: invokespecial : ()V
// 83: ldc 'Module loader '
// 85: invokevirtual append :
(Ljava/lang/String;)Ljava/lang/StringBuilder;
// 88: invokestatic getVersionString : ()Ljava/lang/String;
// 91: invokevirtual append :
(Ljava/lang/String;)Ljava/lang/StringBuilder;
// 94: invokevirtual toString : ()Ljava/lang/String;
// 97: invokevirtual println : (Ljava/lang/String;)V
// 100: return
[jira] [Commented] (BEAM-9252) Problem shading Beam pipeline with Beam 2.20.0-SNAPSHOT
[
https://issues.apache.org/jira/browse/BEAM-9252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17030727#comment-17030727
]
Ismaël Mejía commented on BEAM-9252:
Thanks for double checking Tomo. The commit you mention is the one that bumps
the version of the vendored dependency that is then published independently of
the Beam release (all of that happened during 2.19.0). The use of the
dependency was introduced in master by this commit
[https://github.com/apache/beam/commit/2fd785d] so it is now part of
2.20.0-SNAPSHOT
> Problem shading Beam pipeline with Beam 2.20.0-SNAPSHOT
> ---
>
> Key: BEAM-9252
> URL: https://issues.apache.org/jira/browse/BEAM-9252
> Project: Beam
> Issue Type: Bug
> Components: build-system
>Affects Versions: 2.20.0
>Reporter: Ismaël Mejía
>Priority: Blocker
> Fix For: 2.20.0
>
>
> I was checking today a pipeline against the latest 2.20.0-SNAPSHOT and I
> found that it works perfectly with version 2.19.0, but it is failing with a
> shade related exception that refers to grpc 1.26.0:
> {{[ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-shade-plugin:3.2.1:shade (default) on project
> EventsToIOs: Error creating shaded jar: Problem shading JAR
> /home/ismael/.m2/repository/org/apache/beam/beam-vendor-grpc-1_26_0/0.1/beam-vendor-grpc-1_26_0-0.1.jar
> entry org/apache/beam/vendor/grpc/v1p26p0/org/jboss/modules/Main.class:
> org.apache.maven.plugin.MojoExecutionException: Error in ASM processing class
> org/apache/beam/vendor/grpc/v1p26p0/org/jboss/modules/Main.class: 65536 ->
> [Help 1]}}
> {{There is also a warning that is not present in the build against 2.19.0}}
> {{[WARNING] Discovered module-info.class. Shading will break its strong
> encapsulation.}}
>
> I wonder if we are not doing something wrong during our vendoring, can
> someone take a look please.
> This is relatively easy to reproduce with the beam-samples repo, just clone
> it and run:
> {{git clone [https://github.com/jbonofre/beam-samples
> ]mvn clean verify -Pbeam-release-repo -Dbeam.version=2.20.0-SNAPSHOT}}
> Available logs of the latest run:
> [https://github.com/jbonofre/beam-samples/runs/427537544?check_suite_focus=true]
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (BEAM-9252) Problem shading Beam pipeline with Beam 2.20.0-SNAPSHOT
[
https://issues.apache.org/jira/browse/BEAM-9252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17030700#comment-17030700
]
Tomo Suzuki commented on BEAM-9252:
---
I confirmed that I can reproduce the issue (using bonofre/beam-samples) in
2.20.0-SNAPSHOT, while 2.19.0 works fine.
I see this grpc-vendoring commit has been included in 2.*19*.0.
https://github.com/apache/beam/commit/282bd3645647e29c6889bb61d6f7248e6bb48915
. What has changed for grpc vendoring since then?
> Problem shading Beam pipeline with Beam 2.20.0-SNAPSHOT
> ---
>
> Key: BEAM-9252
> URL: https://issues.apache.org/jira/browse/BEAM-9252
> Project: Beam
> Issue Type: Bug
> Components: build-system
>Affects Versions: 2.20.0
>Reporter: Ismaël Mejía
>Priority: Blocker
> Fix For: 2.20.0
>
>
> I was checking today a pipeline against the latest 2.20.0-SNAPSHOT and I
> found that it works perfectly with version 2.19.0, but it is failing with a
> shade related exception that refers to grpc 1.26.0:
> {{[ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-shade-plugin:3.2.1:shade (default) on project
> EventsToIOs: Error creating shaded jar: Problem shading JAR
> /home/ismael/.m2/repository/org/apache/beam/beam-vendor-grpc-1_26_0/0.1/beam-vendor-grpc-1_26_0-0.1.jar
> entry org/apache/beam/vendor/grpc/v1p26p0/org/jboss/modules/Main.class:
> org.apache.maven.plugin.MojoExecutionException: Error in ASM processing class
> org/apache/beam/vendor/grpc/v1p26p0/org/jboss/modules/Main.class: 65536 ->
> [Help 1]}}
> {{There is also a warning that is not present in the build against 2.19.0}}
> {{[WARNING] Discovered module-info.class. Shading will break its strong
> encapsulation.}}
>
> I wonder if we are not doing something wrong during our vendoring, can
> someone take a look please.
> This is relatively easy to reproduce with the beam-samples repo, just clone
> it and run:
> {{git clone [https://github.com/jbonofre/beam-samples
> ]mvn clean verify -Pbeam-release-repo -Dbeam.version=2.20.0-SNAPSHOT}}
> Available logs of the latest run:
> [https://github.com/jbonofre/beam-samples/runs/427537544?check_suite_focus=true]
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (BEAM-9252) Problem shading Beam pipeline with Beam 2.20.0-SNAPSHOT
[
https://issues.apache.org/jira/browse/BEAM-9252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17030668#comment-17030668
]
Ismaël Mejía commented on BEAM-9252:
PInging you guys to see if someone can figure this out: [~sunjincheng121]
[~lcwik] and [~suztomo] I am lost in particular because of the jboss class.
> Problem shading Beam pipeline with Beam 2.20.0-SNAPSHOT
> ---
>
> Key: BEAM-9252
> URL: https://issues.apache.org/jira/browse/BEAM-9252
> Project: Beam
> Issue Type: Bug
> Components: build-system
>Affects Versions: 2.20.0
>Reporter: Ismaël Mejía
>Priority: Blocker
> Fix For: 2.20.0
>
>
> I was checking today a pipeline against the latest 2.20.0-SNAPSHOT and I
> found that it used to work perfectly and now is failing to shade with an
> exception that refers to grpc 1.26.0
> {{[ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-shade-plugin:3.2.1:shade (default) on project
> EventsToIOs: Error creating shaded jar: Problem shading JAR
> /home/ismael/.m2/repository/org/apache/beam/beam-vendor-grpc-1_26_0/0.1/beam-vendor-grpc-1_26_0-0.1.jar
> entry org/apache/beam/vendor/grpc/v1p26p0/org/jboss/modules/Main.class:
> org.apache.maven.plugin.MojoExecutionException: Error in ASM processing class
> org/apache/beam/vendor/grpc/v1p26p0/org/jboss/modules/Main.class: 65536 ->
> [Help 1]}}
> {{There is also a warning that is not present in the build against 2.19.0}}
> {{[WARNING] Discovered module-info.class. Shading will break its strong
> encapsulation.}}
>
> I wonder if we are not doing something wrong during our vendoring, can
> someone take a look please.
> This is relatively easy to reproduce with the beam-samples repo
> https://github.com/jbonofre/beam-samples/runs/427537544?check_suite_focus=true
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
