[
https://issues.apache.org/jira/browse/FLINK-36646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17894829#comment-17894829
]
Thomas Cooper commented on FLINK-36646:
---------------------------------------
We could set the {{env.java.default-opts.all}} option in the operator's [base
config
file|https://github.com/apache/flink-kubernetes-operator/blob/main/helm/flink-kubernetes-operator/conf/flink-conf.yaml]
to the flink config's {{add-export}} and {{add-open}} opts?
That would allow users to add additional {{env.java.opts.all}} to their jobs.
As we are using {{-XX:IgnoreUnrecognizedVMOptions}}, this should be fine for
JDK 8 and JDK 11 based images as well.
> Support Java 17+ job images more conveniently in the operator
> -------------------------------------------------------------
>
> Key: FLINK-36646
> URL: https://issues.apache.org/jira/browse/FLINK-36646
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
> Reporter: Rob Young
> Priority: Major
>
> Currently if I execute the operator quickstart (for operator 1.10.0) , but
> change the FlinkDeployment image to one of the official java17 based images
> like `flink:1.20.0-scala_2.12-java17`:
> {code:java}
> apiVersion: flink.apache.org/v1beta1
> kind: FlinkDeployment
> metadata:
> name: basic-example
> spec:
> image: flink:1.20.0-scala_2.12-java17
> flinkVersion: v1_20
> flinkConfiguration:
> taskmanager.numberOfTaskSlots: "2"
> serviceAccount: flink
> jobManager:
> resource:
> memory: "2048m"
> cpu: 1
> taskManager:
> resource:
> memory: "2048m"
> cpu: 1
> job:
> jarURI: local:///opt/flink/examples/streaming/StateMachineExample.jar
> parallelism: 2
> upgradeMode: stateless {code}
> then the application fails with exception, eventually caused by:
> {code:java}
> aused by: java.lang.reflect.InaccessibleObjectException: Unable to make field
> private final java.util.Map java.util.Collections$UnmodifiableMap.m
> accessible: module java.base does not "opens java.util" to unnamed module
> @21a947fe
> at java.lang.reflect.AccessibleObject.checkCanSetAccessible(Unknown
> Source) ~[?:?]
> at java.lang.reflect.AccessibleObject.checkCanSetAccessible(Unknown
> Source) ~[?:?]
> at java.lang.reflect.Field.checkCanSetAccessible(Unknown Source) ~[?:?]
> at java.lang.reflect.Field.setAccessible(Unknown Source) ~[?:?]
> at
> org.apache.flink.streaming.runtime.translators.DataStreamV2SinkTransformationTranslator.registerSinkTransformationTranslator(DataStreamV2SinkTransformationTranslator.java:104)
> ~[flink-dist-1.20.0.jar:1.20.0]
> at
> org.apache.flink.datastream.impl.ExecutionEnvironmentImpl.<clinit>(ExecutionEnvironmentImpl.java:96)
> ~[flink-dist-1.20.0.jar:1.20.0]
> at
> org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:109)
> ~[flink-dist-1.20.0.jar:1.20.0]
> at
> org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.runApplicationEntryPoint(ApplicationDispatcherBootstrap.java:301)
> ~[flink-dist-1.20.0.jar:1.20.0]
> at
> org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.lambda$runApplicationAsync$2(ApplicationDispatcherBootstrap.java:254)
> ~[flink-dist-1.20.0.jar:1.20.0]
> at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
> ~[?:?]
> at java.util.concurrent.FutureTask.run(Unknown Source) ~[?:?]
> at
> org.apache.flink.runtime.concurrent.pekko.ActorSystemScheduledExecutorAdapter$ScheduledFutureTask.run(ActorSystemScheduledExecutorAdapter.java:172)
> ~[?:?]
> at
> org.apache.flink.runtime.concurrent.ClassLoadingUtils.runWithContextClassLoader(ClassLoadingUtils.java:68)
> ~[flink-dist-1.20.0.jar:1.20.0]
> at
> org.apache.flink.runtime.concurrent.ClassLoadingUtils.lambda$withContextClassLoader$0(ClassLoadingUtils.java:41)
> ~[flink-dist-1.20.0.jar:1.20.0]
> ... 7 more {code}
> the user can fix this by setting:
> {code:java}
> env.java.opts.all: "--add-exports=java.base/sun.net.util=ALL-UNNAMED
> --add-exports=java.rmi/sun.rmi.registry=ALL-UNNAMED
> --add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED
> --add-exports=jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED
> --add-exports=jdk.compiler/com.sun.tools.javac.parser=ALL-UNNAMED
> --add-exports=jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED
> --add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED
> --add-exports=java.security.jgss/sun.security.krb5=ALL-UNNAMED
> --add-opens=java.base/java.lang=ALL-UNNAMED
> --add-opens=java.base/java.net=ALL-UNNAMED
> --add-opens=java.base/java.io=ALL-UNNAMED
> --add-opens=java.base/java.nio=ALL-UNNAMED
> --add-opens=java.base/sun.nio.ch=ALL-UNNAMED
> --add-opens=java.base/java.lang.reflect=ALL-UNNAMED
> --add-opens=java.base/java.text=ALL-UNNAMED
> --add-opens=java.base/java.time=ALL-UNNAMED
> --add-opens=java.base/java.util=ALL-UNNAMED
> --add-opens=java.base/java.util.concurrent=ALL-UNNAMED
> --add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED
> --add-opens=java.base/java.util.concurrent.locks=ALL-UNNAMED"{code}
> in the FlinkDeployment flinkConfiguration, matching the [default
> configuration from the flink binary
> distribution|https://github.com/apache/flink/blob/b1fe7b4099497f02b4658df7c3de8e45b62b7e21/flink-dist/src/main/resources/config.yaml#L24]
> It would be cool if you could use Java 17+ based job images without having to
> set this up and the operator or flink shell scripts could set this up for you.
> From flink 1.18 the config.sh script is setting
> `-XX:IgnoreUnrecognizedVMOptions` so it would appear safe to redundantly set
> these JVM params for Flink 1_18\{+}. Maybe if there is no user-supplied
> `env.java.opts.all` the operator could set it, else append them to the
> user-supplied opts.
> I'm wondering if this should be a core flink responsibility though, maybe
> `config.sh` could set these parameters, recent versions of config.sh are
> already [sniffing for JVM
> 17+|https://github.com/apache/flink/blob/b1fe7b4099497f02b4658df7c3de8e45b62b7e21/flink-dist/src/main/flink-bin/bin/config.sh#L288]
> so that it can set `-Djava.security.manager=allow`, maybe it would be
> appropriate for it to also set the add-opens up too?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
