tagarr commented on PR #689:
URL:
https://github.com/apache/flink-kubernetes-operator/pull/689#issuecomment-1802587767
Hi @gaborgsomogyi I didn't appreciate that there could be 100's or more of
FlinkDeployments running on a cluster. If that's the case then my solution
wouldn't be the best. What if I provide an optional secret mount for the
truststore and optional secretKeyRef env var for the store password. Then
modify the config for creating the rest client for the operator to point to
this store ?
Additionally, I only created the OperatorKubernetesClusterDescriptor class
so that the call to deployClusterInternal didn't throw an exception as the
config for the actual cluster was being used. If instead of doing this I caught
the exception and checked that it was a ClusterRetrieveException I would be
able to reduce the changes considerably. Do you think this would be acceptable ?
For reference the exception thrown by the operator is:
```
[m[33m2023-11-08 15:50:27,797[m [36mo.a.f.k.o.l.AuditUtils [m
[32m[INFO ][flink/basic-secure] >>> Event | Warning |
CLUSTERDEPLOYMENTEXCEPTION |
org.apache.flink.client.deployment.ClusterRetrieveException: Could not create
the RestClusterClient.
[m[33m2023-11-08 15:50:27,800[m [36mo.a.f.k.o.r.ReconciliationUtils[m
[33m[WARN ][flink/basic-secure] Attempt count: 0, last attempt: false
[m[33m2023-11-08 15:50:27,886[m [36mo.a.f.k.o.l.AuditUtils [m
[32m[INFO ][flink/basic-secure] >>> Status | Error | UPGRADING |
{"type":"org.apache.flink.kubernetes.operator.exception.ReconciliationException","message":"java.lang.RuntimeException:
org.apache.flink.client.deployment.ClusterRetrieveException: Could not create
the
RestClusterClient.","additionalMetadata":{},"throwableList":[{"type":"java.lang.RuntimeException","message":"org.apache.flink.client.deployment.ClusterRetrieveException:
Could not create the
RestClusterClient.","additionalMetadata":{}},{"type":"org.apache.flink.client.deployment.ClusterRetrieveException","message":"Could
not create the RestClusterClient.","additionalMetadata":{}}]}
[m[33m2023-11-08 15:50:27,890[m
[36mi.j.o.p.e.ReconciliationDispatcher[m [1;31m[ERROR][flink/basic-secure]
Error during event processing ExecutionScope{ resource id:
ResourceID{name='basic-secure', namespace='flink'}, version: 7366433} failed.
org.apache.flink.kubernetes.operator.exception.ReconciliationException:
java.lang.RuntimeException:
org.apache.flink.client.deployment.ClusterRetrieveException: Could not create
the RestClusterClient.
at
org.apache.flink.kubernetes.operator.controller.FlinkDeploymentController.reconcile(FlinkDeploymentController.java:148)
at
org.apache.flink.kubernetes.operator.controller.FlinkDeploymentController.reconcile(FlinkDeploymentController.java:56)
at
io.javaoperatorsdk.operator.processing.Controller$1.execute(Controller.java:138)
at
io.javaoperatorsdk.operator.processing.Controller$1.execute(Controller.java:96)
at
org.apache.flink.kubernetes.operator.metrics.OperatorJosdkMetrics.timeControllerExecution(OperatorJosdkMetrics.java:80)
at
io.javaoperatorsdk.operator.processing.Controller.reconcile(Controller.java:95)
at
io.javaoperatorsdk.operator.processing.event.ReconciliationDispatcher.reconcileExecution(ReconciliationDispatcher.java:139)
at
io.javaoperatorsdk.operator.processing.event.ReconciliationDispatcher.handleReconcile(ReconciliationDispatcher.java:119)
at
io.javaoperatorsdk.operator.processing.event.ReconciliationDispatcher.handleDispatch(ReconciliationDispatcher.java:89)
at
io.javaoperatorsdk.operator.processing.event.ReconciliationDispatcher.handleExecution(ReconciliationDispatcher.java:62)
at
io.javaoperatorsdk.operator.processing.event.EventProcessor$ReconcilerExecutor.run(EventProcessor.java:414)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown
Source)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
Source)
at java.base/java.lang.Thread.run(Unknown Source)
Caused by: java.lang.RuntimeException:
org.apache.flink.client.deployment.ClusterRetrieveException: Could not create
the RestClusterClient.
at
org.apache.flink.kubernetes.KubernetesClusterDescriptor.lambda$createClusterClientProvider$1(KubernetesClusterDescriptor.java:121)
at
org.apache.flink.kubernetes.KubernetesClusterDescriptor.deployApplicationCluster(KubernetesClusterDescriptor.java:217)
at
org.apache.flink.client.deployment.application.cli.ApplicationClusterDeployer.run(ApplicationClusterDeployer.java:67)
at
org.apache.flink.kubernetes.operator.service.NativeFlinkService.deployApplicationCluster(NativeFlinkService.java:104)
at
org.apache.flink.kubernetes.operator.service.AbstractFlinkService.submitApplicationCluster(AbstractFlinkService.java:189)
at
org.apache.flink.kubernetes.operator.reconciler.deployment.ApplicationReconciler.deploy(ApplicationReconciler.java:182)
at
org.apache.flink.kubernetes.operator.reconciler.deployment.ApplicationReconciler.deploy(ApplicationReconciler.java:60)
at
org.apache.flink.kubernetes.operator.reconciler.deployment.AbstractFlinkResourceReconciler.reconcile(AbstractFlinkResourceReconciler.java:120)
at
org.apache.flink.kubernetes.operator.controller.FlinkDeploymentController.reconcile(FlinkDeploymentController.java:136)
... 13 more
Caused by: org.apache.flink.client.deployment.ClusterRetrieveException:
Could not create the RestClusterClient.
... 22 more
Caused by: org.apache.flink.util.ConfigurationException: Failed to
initialize SSLContext for the REST client
at
org.apache.flink.runtime.rest.RestClientConfiguration.fromConfiguration(RestClientConfiguration.java:107)
at
org.apache.flink.client.program.rest.RestClusterClientConfiguration.fromConfiguration(RestClusterClientConfiguration.java:78)
at
org.apache.flink.client.program.rest.RestClusterClient.<init>(RestClusterClient.java:225)
at
org.apache.flink.client.program.rest.RestClusterClient.<init>(RestClusterClient.java:197)
at
org.apache.flink.kubernetes.KubernetesClusterDescriptor.lambda$createClusterClientProvider$1(KubernetesClusterDescriptor.java:114)
... 21 more
Caused by: java.nio.file.NoSuchFileException: /etc/tls/truststore.jks
at java.base/sun.nio.fs.UnixException.translateToIOException(Unknown
Source)
at java.base/sun.nio.fs.UnixException.rethrowAsIOException(Unknown
Source)
at java.base/sun.nio.fs.UnixException.rethrowAsIOException(Unknown
Source)
at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(Unknown
Source)
at java.base/java.nio.file.Files.newByteChannel(Unknown Source)
at java.base/java.nio.file.Files.newByteChannel(Unknown Source)
at
java.base/java.nio.file.spi.FileSystemProvider.newInputStream(Unknown Source)
at java.base/java.nio.file.Files.newInputStream(Unknown Source)
at
org.apache.flink.runtime.net.SSLUtils.getTrustManagerFactory(SSLUtils.java:218)
at
org.apache.flink.runtime.net.SSLUtils.createRestNettySSLContext(SSLUtils.java:400)
at
org.apache.flink.runtime.net.SSLUtils.createRestNettySSLContext(SSLUtils.java:367)
at
org.apache.flink.runtime.net.SSLUtils.createRestClientSSLEngineFactory(SSLUtils.java:161)
at
org.apache.flink.runtime.rest.RestClientConfiguration.fromConfiguration(RestClientConfiguration.java:105)
... 25 more
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
