Sriram Ganesh created FLINK-31135:
-------------------------------------
Summary: ConfigMap DataSize went > 1 MB and cluster stopped working
Key: FLINK-31135
URL: https://issues.apache.org/jira/browse/FLINK-31135
Project: Flink
Issue Type: Improvement
Affects Versions: kubernetes-operator-1.2.0
Reporter: Sriram Ganesh
I am Flink Operator to manage clusters. Flink version: 1.15.2. Flink jobs
failed with the below error. It seems the config map size went beyond 1 MB
(default size).
Since it is managed by the operator and config maps are not updated with any
manual intervention, I suspect it could be an operator issue.
{code:java}
Caused by: io.fabric8.kubernetes.client.KubernetesClientException: Failure
executing: PUT at:
https://<IP>/api/v1/namespaces/<NS>/configmaps/<job>-config-map. Message:
ConfigMap "<job>-config-map" is invalid: []: Too long: must have at most
1048576 bytes. Received status: Status(apiVersion=v1, code=422,
details=StatusDetails(causes=[StatusCause(field=[], message=Too long: must have
at most 1048576 bytes, reason=FieldValueTooLong, additionalProperties={})],
group=null, kind=ConfigMap, name=<job>-config-map, retryAfterSeconds=null,
uid=null, additionalProperties={}), kind=Status, message=ConfigMap
"<job>-config-map" is invalid: []: Too long: must have at most 1048576 bytes,
metadata=ListMeta(_continue=null, remainingItemCount=null,
resourceVersion=null, selfLink=null, additionalProperties={}), reason=Invalid,
status=Failure, additionalProperties={}).
at
io.fabric8.kubernetes.client.dsl.base.OperationSupport.requestFailure(OperationSupport.java:673)
~[flink-dist-1.15.2.jar:1.15.2]
at
io.fabric8.kubernetes.client.dsl.base.OperationSupport.assertResponseCode(OperationSupport.java:612)
~[flink-dist-1.15.2.jar:1.15.2]
at
io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:560)
~[flink-dist-1.15.2.jar:1.15.2]
at
io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:521)
~[flink-dist-1.15.2.jar:1.15.2]
at
io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleUpdate(OperationSupport.java:347)
~[flink-dist-1.15.2.jar:1.15.2]
at
io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleUpdate(OperationSupport.java:327)
~[flink-dist-1.15.2.jar:1.15.2]
at
io.fabric8.kubernetes.client.dsl.base.BaseOperation.handleUpdate(BaseOperation.java:781)
~[flink-dist-1.15.2.jar:1.15.2]
at
io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation.lambda$replace$1(HasMetadataOperation.java:183)
~[flink-dist-1.15.2.jar:1.15.2]
at
io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation.replace(HasMetadataOperation.java:188)
~[flink-dist-1.15.2.jar:1.15.2]
at
io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation.replace(HasMetadataOperation.java:130)
~[flink-dist-1.15.2.jar:1.15.2]
at
io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation.replace(HasMetadataOperation.java:41)
~[flink-dist-1.15.2.jar:1.15.2]
at
org.apache.flink.kubernetes.kubeclient.Fabric8FlinkKubeClient.lambda$attemptCheckAndUpdateConfigMap$11(Fabric8FlinkKubeClient.java:325)
~[flink-dist-1.15.2.jar:1.15.2]
at
java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1700)
~[?:?]
... 3 more {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
