pnowojski commented on code in PR #23589:
URL: https://github.com/apache/flink/pull/23589#discussion_r1375899419
##########
flink-streaming-java/src/main/java/org/apache/flink/streaming/api/graph/StreamConfig.java:
##########
@@ -795,6 +795,20 @@ public boolean isGraphContainingLoops() {
return config.getBoolean(GRAPH_CONTAINING_LOOPS, false);
}
+ /**
+ * In general, we don't clear any configuration. However, the {@link
#SERIALIZED_UDF} may be
+ * very large when operator includes some large objects, the
SERIALIZED_UDF is used to create a
+ * StreamOperator and usually only needs to be called once. {@link
#CHAINED_TASK_CONFIG} may be
+ * large as well due to the StreamConfig of all non-head operators in
OperatorChain will be
+ * serialized and stored in CHAINED_TASK_CONFIG. They can be cleared to
reduce the memory after
+ * StreamTask is initialized. If so, TM will have more memory during
running. See FLINK-33315
+ * and FLINK-33317 for more information.
+ */
+ public void clearInitialConfigs() {
+ config.removeKey(SERIALIZED_UDF);
+ config.removeKey(CHAINED_TASK_CONFIG);
Review Comment:
nit: can we mark removal of those keys with some special value instead, so
that we could throw a meaningful error in case of someone trying to read an
already cleared value?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
