[
https://issues.apache.org/jira/browse/FLINK-11256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16738228#comment-16738228
]
Aljoscha Krettek commented on FLINK-11256:
------------------------------------------
Please don't try to be too complicated on this. We will have to rework the
basic Flink DAG and operator interfaces in the future and any work that we now
put into the StreamGraph abstractions will be legacy code by then.
If it's very small fixes it's probably ok but I wouldn't do anything bigger.
> Referencing StreamNode objects directly in StreamEdge causes the sizes of
> JobGraph and TDD to become unnecessarily large
> ------------------------------------------------------------------------------------------------------------------------
>
> Key: FLINK-11256
> URL: https://issues.apache.org/jira/browse/FLINK-11256
> Project: Flink
> Issue Type: Bug
> Components: Streaming
> Affects Versions: 1.7.0, 1.7.1
> Reporter: Haibo Suen
> Assignee: Haibo Suen
> Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When a job graph is generated from StreamGraph, StreamEdge(s) on the stream
> graph are serialized to StreamConfig and stored into the job graph. After
> that, the serialized bytes will be included in the TDD and distributed to TM.
> Because StreamEdge directly reference to StreamNode objects includingÂ
> sourceVertex and targetVertex, these objects are also written transitively on
> serializing StreamEdge. But these StreamNode objects are not needed in JM and
> Task. For a large size topology, this will causes JobGraph/TDD to become much
> larger than that actually need, and more likely to occur rpc timeout when
> transmitted.
> In StreamEdge, only the ID of StreamNode should be stored to avoid this
> situation.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
