[
https://issues.apache.org/jira/browse/FLINK-10412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16641480#comment-16641480
]
ASF GitHub Bot commented on FLINK-10412:
----------------------------------------
tillrohrmann commented on issue #6755: [FLINK-10412] toString field in
AbstractID should be transient to avoid been serialized
URL: https://github.com/apache/flink/pull/6755#issuecomment-427747927
I took a look at the different sizes of a serialized `AbstractID` with and
without the transient keyword:
* old `AbstractID`: 126 bytes
* old `AbstractID` with `toString` being called before serialization: 160
bytes
* transient `AbstractID`: 93 bytes
The issue has been created because ZhuZhu observed scalability issues when
deploying very large jobs. I think he has a point given the different object
sizes, because the `InputChannelDeploymentDescriptor` consists to a good part
of `AbstractIDs`. Given this and that serialization is not a problem, I'm
actually in favour of this improvement.
What we could do @yanghua is to serialize the old `AbstractID` and store the
serialized data as a file in the testing resource folder. Then we could add a
test, that the new AbstractID can be deserialized from this file.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> toString field in AbstractID should be transient to avoid been serialized
> -------------------------------------------------------------------------
>
> Key: FLINK-10412
> URL: https://issues.apache.org/jira/browse/FLINK-10412
> Project: Flink
> Issue Type: Bug
> Components: Distributed Coordination
> Affects Versions: 1.7.0
> Reporter: Zhu Zhu
> Assignee: vinoyang
> Priority: Major
> Labels: deploy,deployment, pull-request-available, serialization
>
> The toString field in AbstractID will be serialized currently, which makes
> RPC messages body like InputChannelDeploymentDescriptor and PartitionInfo
> larger (50%+).
> It adds more pressure to JM memory especially in large scale job scheduling
> (10000x10000 ALL-to-ALL connection).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
