[
https://issues.apache.org/jira/browse/BEAM-5859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16664067#comment-16664067
]
Maximilian Michels commented on BEAM-5859:
------------------------------------------
Something related which we could also fix: The names of the operators in
metrics are truncated because Metric Groups only support names up to 80 chars.
We should at least include some sort of ID to make it easier to identify such
operators.
{noformat}
[CHAIN Filter (UnionFixFilter) -> Map (Key Extractor) -> GroupCombine
(GroupCombine at GroupCombine: PAssert$0/GroupGlobally/GroupDummyAndContents)
-> Map (Key Extractor) (8/8)] WARN org.apache.flink.metrics.MetricGroup - The
operator name exceeded the 80 characters length limit and was truncated.
{noformat}
> Improve Traceability of Pipeline translation
> --------------------------------------------
>
> Key: BEAM-5859
> URL: https://issues.apache.org/jira/browse/BEAM-5859
> Project: Beam
> Issue Type: Improvement
> Components: runner-flink
> Reporter: Maximilian Michels
> Priority: Major
> Labels: portability, portability-flink
> Attachments: tfx.png, wordcount.png
>
>
> Users often ask how they can reason about the pipeline translation. The Flink
> UI display a confusingly large graph without any trace of the original Beam
> pipeline:
> WordCount:
> !wordcount.png!
> TFX:
> !tfx.png!
> Some aspects which make understanding these graphs hard:
> * Users don't know how the Runner maps Beam to Flink concepts
> * The UI is awfully slow / hangs when the pipeline is reasonable complex
> * The operator names seem to use {{transform.getUniqueName()}} which doesn't
> generate readable name
> * So called Chaining combines operators into a single operator which makes
> understanding which Beam concept belongs to which Flink concept even harder
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
