[
https://issues.apache.org/jira/browse/BEAM-9188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17121710#comment-17121710
]
Kenneth Knowles commented on BEAM-9188:
---------------------------------------
This issue is assigned but has not received an update in 30 days so it has been
labeled "stale-assigned". If you are still working on the issue, please give an
update and remove the label. If you are no longer working on the issue, please
unassign so someone else may work on it. In 7 days the issue will be
automatically unassigned.
> Improving speed of splitting for Custom Sources
> -----------------------------------------------
>
> Key: BEAM-9188
> URL: https://issues.apache.org/jira/browse/BEAM-9188
> Project: Beam
> Issue Type: Improvement
> Components: runner-dataflow
> Reporter: Radosław Stankiewicz
> Assignee: Radosław Stankiewicz
> Priority: P3
> Labels: stale-assigned
> Time Spent: 4h 40m
> Remaining Estimate: 0h
>
> At this moment Custom Source in being split and serialized in sequence. If
> there are many splits, it takes time to process all splits.
>
> Example: it takes 2s to calculate size and serialize CassandraSource due to
> connection setup and teardown. With 100+ splits, it's a lot of time spent in
> 1 worker.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
