Radosław Stankiewicz created BEAM-9188:
------------------------------------------
Summary: Improving speed of splitting for Custom Sources
Key: BEAM-9188
URL: https://issues.apache.org/jira/browse/BEAM-9188
Project: Beam
Issue Type: Improvement
Components: runner-dataflow
Reporter: Radosław Stankiewicz
Assignee: Radosław Stankiewicz
At this moment Custom Source in being split and serialized in sequence. If
there are many splits, it takes time to process all splits.
Example: it takes 2s to calculate size and serialize CassandraSource due to
connection setup and teardown. With 100+ splits, it's a lot of time spent in 1
worker.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
