Chamikara Jayalath created BEAM-564:
---------------------------------------
Summary: Update source framework so that remaining and consumed
number of split points can be reported
Key: BEAM-564
URL: https://issues.apache.org/jira/browse/BEAM-564
Project: Beam
Issue Type: Improvement
Components: sdk-py
Reporter: Chamikara Jayalath
Assignee: Chamikara Jayalath
We have to update Python SDK source framework so that sources can report
consumed and remaining number of split points. Runners can use this information
to determine how many times a given source can be split into and parallelize
reading accordingly.
Corresponding API for JAVA SDK is here:
https://github.com/apache/incubator-beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/io/BoundedSource.java#L258
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
