[
https://issues.apache.org/jira/browse/BEAM-7443?focusedWorklogId=251087&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-251087
]
ASF GitHub Bot logged work on BEAM-7443:
----------------------------------------
Author: ASF GitHub Bot
Created on: 30/May/19 16:35
Start Date: 30/May/19 16:35
Worklog Time Spent: 10m
Work Description: lukecwik commented on pull request #8641: [BEAM-7443]
Create a BoundedSource -> SDF wrapper in Python SDK
URL: https://github.com/apache/beam/pull/8641#discussion_r289053240
##########
File path: sdks/python/apache_beam/io/iobase.py
##########
@@ -910,6 +913,67 @@ def from_runner_api_parameter(parameter, context):
Read.from_runner_api_parameter)
+class SDFBoundedSourceWrapper(ptransform.PTransform):
+ """A ``PTransform`` that uses SDF to read from a ``BoundedSource``.
+ NOTE: This transform can only be used with beam_fn_api enabled.
+ """
+
+ def __init__(self, source):
+ if not isinstance(source, BoundedSource):
+ raise RuntimeError('SDFBoundedSourceWrapper can only wrap BoundedSource')
+ super(SDFBoundedSourceWrapper, self).__init__()
+ self.source = source
+
+ def _get_desired_chunk_size(self):
+ total_size = self.source.estimate_size()
Review comment:
Were you planning to refactor the code to only have one copy of
`_get_desired_chunk_size`?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 251087)
Time Spent: 1h 20m (was: 1h 10m)
> BoundedSource->SDF needs a wrapper in Python SDK
> -------------------------------------------------
>
> Key: BEAM-7443
> URL: https://issues.apache.org/jira/browse/BEAM-7443
> Project: Beam
> Issue Type: New Feature
> Components: sdk-py-core
> Reporter: Boyuan Zhang
> Assignee: Boyuan Zhang
> Priority: Major
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
