[
https://issues.apache.org/jira/browse/BEAM-13993?focusedWorklogId=732086&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-732086
]
ASF GitHub Bot logged work on BEAM-13993:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 24/Feb/22 02:36
Start Date: 24/Feb/22 02:36
Worklog Time Spent: 10m
Work Description: tvalentyn edited a comment on pull request #16920:
URL: https://github.com/apache/beam/pull/16920#issuecomment-1049436020
This looks to be one of the lesser known features of Dataflow runner V1, and
is marked experimental. I wasn't aware of it. @rahuliyer95 would you be
interested in making it a non-experimental first-class citizen in Beam that
would work for all portable runners? I would recommend an advertisement on dev
describing your use-cases for this, settle on the expected plugin format (class
vs module to remove the ambiguity or allow both if that makes sense sometimes -
if so you could add rationale), adding unit tests and proper documentation,
removing the 'experimental' status from the doc. You can start the conversation
on dev@ with a proposal to remove the 'experimental' bit and reference your PR.
This is what the import code looks like in the Dataflow Python worker:
```
for beam_plugin in sdk_pipeline_options.get('beam_plugins', []):
try:
module_name, class_name = beam_plugin.rsplit('.', 1)
_ = getattr(importlib.import_module(module_name), class_name)
logging.info('Successfully imported beam plugin %s', beam_plugin)
except ImportError:
logging.warning('Failed to import beam plugin %s', beam_plugin,
exc_info=True)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 732086)
Time Spent: 40m (was: 0.5h)
> Python portable SDK harness does not import beam_plugins at startup
> -------------------------------------------------------------------
>
> Key: BEAM-13993
> URL: https://issues.apache.org/jira/browse/BEAM-13993
> Project: Beam
> Issue Type: Improvement
> Components: sdk-py-harness
> Reporter: Steve Niemitz
> Assignee: Steve Niemitz
> Priority: P2
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Currently on the portable runner `–beam_plugins` is ignored. They should
> instead be imported on SDK harness startup.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
