[
https://issues.apache.org/jira/browse/BEAM-6942?focusedWorklogId=226404&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-226404
]
ASF GitHub Bot logged work on BEAM-6942:
----------------------------------------
Author: ASF GitHub Bot
Created on: 11/Apr/19 23:19
Start Date: 11/Apr/19 23:19
Worklog Time Spent: 10m
Work Description: aaltay commented on pull request #8225: [BEAM-6942]
Make modifications to pipeline options to be visible to all views.
URL: https://github.com/apache/beam/pull/8225#discussion_r274708267
##########
File path: sdks/python/apache_beam/options/pipeline_options.py
##########
@@ -246,7 +276,40 @@ def display_data(self):
return self.get_all_options(True)
def view_as(self, cls):
+ """Returns a view of current object as provided PipelineOption subclass.
+
+ Example Usage::
+
+ options = PipelineOptions(['--runner', 'Direct', '--streaming'])
+ standard_options = options.view_as(StandardOptions)
+ if standard_options.streaming:
+ # ... start a streaming job ...
+
+ Note that options objects may have multiple views, and modifications
+ of values in any view-object will apply to current object and other
+ view-objects.
+
+ Args:
+ cls: PipelineOptions class or any of its subclasses.
+
+ Returns:
+ An instance of cls that is intitialized using options contained in
current
+ object.
+
+ """
view = cls(self._flags)
+ for option_name in view._visible_option_list():
+ # Initialize values of keys defined by a cls.
+ #
+ # Note that we do initialization only once per key to make sure that
+ # values in _all_options dict are not-recreated with each new view.
+ # This is important to make sure that values of multi-options keys are
+ # backed by the same list across multiple views, and that any overrides
of
+ # pipeline options already stored in _all_options are preserved.
+ if option_name not in self._all_options:
+ self._all_options[option_name] = getattr(view._visible_options,
+ option_name)
+ # Note that views will still store _all_options of the source object.
Review comment:
Would not this result in views that expose attributes that are not meant to
be part of that view?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 226404)
Time Spent: 7h (was: 6h 50m)
> Pipeline options to experiment propagation is not working in Dataflow runner.
> -----------------------------------------------------------------------------
>
> Key: BEAM-6942
> URL: https://issues.apache.org/jira/browse/BEAM-6942
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
> Reporter: Valentyn Tymofieiev
> Assignee: Valentyn Tymofieiev
> Priority: Major
> Time Spent: 7h
> Remaining Estimate: 0h
>
> Relevant code:
> [https://github.com/apache/beam/blob/master/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py#L356-L388]
> 3 experiments/options are affected. We need to fix it in 2.12.0
> cc: [~altay] [~apilloud]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
