[
https://issues.apache.org/jira/browse/BEAM-10275?focusedWorklogId=452684&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-452684
]
ASF GitHub Bot logged work on BEAM-10275:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 29/Jun/20 23:19
Start Date: 29/Jun/20 23:19
Worklog Time Spent: 10m
Work Description: TheNeuralBit commented on pull request #12032:
URL: https://github.com/apache/beam/pull/12032#issuecomment-651418048
@angoenka agreed it would be best if we could just fix BEAM-10274. But I
think that's a tricky problem that requires some design work.
Currently this is a mismatch between Dataflow's production worker and the
SDK harness - it means there are pipelines that can run without issue on the
production worker, but fail when using the fn API - so I think we should get it
fixed quickly.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 452684)
Time Spent: 50m (was: 40m)
> sdk_worker_main.py eagerly parses pipeline options
> --------------------------------------------------
>
> Key: BEAM-10275
> URL: https://issues.apache.org/jira/browse/BEAM-10275
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-harness
> Reporter: Brian Hulette
> Assignee: Brian Hulette
> Priority: P2
> Time Spent: 50m
> Remaining Estimate: 0h
>
> sdk_worker_main.py eagerly parses pipeline options beause of the call to
> get_all_options here:
> https://github.com/apache/beam/blob/61b665640d6c0f91751bba59782c0ac6aceacba6/sdks/python/apache_beam/runners/worker/sdk_worker_main.py#L138
> This can cause the worker to crash if any option that can't be read at
> execution is time is used, even if we don't need to access it at execution
> time (e.g. json.loads arguments, described in BEAM-10274)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
