alisha-1000 commented on issue #26034: URL: https://github.com/apache/beam/issues/26034#issuecomment-6968654840
Hi @tvalentyn, I'd like to work on this issue if it's still relevant. I've reviewed #25943 and the `PipelineOptions.from_dictionary(...)` call in `sdk_worker_main.py`. My understanding is that options should be parsed once at job submission, and workers should read the resolved values from the portable pipeline representation instead of parsing them again. Proposed approach: 1. **Reproduce the problem.** Add a test that shows the difference between submission-time and worker-side options, so the fix has a clear target. 2. **Agree on the design.** Post a short proposal here covering where the parsed options are stored, how workers read them, and any impact on runners, before making larger changes. 3. **Implement in small steps.** Send small, reviewable PRs rather than one large change. Before I begin, could you confirm: - Is this still the direction you'd like to take? - Are there runner-specific cases, such as options set or overridden on the worker side, that the design should account for? Thank you, and I look forward to your feedback. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
