[PR] Hint Suggestions for invalid pipeline options [beam]

Fri, 05 Sep 2025 11:38:48 -0700 


tarun-google opened a new pull request, #36072:
URL: https://github.com/apache/beam/pull/36072

   Fixes #28402
   
   The base fix #29474 added the warning log for invalid pipeline arguments. 
But as shown in the fix PR image itself the warning shows multiple times. The 
reason being get_all_options is called multiple times. In an yaml pipeline i 
ran its called 4 times, one while parsing yaml and then couple of times while 
calling expansion service and then during the pipeline submission. 
   
   1. made an explicit call during run_pipeline which i expect to be last stage 
of pipeline execution every path share. And added warning logs only during that 
call. 
   2. Added suggestions(if available) using python difflib library
   
   The cost of argument parsing and suggestions is minimal and one time, at the 
beginning of pipeline execution. 
   
   Before Fix:
   <img width="1347" height="171" alt="Screenshot 2025-09-05 at 11 35 39 AM" 
src="https://github.com/user-attachments/assets/d82340c2-0c88-457b-b002-01f77c771d04";
 />
   
   
   After Fix:
   <img width="1238" height="157" alt="Screenshot 2025-09-05 at 11 34 47 AM" 
src="https://github.com/user-attachments/assets/66a9733c-ff8a-4ca0-8d88-67da122b7b19";
 />
   
   
   GitHub Actions Tests Status (on master branch)
   
------------------------------------------------------------------------------------------------
   [![Build python source distribution and 
wheels](https://github.com/apache/beam/actions/workflows/build_wheels.yml/badge.svg?event=schedule&&?branch=master)](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
   [![Python 
tests](https://github.com/apache/beam/actions/workflows/python_tests.yml/badge.svg?event=schedule&&?branch=master)](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
   [![Java 
tests](https://github.com/apache/beam/actions/workflows/java_tests.yml/badge.svg?event=schedule&&?branch=master)](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
   [![Go 
tests](https://github.com/apache/beam/actions/workflows/go_tests.yml/badge.svg?event=schedule&&?branch=master)](https://github.com/apache/beam/actions?query=workflow%3A%22Go+tests%22+branch%3Amaster+event%3Aschedule)
   
   See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more 
information about GitHub Actions CI or the [workflows 
README](https://github.com/apache/beam/blob/master/.github/workflows/README.md) 
to see a list of phrases to trigger workflows.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to