Sudhanva Huruli created SPARK-62020:
---------------------------------------
Summary: spark-pipelines run names a non-existent --full_refresh
option in the CONFLICTING_PIPELINE_REFRESH_OPTIONS error
Key: SPARK-62020
URL: https://issues.apache.org/jira/browse/SPARK-62020
Project: Spark
Issue Type: Bug
Components: Declarative Pipelines
Affects Versions: 4.3.0, 4.2.0, 4.1.0
Reporter: Sudhanva Huruli
When spark-pipelines run is given both --full-refresh-all and --full-refresh,
the CONFLICTING_PIPELINE_REFRESH_OPTIONS error names the conflicting option as
--full_refresh (underscore). The CLI only defines --full-refresh; passing
--full_refresh fails with 'unrecognized arguments'. The message should name
--full-refresh, as the sibling check for --refresh does. This is a one-word fix
in python/pyspark/pipelines/cli.py and its unit test; it may qualify as a MINOR
change under the contributing guide.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]