Sandy Ryza created SPARK-52511:
----------------------------------
Summary: Support "dry-run" in the spark-pipelines CLI
Key: SPARK-52511
URL: https://issues.apache.org/jira/browse/SPARK-52511
Project: Spark
Issue Type: Improvement
Components: Declarative Pipelines
Affects Versions: 4.1.0
Reporter: Sandy Ryza
We should add support for a `dry-run` command in the `spark-pipelines` CLI:
```
spark-pipelines dry-run
```
This should perform graph resolution and analysis, but no metadata
materialization or graph execution. I.e. it should invoke
`DataflowGraph.resolve().validate()`, and stream any errors back to the client.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
