Fabian-Schneider01 opened a new pull request, #57001:
URL: https://github.com/apache/spark/pull/57001
### What changes were proposed in this pull request?
Migrate the pipeline-dataset guards in
`SparkSqlParser.visitCreatePipelineDataset` off the legacy
`_LEGACY_ERROR_TEMP_0035` ("Operation not allowed: ...") error onto named error
conditions. Six new sub-conditions are added under `UNSUPPORTED_FEATURE`
(SQLSTATE `0A000`), following the existing
`UNSUPPORTED_FEATURE.CREATE_PIPELINE_DATASET_QUERY_EXECUTION` precedent, each
with a `pipelineDatasetType` message parameter (`MATERIALIZED VIEW` or
`STREAMING TABLE`):
- `PIPELINE_DATASET_BUCKETING`
- `PIPELINE_DATASET_COLUMN_CONSTRAINTS`
- `PIPELINE_DATASET_HIVE_SERDE`
- `PIPELINE_DATASET_LOCATION`
- `PIPELINE_DATASET_OPTIONS`
- `PIPELINE_DATASET_STORED_AS`
The guards now throw via new `QueryParsingErrors` helpers instead of
`ParserUtils.operationNotAllowed`.
Scope notes:
- The column-constraints guard is included as well: it lives in the same
method, has the same shape, and previously raised the same legacy condition.
- The JIRA also mentions the MATERIALIZED-VIEW-with-AUTO-CDC guard and the
AUTO CDC streaming-source check; those guards are introduced by the still-open
PR #56419 (SPARK-56249) and can adopt this pattern when they land.
- The "Unable to find query for CREATE ..." guard in the same method is
intentionally left unchanged: a missing `AS` query is a malformed statement
rather than an unsupported feature, so it deserves a differently-shaped
condition and can be handled separately.
### Why are the changes needed?
Per `common/utils/src/main/resources/error/README.md`, user-facing errors
should carry a proper error condition and SQLSTATE rather than a
`_LEGACY_ERROR_TEMP_*` placeholder. These guards are user-reachable from
pipeline SQL definitions, and named conditions make the errors
machine-identifiable via `getCondition()` and consistent with the other
declarative-pipelines error conditions.
### Does this PR introduce _any_ user-facing change?
Yes, but only for the declarative pipelines SQL syntax that has not shipped
in a release yet (master only): the error condition and message for invalid
`CREATE MATERIALIZED VIEW` / `CREATE STREAMING TABLE` pipeline statements
change.
Before:
```
[_LEGACY_ERROR_TEMP_0035] Operation not allowed: Specifying location is not
supported for CREATE STREAMING TABLE statements. The storage location for a
pipeline dataset is managed by the pipeline itself.(line 1, pos 0)
```
After:
```
[UNSUPPORTED_FEATURE.PIPELINE_DATASET_LOCATION] The feature is not
supported: The LOCATION clause in a CREATE STREAMING TABLE statement. The
storage location for a pipeline dataset is managed by the pipeline itself.
SQLSTATE: 0A000(line 1, pos 0)
```
### How was this patch tested?
Updated the existing tests to check the new conditions with `checkError()`:
- `CreatePipelineDatasetAsSelectParserSuiteBase` (exercised for both dataset
types via `CreateMaterializedViewAsSelectParserSuite` and
`CreateStreamingTableAsSelectParserSuite`)
- `SqlPipelineSuite` ("Setting dataset location is disallowed") migrated
from a message-substring assertion to `checkError()`
Ran locally:
```
build/sbt 'core/testOnly *SparkThrowableSuite' \
'sql/testOnly *CreateMaterializedViewAsSelectParserSuite
*CreateStreamingTableAsSelectParserSuite' \
'pipelines/testOnly *SqlPipelineSuite -- -z "location is disallowed"'
```
All passed.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code (claude-fable-5)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
