macdoor opened a new pull request, #18020:
URL: https://github.com/apache/dolphinscheduler/pull/18020
<!--Thanks very much for contributing to Apache DolphinScheduler, we are
happy that you want to help us improve DolphinScheduler! -->
## Was this PR generated or assisted by AI?
YES.
The implementation and refactoring were done by me, with AI assistance
mainly for:
- Designing the overall approach (backend + UI + tests) based on existing
patterns (e.g. HiveCli, FlinkSqlGateway, PR #17987 review).
- Drafting some code snippets and PR/issue descriptions, which I reviewed
and adapted to fit DolphinScheduler’s style.
## Purpose of the pull request
[Improvement][task-sql] Support executing SQL from **Resource Center files**
as well as inline scripts, and apply **parameter placeholders** (`${...}`,
`$[...]`) consistently for SQL tasks.
This makes it easier to manage SQL scripts (version control, reuse across
workflows) and aligns `task-sql` with other resource-aware task types.
## Brief change log
### Backend
- **SqlParameters** (task-api)
- Add fields:
- `sqlSource`: `"SCRIPT"` (default) or `"FILE"`
- `sqlResource`: resource path in Resource Center
- Keep existing `sql` for inline SQL (backward compatible).
- Update:
- `checkParameters()` to accept either `sql` **or** `sqlResource`
- `getResourceFilesList()` to return a `ResourceInfo` when `sqlResource`
is set, so Worker downloads the file.
- **SqlTask** (task-sql)
- Add `ensureSqlContent()`:
- If `sql` is empty and `sqlResource` is set:
- Use `TaskExecutionContext.getResourceContext()` to resolve local
file path.
- Read SQL text from the downloaded resource file and set it into
`SqlParameters.sql`.
- Keep the existing split/bind/execute logic unchanged.
- Improve title placeholder replacement:
- Build a placeholder map even when `prepareParamsMap` is `null` (fall
back to `Collections.emptyMap()`).
- Apply `ParameterUtils.convertParameterPlaceholders` for `title` so
time placeholders (e.g. `$[yyyyMMdd]`) still work even without custom params.
- **Tests**
- Add `SqlTaskTest`:
- `testSqlLoadedFromResourceFileWhenSqlIsEmpty`:
- Creates a temp SQL file.
- Configures `sqlResource` to point to it while leaving `sql` empty.
- Uses `ResourceContext` on `TaskExecutionContext`.
- Verifies `SqlTask` loads the content into `SqlParameters.sql`.
### Frontend (UI)
- **Task form for SQL task**
- `use-sql.ts`:
- Extend `model` with:
- `sqlSource`: `'SCRIPT' | 'FILE'` (default `'SCRIPT'`)
- `sqlResource`: selected resource path.
- `fields/use-sql.ts`:
- Add a **SQL Source** radio group:
- `SCRIPT`: show the existing SQL editor (`field: 'sql'`).
- `FILE`: hide the editor and show a Resource Center file selector
bound to `sqlResource` (single-select).
- Keep original validation for `sql` when `sqlSource === 'SCRIPT'`.
- **Localization**
- Add i18n keys in `en_US` and `zh_CN`:
- `sql_source`, `sql_source_script`, `sql_source_file`,
`sql_resource_file`.
## Verify this pull request
This change added tests and can be verified as follows:
- **Unit tests**
- `SqlParametersTest`:
- Verifies `getResourceFilesList()` returns a non-empty list when
`sqlResource` is set.
- `SqlTaskTest`:
- `testSqlLoadedFromResourceFileWhenSqlIsEmpty`: verifies `SqlTask`
loads SQL content from a Resource Center file via `ResourceContext` when `sql`
is empty and `sqlResource` is set.
- **Manual verification**
- Build backend and UI on `dev` with this branch.
- In UI:
- Create a SQL task with:
- `SQL Source = SCRIPT`, edit inline SQL as before → task behavior
unchanged.
- `SQL Source = FILE`, select a `.sql` file from Resource Center →
task executes the SQL from that file.
- Use `${...}` and `$[...]` placeholders in both modes to confirm they
are resolved as expected.
## Pull Request Notice
[Pull Request
Notice](https://github.com/apache/dolphinscheduler/blob/dev/docs/docs/en/contribute/join/pull-request.md)
This PR does **not** introduce incompatible changes to existing SQL tasks:
- Existing tasks without `sqlSource/sqlResource` continue to run as before
(treated as `SCRIPT` + `sql`).
- New fields are backward compatible and optional.
closes #18019
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
