macdoor opened a new issue, #18019: URL: https://github.com/apache/dolphinscheduler/issues/18019
### Search before asking - [x] I had searched in the [issues](https://github.com/apache/dolphinscheduler/issues?q=is%3Aissue) and found no similar feature requirement. ### Description ## Purpose Using Resource Center SQL files makes it much easier to manage SQL scripts: - SQL scripts can be version-controlled and reviewed independently (e.g. via Git) - Multiple workflows can share the same SQL file, avoiding duplication - Updating SQL logic only requires updating the file, without editing every SQL task node Extend the **SQL** task type to support: 1. Executing SQL from a **Resource Center file** (in addition to inline script) 2. Applying **parameter placeholders** (`${...}`, `$[...]`) consistently for both inline SQL and file-based SQL This is similar to what other task types already support (e.g. HiveCli, Shell, Spark, Python, Flink/FlinkSqlGateway). ## Background In current `dev`: - `SqlParameters` only has a single `sql` field - The SQL task UI only supports inline SQL text - Users who keep large SQL scripts in the **Resource Center** cannot reference them directly from the SQL task - Parameter placeholder replacement is partially handled when binding parameters, but there is no way to: - load SQL content from a file first - then apply placeholders on that content For large / shared SQL scripts, putting them into the Resource Center and referencing them from tasks is a common pattern. Tasks like **HiveCli**, **Shell**, **Spark**, **Python** already support this mode. ## Proposal ### Backend 1. **Extend `SqlParameters`** (`dolphinscheduler-task-api`): - Add fields: - `sqlSource`: `"SCRIPT"` (default) or `"FILE"` - `sqlResource`: resource path in Resource Center - Keep existing `sql` for inline SQL (backward compatible). - Update: - `checkParameters()` to accept either `sql` OR `sqlResource` - `getResourceFilesList()` to return a `ResourceInfo` when `sqlResource` is set, so Worker downloads the file 2. **Extend `SqlTask`** (`dolphinscheduler-task-sql`): - Before splitting SQL, ensure SQL content: private void ensureSqlContent() { if (StringUtils.isNotEmpty(sqlParameters.getSql())) { return; } if (StringUtils.isEmpty(sqlParameters.getSqlResource())) { return; } // Use TaskExecutionContext.getResourceContext() // to resolve local path and read SQL text into sqlParameters.sql } ### Are you willing to submit a PR? - [x] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
