Thanks for the proposal. As I read it, both DSIP-110 and a generic-JDBC approach are about extending JDBC datasources, not arbitrary connection types.
If the goal is supporting a new database without modifying DolphinScheduler code, I would prefer a built-in GENERIC_JDBC type: the user supplies jdbcUrl, driverClass, and credentials, and drops the driver jar on the classpath. That matches how NiFi / Spark / Kafka Connect JDBC work, and it can be done with a small additive change (one DbType, one processor, one form). We already have JdbcDriverConnectionProvider that connects with an arbitrary driver and URL; what is missing is exposing that as a user-facing datasource type. DSIP-110’s string type identity and external plugin registry are a larger refactor (SPI, DAO tinyint → varchar, UI type lists). They are valuable if we want named third-party types in the dropdown, but they are not required for “any JDBC database can run SQL/Procedure.” I would keep that as a follow-up, and ship GENERIC_JDBC first. The two can coexist. On 2026/08/28 08:40:55 eye-gu wrote: > Hi community, > > I'd like to propose DSIP-110: Support Custom Datasource Types via External > Plugins. > > Motivation > > DolphinScheduler's datasource layer is pluginized in name but not in fact. > The plugin contract is hard-wired to the compiled-in DbType enum — > DataSourceProcessor#getDbType() returns a compile-time enum, the > t_ds_datasource.type column stores the enum's integer code, and both > frontend surfaces (the datasource creation dialog and the task-node > datasource selector) hardcode the type lists. As a result, an externally > compiled plugin jar can only override a built-in type; it can never > introduce a new one. Users whose databases are not among the 28 built-in > types have no supported extension path short of forking the project. > > Goal > > A deployment administrator can install an externally compiled datasource > plugin package — without modifying or recompiling any DolphinScheduler code > — and the new type becomes usable end-to-end: visible in the UI, full > lifecycle (create / connectivity test / authorize / metadata browsing), and > executable in SQL and Procedure tasks. Installation is dropping the plugin > jar into the plugins directory and restarting the servers. > > Design summary (full detail in the GitHub issue) > > String-based type identity in the SPI contract (DataSourceProcessor / param > DTOs / plugin manager registry), with fail-fast duplicate-name detection; > DbType degrades to named constants for built-ins. > Type registry + discovery API: GET /datasources/types returns installed > types with metadata (label, default port, capability flags) — the single > contract for both frontend surfaces. > Persistence: t_ds_datasource.type converted from tinyint (enum code) to > varchar (type name), with upgrade scripts for MySQL/PostgreSQL. > api-server adaptation: string-based routing, unknown-type validation, > capability-driven dialect defaults. > Task execution path: master/worker contexts and SQL/Procedure/DataX/Sqoop > call sites switch from DbType.valueOf to string resolution. > Frontend: dynamic type lists from the API; custom types render a generic > JDBC fallback form (host/port/user/password/database/other). > Packaging & driver conventions for external plugin jars (single shaded jar > bundling the JDBC driver, install on all api/master/worker nodes). > Alignment with the project security model: plugin installation is an > administrator/deployment action, inside the existing trust boundary. > A reference H2 datasource plugin (DbType contains H2 but no plugin module > exists today) is planned as the living example and the CI test vehicle for > the whole path. > > Open design decisions I'd especially like feedback on > > D1 — Fate of DbType: recommended to keep it as a deprecated constants > holder, with all runtime paths using string type names (mirrors how task > plugins use String taskType). > D2 — Storage: recommended in-place type column conversion tinyint → > varchar(64) with data migration, over a side mapping table. > D3 — UI for custom types: recommended a generic JDBC fallback form for v1; > plugin-declared form schema deferred to a follow-up DSIP. > D4 — v1 task coverage: SQL + Procedure only; DataX/Sqoop stay built-in-only > via the existing whitelist mechanism. > D5 — Exact response shape of GET /datasources/types. > I already added a GitHub Issue for my proposal, which you could see in: > https://github.com/apache/dolphinscheduler/issues/18600 > > Looking forward any feedback for this thread. > > Best regards, eye-gu >
