chenxu8989 opened a new issue, #18713: URL: https://github.com/apache/dolphinscheduler/issues/18713
### Search before asking - [x] I had searched in the [issues](https://github.com/apache/dolphinscheduler/issues?q=is%3Aissue) and found no similar issues. ### What happened In a sub-workflow that compares row counts of two tables, an SQL task (`统计源库行数`) declares an OUT parameter `SRC_CNT` (type `LONG`) on `SELECT COUNT(*) AS SRC_CNT FROM <table>`; a downstream non-query task (`写入比对结果表`) declares two IN parameters `SRC_CNT` and `TGT_CNT` (type `LONG`) that interpolate the upstream values into an `INSERT` statement. When the actual row count happens to be a "round" BigDecimal that has trailing zeros in its integer part (e.g. `11423000`), the downstream task fails with: ``` Caused by: java.lang.NumberFormatException: For input string: "1.1423E7" at java.lang.Long.parseLong(Long.java:589) at java.lang.Long.parseLong(Long.java:631) at org.apache.dolphinscheduler.plugin.task.api.utils.ParameterUtils.setInParameter(ParameterUtils.java:139) at org.apache.dolphinscheduler.plugin.task.sql.SqlTask.prepareStatementAndBind(SqlTask.java:361) ... 11 common frames omitted Caused by: org.apache.dolphinscheduler.plugin.task.api.TaskException: SQL task prepareStatementAndBind error at org.apache.dolphinscheduler.plugin.task.sql.SqlTask.prepareStatementAndBind(SqlTask.java:368) at org.apache.dolphinscheduler.plugin.task.sql.SqlTask.executeUpdate(SqlTask.java:314) ... Caused by: org.apache.dolphinscheduler.plugin.task.api.TaskException: Execute sql task failed at org.apache.dolphinscheduler.plugin.task.sql.SqlTask.handle(SqlTask.java:158) ... ``` The non-query INSERT task never runs; the sub-workflow and the parent workflow fail. ### What you expected to happen A `COUNT(*)` value that fits in a `LONG` should round-trip from the source task's `varPool` to the downstream task's PreparedStatement parameter in plain integer form (`"11423000"`), not as scientific-notation JSON (`"1.1423E7"`). The OUT → IN parameter contract for `type=LONG` should be self-consistent within the SQL task plugin, regardless of the integer BigDecimal's "roundness". The two facts that need to align are: 1. The OUT value placed into `taskInstance.varPool` (consumed by downstream tasks) 2. The downstream task's `setInParameter(type=LONG, value=...)` call Today they are inconsistent: (1) is produced via `ObjectMapper.valueToTree(BigDecimal)` and may emit `1.1423E+7` because Jackson's `BigDecimalSerializer` calls `stripTrailingZeros().toString()`; (2) calls `Long.parseLong(value)` and rejects that string. ### How to reproduce ### Minimal scenario Two SQL tasks in the same workflow, the second one declaring IN parameters with `type=LONG`. **Task A** (query, OUT `LONG`): ```sql SELECT COUNT(*) AS SRC_CNT FROM your_table; ``` Declare one OUT parameter: | prop | direct | type | value | |------|--------|------|-------| | `SRC_CNT` | OUT | LONG | | **Task B** (non-query, IN `LONG`), downstream of Task A: ```sql INSERT INTO some_log_table (id, src_cnt, ts) VALUES (SYS_GUID(), ${SRC_CNT}, SYSDATE); ``` Declare one IN parameter: | prop | direct | type | value | |------|--------|------|-------| | `SRC_CNT` | IN | LONG | | Trigger the workflow with the source table containing a row count that **is divisible by 10** (e.g. exactly 11423000 rows). The downstream task fails with the stack trace above. Trigger the same workflow with a row count **not divisible by 10** (e.g. 11423001 or 11265629) — the task succeeds and inserts `11265629` / `11423001` correctly. ### Why the trigger condition is "trailing-zero count" The decision is made in `JSONUtils.toJsonNode(Object)` (`dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/utils/JSONUtils.java`), which is `objectMapper.valueToTree(obj)`. Jackson's default `BigDecimalSerializer` serializes a BigDecimal as `stripTrailingZeros().toString()`. When the BigDecimal has trailing zeros in its integer part, `stripTrailingZeros` reduces its scale, which makes `toString()` emit scientific notation. Verified by directly running the same Jackson pipeline (no DS changes): ``` 11423000 -> tree=1.1423E+7 final=1.1423E7 parseLong=FAIL 11423001 -> tree=11423001 final=11423001 parseLong=OK 11423010 -> tree=1.142301E+7 final=1.142301E7 parseLong=FAIL 11265629 -> tree=11265629 final=11265629 parseLong=OK 10000000 -> tree=1E+7 final=1.0E7 parseLong=FAIL ``` So the failure is not tied to the row count being "large" — it is tied to the count being divisible by 10. Other "round" forms (decimal placeholders, e.g. `11423000.0` with `scale=1`) serialize fine; the issue is specifically `stripTrailingZeros` on integers ending in zero. ### Source-of-truth code paths - `dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/utils/JSONUtils.java`, the shared `ObjectMapper` does not enable `WRITE_BIGDECIMAL_AS_PLAIN` (and no `BigDecimalSerializer` is registered), so default Jackson behavior applies. - `dolphinscheduler-task-plugin/dolphinscheduler-task-sql/src/main/java/org/apache/dolphinscheduler/plugin/task/sql/SqlTask.java`, `resultProcess(...)` calls `JSONUtils.toJsonNode(resultSet.getObject(i))` to build the OUT parameter JSON. - `dolphinscheduler-task-plugin/dolphinscheduler-task-api/src/main/java/org/apache/dolphinscheduler/plugin/task/api/parameters/SqlParameters.java`, `dealOutParam(...)` reads the JSON via `getListMapByString(...)` → `JSONUtils.toMap(jsonNode.toString())` and assigns `info.setValue(String.valueOf(firstRow.get(info.getProp())))` — this is the value that lands in `taskInstance.varPool`. - `dolphinscheduler-task-plugin/dolphinscheduler-task-api/src/main/java/org/apache/dolphinscheduler/plugin/task/api/utils/ParameterUtils.java`, `setInParameter(...)`, for `DataType.LONG` does `stmt.setLong(index, Long.parseLong(value))` and rejects the scientific-notation string. I have checked that the same code paths exist unchanged on the current `origin/dev` branch (latest commit `c366e26217` on 2026-10-15). The problem is present in 3.3.2 and still present on `dev`. ### Workaround (what I am doing until the fix lands) Change the downstream IN parameters from `LONG` to `VARCHAR`. The target columns are `NUMBER(12)`, and the Oracle JDBC driver performs an implicit `to_number()` on the bind, which accepts the `1.1423E7` form just fine. This is a configuration change in DolphinScheduler's web UI; no SQL change is required. ### Anything else - Frequency: every time the sub-workflow runs against a table whose row count is divisible by 10 (e.g. growing from 11423000 to 11423009, all would fail; growing from 11423010 to 11423019 would succeed). For a long-running comparison workflow the failure window is therefore not "rare" — it covers roughly one in every 10 values of the count. - Severity for us: medium. The comparison sub-workflow is the last step of a 30-minute evaluation workflow, and a failure here also fails the parent workflow, so this is a hard stop until a manual re-run happens to land on a "non-round" count. - Related behaviors I have ruled out: - The SHELL task in the same sub-workflow that compares `SRC_CNT` and `TGT_CNT` with `[ "${SRC_CNT}" -eq "${TGT_CNT}" ]` is **not** affected, because the SHELL task plugin does parameter substitution by literal string, not via `setInParameter` → `Long.parseLong`. (You can keep the SHELL IN parameter as `LONG` and it still works; the failure is strictly in the SQL INSERT path.) - `Long` JSON serialization is not the cause: the OUT value is wrapped by `getListMapByString` → `JSONUtils.toMap(jsonNode.toString())` which deserializes numbers into `String` map values, so the "scientific notation" is already a property of the **string** stored in `varPool` before any consumer reads it. ### Environment - Apache DolphinScheduler 3.3.2 (also confirmed present on `origin/dev` at the time of reporting). - Datasource: Oracle (Oracle JDBC driver returns `BigDecimal` for `COUNT(*)`); other JDBC sources that also return `BigDecimal` (most relational DBs) are likely affected by the same code path. - Workflow: 5-task sub-workflow (DELETE → 2x COUNT query → 1x INSERT → 1x SHELL), as detailed above. - Worker log excerpt (already included at the top of this report). ## Chinese summary (for the Chinese-speaking maintainers) 简单说：SQL 任务的 OUT 参数经过 `JSONUtils.toJsonNode(BigDecimal)` 序列化时，Jackson 默认会调 `stripTrailingZeros().toString()`。当 `COUNT(*)` 返回值的末位是 0（如 11423000），会被序列化成 `1.1423E+7`；下游 SQL 任务的 IN 参数 `type=LONG` 走 `Long.parseLong`，解析不了这个字符串，任务失败。 修复点最小的是在 `JSONUtils` 的共享 `ObjectMapper` 上加 `SerializationFeature.WRITE_BIGDECIMAL_AS_PLAIN = true`，让 BigDecimal 始终输出 `toPlainString()` 形式（如 `11423000`），与下游 `parseLong` 配套。 版本：3.3.2 仍存在；最新 `origin/dev`（2026-10-15 commit `c366e26217`）也已确认仍未修复。 ### Version 3.3.2 ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
