zhengxiang378928908-code opened a new pull request, #11690:
URL: https://github.com/apache/seatunnel/pull/11690
### Purpose of this pull request
Fixes #10924.
When Doris auto-creates a table, a decimal column with a scale above 9
produces DDL that Doris 1.x rejects:
```
CatalogException: ErrorCode:[API-03], ErrorDescription:[Catalog initialize
failed]
- create table statement execute failed
Scale of decimal must between 0 and 9. Scale was set to: 10
```
**Root cause.** `AbstractDorisTypeConverter#sampleReconvert` validates the
scale only against the precision:
```java
if (scale < 0) {
scale = 0;
} else if (scale > precision) {
scale = precision;
}
```
That is the DECIMALV3 rule — scale must not exceed precision. Doris 1.x
decimals are DecimalV2, which is fixed at `DECIMAL(27, 9)` and caps the scale
at **9 regardless of the precision**. A MySQL `DECIMAL(20,10)` therefore passes
the check (10 ≤ 20) and reaches Doris unchanged.
The same gap exists in the `precision <= 0` fallback, which assigns
`MAX_SCALE` (10) — already one over the Doris 1.x limit.
**Fix.** Add an overridable `getMaxDecimalScale()` and cap the emitted scale
with it:
- the default returns `MAX_PRECISION`, so DECIMALV3 imposes no limit beyond
the existing `scale <= precision` rule and **`DorisTypeConverterV2` behaviour
is unchanged**
- `DorisTypeConverterV1` overrides it with 9
The cap is applied before the existing precision-based clamp, and an
out-of-range scale is logged with the same warning style used by the
neighbouring branches.
I deliberately scoped this to the scale. `DorisTypeConverterV1` also
inherits `MAX_PRECISION = 38` even though DecimalV2 tops out at 27, and
`sampleReconvert` hardcodes the `DECIMALV3` type name on the Doris 1.x path
too. Both look questionable, but neither is needed to fix the reported failure
and I could not verify either against a real Doris 1.x instance, so I left them
alone rather than guess. Happy to open follow-up issues if maintainers agree
they are problems.
### Does this PR introduce _any_ user-facing change?
Yes, for Doris 1.x only: a decimal column whose scale exceeds 9 is now
emitted as `DECIMALV3(p,9)` instead of producing DDL that fails. Values are
truncated to 9 decimal places rather than the job failing at table creation,
which matches how the surrounding code already handles out-of-range precision
and scale (clamp plus a warning log).
Doris 2.x and later are unaffected — `DorisTypeConverterV2` does not
override `getMaxDecimalScale()`, and the default keeps the previous behaviour
exactly.
No config option or default value changes.
### How was this patch tested?
Unit tests in both converter tests:
- `DorisTypeConvertorV1Test#testReconvertDecimal` — added a `DecimalType(20,
10)` case asserting the emitted type is `DECIMALV3(20,9)`, and updated the
existing `DecimalType(0, 0)` fallback assertion from scale 10 to 9. That
existing assertion encoded the buggy value, so it had to change; it is called
out here rather than folded in silently.
- `DorisTypeConvertorV2Test#testReconvertDecimal` — added the same
`DecimalType(20, 10)` case asserting `DECIMALV3(20,10)` is preserved, pinning
the V2 behaviour so a future change to the cap cannot silently alter it.
The assertions use the literal `9` rather than the new constant, so a wrong
constant value would still be caught.
```
./mvnw test -pl seatunnel-connectors-v2/connector-doris
-Dtest='DorisTypeConvertorV1Test,DorisTypeConvertorV2Test'
Tests run: 76, Failures: 0, Errors: 0, Skipped: 0
```
Reverting only the two main-code files makes V1 fail while V2 stays green,
which confirms both that the test reproduces the bug and that V2 is untouched:
```
[ERROR] DorisTypeConvertorV1Test.testReconvertDecimal:496
expected: <DECIMALV3(38,9)> but was: <DECIMALV3(38,10)>
[INFO] DorisTypeConvertorV2Test Tests run: 39, Failures: 0, Errors: 0,
Skipped: 0
```
No E2E test was added. `connector-doris-e2e` exists, but its containers run
a modern Doris, and this failure only occurs on the Doris 1.x path —
reproducing it end to end would mean adding a Doris 1.x container purely for
this case. The defect is entirely in pure type-mapping logic that the unit
tests cover directly. I do not have a Doris 1.x instance to verify against, so
the DecimalV2 `(27, 9)` limit is taken from the Doris documentation and the
error message in the issue rather than from my own reproduction.
`./mvnw spotless:apply` was run over the module.
### Check list
* [x] If any new Jar binary package adding in your PR, please add License
Notice according
[New License
Guide](https://github.com/apache/seatunnel/blob/dev/docs/en/developer/new-license.md)
— no new dependency
* [x] If necessary, please update the documentation to describe the new
feature. https://github.com/apache/seatunnel/tree/dev/docs — not applicable, no
option or documented behaviour changed
* [x] If necessary, please update `incompatible-changes.md` to describe the
incompatibility caused by this PR — not applicable, the change turns a hard
failure into the clamp-and-warn behaviour already used for out-of-range
precision
* [x] If you are contributing the connector code, please check that the
following files are updated — not applicable, no new connector
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
