hongzhi-gao opened a new pull request, #19337:
URL: https://github.com/apache/iotdb/pull/19337
## Description
### Content1: Behavior and configuration
Add `enable_insert_type_cast`, a cluster-wide, hot-reloadable option that
defaults to `false`. When enabled, conflicting insert values can use explicit
query CAST conversions during schema validation. The change covers tree SQL,
typed/untyped Session writes, Tablets, and table FIELD values. It does not add
OBJECT support.
For example, with an existing `INT32` series:
```sql
CREATE TIMESERIES root.cast_demo.d.i WITH DATATYPE=INT32;
SET CONFIGURATION 'enable_insert_type_cast'='true';
INSERT INTO root.cast_demo.d(time, i) VALUES (1, '5');
INSERT INTO root.cast_demo.d(time, i) VALUES (2, 5.6);
SELECT i FROM root.cast_demo.d;
-- time=1: 5; time=2: 6
```
A typed Session/Tablet `TEXT` value `"5"` can likewise be written to an
`INT32` field. Disabling the option restores the previous mismatch behavior.
Existing compatible conversions remain in place, except that enabled `TIMESTAMP
-> TEXT/STRING` uses query CAST's datetime formatting instead of the decimal
timestamp string. Each analyzed request captures the option and time zone once.
### Content2: Conversion rules and failure handling
Reuse the existing `TypeServices` object-value CAST services and compare
persisted results with column-execution CAST in integration tests. Preserve SQL
expression source types so that `DOUBLE 5.6`, `STRING '5.6'`, and `BLOB X'35'`
are not treated as interchangeable strings.
| Input and target | Query column CAST | Insert with the option enabled |
| --- | --- | --- |
| `STRING '5' -> INT32` | `5` | `5` |
| `DOUBLE 5.6 -> INT32` | `6` | `6` |
| `STRING '5.6' -> INT32` | Conversion fails | Conversion fails |
| `INT32 20260228 -> DATE` | DATE encoding `20260228` | Valid date
`2026-02-28` |
| `INT32 20260230 -> DATE` | The conversion branch passes through the
integer encoding | Reject the invalid date before persistence |
| `BOOLEAN true/false -> DATE` | Integer encoding `1/0` | Reject because
neither encoding is a valid date |
| Typed `TIMESTAMP 5 -> TEXT/STRING`, millisecond precision | A datetime
string using the applicable query time zone | The same formatted string; with
the option disabled, retain the previous `"5"` behavior |
| `BLOB X'35' -> INT32` | Fails when parsing the hexadecimal representation
| Fails, rather than interpreting the bytes as `STRING '5'` |
The supported source/target types are `INT32`, `INT64`, `FLOAT`, `DOUBLE`,
`BOOLEAN`, `TEXT`, `STRING`, `DATE`, `TIMESTAMP`, and `BLOB`; individual values
must satisfy the target conversion. `OBJECT`, `UNKNOWN`, and `VECTOR` are
excluded from the new conversion path. Table TAG/ATTRIBUTE and time-column
handling retain their existing rules. Tree timestamp-to-text formatting follows
the server time zone; table conversions use the session time zone.
A Tablet column is converted into a temporary array and replaced only after
the entire column succeeds. Original NULL positions are preserved. Failed
conversions follow `enable_partial_insert`: with partial insertion enabled, the
failed measurement/column is rejected while other columns can succeed;
otherwise the insertion fails. Typed conversion errors include the
source/target types, offending timestamp/value, and a bounded conversion
reason. Pipe forced conversions retain their existing behavior. WAL, RPC, and
TsFile formats are unchanged.
### Content3: Verification
Executed:
- `IoTDBInsertTypeCastIT`: **7 tests passed**, no failures/errors/skips, on
**1 ConfigNode + 3 DataNodes** with both tree and table categories selected.
The global-switch test sends writes through each DataNode across
disabled/enabled/disabled phases and checks the observed data.
- The matrix covers **10 source types × 10 target types (100 pairs,
including identity)** and **140 sample/target combinations**, with
FLUSH/readback and query CAST comparisons for successful conversions. It also
asserts NULL results for rejected fields.
- `SchemaValidatorTest` and `InsertValuesTest`: **7 unit tests passed**, no
failures/errors/skips.
- Spotless formatting and `check-it-selection.py` passed.
Command for the final integration-test run:
```bash
mvn -B -o verify -pl integration-test -am -Pwith-integration-tests,ClusterIT
\
-DskipUTs -Dit.test=IoTDBInsertTypeCastIT \
-DintegrationTest.includedGroups=org.apache.iotdb.itbase.category.ClusterIT,org.apache.iotdb.itbase.category.TableClusterIT
\
-DfailIfNoTests=false -Dfailsafe.failIfNoSpecifiedTests=false \
-Ddevelocity.cache.local.enabled=false
-Ddevelocity.cache.remote.enabled=false -Ddevelocity.off=true
```
Full-reactor English and Chinese test compilation both passed (50 modules):
```bash
mvn -B -o test-compile -DskipTests \
-Ddevelocity.cache.local.enabled=false
-Ddevelocity.cache.remote.enabled=false -Ddevelocity.off=true
mvn -B -o test-compile -DskipTests -Pwith-zh-locale \
-Ddevelocity.cache.local.enabled=false
-Ddevelocity.cache.remote.enabled=false -Ddevelocity.off=true
```
The compile-only reactor initially lacked the distribution module's three
ZIP dependencies. Installing the server/CLI/ConfigNode ZIPs already produced by
the test build into the local Maven repository resolved this; no source or
dependency-configuration change was needed.
Not measured/verified here: Apache IoTDB performance benchmarks,
crash/recovery fault injection, or rolling upgrades. The complete type matrix
is exercised through table Tablets; other entry points have targeted coverage.
<hr>
This PR has:
- [x] been self-reviewed.
- [x] added documentation for new or modified features or behaviors in the
configuration template and this description.
- [x] added comments explaining non-obvious conversion semantics.
- [x] added integration tests.
- [x] been tested in a test IoTDB cluster.
<hr>
##### Key changed/added classes (or packages if there are too many classes)
in this PR
`SchemaValidator`, `InsertBaseStatement`, `InsertRowStatement`,
`InsertTabletStatement`, `InsertValues`, `AstUtil`, `IoTDBInsertTypeCastIT`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]