hongzhi-gao opened a new pull request, #19337:
URL: https://github.com/apache/iotdb/pull/19337

   ## Description
   
   ### Content1: Behavior and configuration
   
   Add `enable_insert_type_cast`, a cluster-wide, hot-reloadable option that 
defaults to `false`. When enabled, conflicting insert values can use explicit 
query CAST conversions during schema validation. The change covers tree SQL, 
typed/untyped Session writes, Tablets, and table FIELD values. It does not add 
OBJECT support.
   
   For example, with an existing `INT32` series:
   
   ```sql
   CREATE TIMESERIES root.cast_demo.d.i WITH DATATYPE=INT32;
   SET CONFIGURATION 'enable_insert_type_cast'='true';
   INSERT INTO root.cast_demo.d(time, i) VALUES (1, '5');
   INSERT INTO root.cast_demo.d(time, i) VALUES (2, 5.6);
   SELECT i FROM root.cast_demo.d;
   -- time=1: 5; time=2: 6
   ```
   
   A typed Session/Tablet `TEXT` value `"5"` can likewise be written to an 
`INT32` field. Disabling the option restores the previous mismatch behavior. 
Existing compatible conversions remain in place, except that enabled `TIMESTAMP 
-> TEXT/STRING` uses query CAST's datetime formatting instead of the decimal 
timestamp string. Each analyzed request captures the option and time zone once.
   
   ### Content2: Conversion rules and failure handling
   
   Reuse the existing `TypeServices` object-value CAST services and compare 
persisted results with column-execution CAST in integration tests. Preserve SQL 
expression source types so that `DOUBLE 5.6`, `STRING '5.6'`, and `BLOB X'35'` 
are not treated as interchangeable strings.
   
   | Input and target | Query column CAST | Insert with the option enabled |
   | --- | --- | --- |
   | `STRING '5' -> INT32` | `5` | `5` |
   | `DOUBLE 5.6 -> INT32` | `6` | `6` |
   | `STRING '5.6' -> INT32` | Conversion fails | Conversion fails |
   | `INT32 20260228 -> DATE` | DATE encoding `20260228` | Valid date 
`2026-02-28` |
   | `INT32 20260230 -> DATE` | The conversion branch passes through the 
integer encoding | Reject the invalid date before persistence |
   | `BOOLEAN true/false -> DATE` | Integer encoding `1/0` | Reject because 
neither encoding is a valid date |
   | Typed `TIMESTAMP 5 -> TEXT/STRING`, millisecond precision | A datetime 
string using the applicable query time zone | The same formatted string; with 
the option disabled, retain the previous `"5"` behavior |
   | `BLOB X'35' -> INT32` | Fails when parsing the hexadecimal representation 
| Fails, rather than interpreting the bytes as `STRING '5'` |
   
   The supported source/target types are `INT32`, `INT64`, `FLOAT`, `DOUBLE`, 
`BOOLEAN`, `TEXT`, `STRING`, `DATE`, `TIMESTAMP`, and `BLOB`; individual values 
must satisfy the target conversion. `OBJECT`, `UNKNOWN`, and `VECTOR` are 
excluded from the new conversion path. Table TAG/ATTRIBUTE and time-column 
handling retain their existing rules. Tree timestamp-to-text formatting follows 
the server time zone; table conversions use the session time zone.
   
   A Tablet column is converted into a temporary array and replaced only after 
the entire column succeeds. Original NULL positions are preserved. Failed 
conversions follow `enable_partial_insert`: with partial insertion enabled, the 
failed measurement/column is rejected while other columns can succeed; 
otherwise the insertion fails. Typed conversion errors include the 
source/target types, offending timestamp/value, and a bounded conversion 
reason. Pipe forced conversions retain their existing behavior. WAL, RPC, and 
TsFile formats are unchanged.
   
   ### Content3: Verification
   
   Executed:
   
   - `IoTDBInsertTypeCastIT`: **7 tests passed**, no failures/errors/skips, on 
**1 ConfigNode + 3 DataNodes** with both tree and table categories selected. 
The global-switch test sends writes through each DataNode across 
disabled/enabled/disabled phases and checks the observed data.
   - The matrix covers **10 source types × 10 target types (100 pairs, 
including identity)** and **140 sample/target combinations**, with 
FLUSH/readback and query CAST comparisons for successful conversions. It also 
asserts NULL results for rejected fields.
   - `SchemaValidatorTest` and `InsertValuesTest`: **7 unit tests passed**, no 
failures/errors/skips.
   - Spotless formatting and `check-it-selection.py` passed.
   
   Command for the final integration-test run:
   
   ```bash
   mvn -B -o verify -pl integration-test -am -Pwith-integration-tests,ClusterIT 
\
     -DskipUTs -Dit.test=IoTDBInsertTypeCastIT \
     
-DintegrationTest.includedGroups=org.apache.iotdb.itbase.category.ClusterIT,org.apache.iotdb.itbase.category.TableClusterIT
 \
     -DfailIfNoTests=false -Dfailsafe.failIfNoSpecifiedTests=false \
     -Ddevelocity.cache.local.enabled=false 
-Ddevelocity.cache.remote.enabled=false -Ddevelocity.off=true
   ```
   
   Full-reactor English and Chinese test compilation both passed (50 modules):
   
   ```bash
   mvn -B -o test-compile -DskipTests \
     -Ddevelocity.cache.local.enabled=false 
-Ddevelocity.cache.remote.enabled=false -Ddevelocity.off=true
   mvn -B -o test-compile -DskipTests -Pwith-zh-locale \
     -Ddevelocity.cache.local.enabled=false 
-Ddevelocity.cache.remote.enabled=false -Ddevelocity.off=true
   ```
   
   The compile-only reactor initially lacked the distribution module's three 
ZIP dependencies. Installing the server/CLI/ConfigNode ZIPs already produced by 
the test build into the local Maven repository resolved this; no source or 
dependency-configuration change was needed.
   
   Not measured/verified here: Apache IoTDB performance benchmarks, 
crash/recovery fault injection, or rolling upgrades. The complete type matrix 
is exercised through table Tablets; other entry points have targeted coverage.
   
   <hr>
   
   This PR has:
   - [x] been self-reviewed.
   - [x] added documentation for new or modified features or behaviors in the 
configuration template and this description.
   - [x] added comments explaining non-obvious conversion semantics.
   - [x] added integration tests.
   - [x] been tested in a test IoTDB cluster.
   
   <hr>
   
   ##### Key changed/added classes (or packages if there are too many classes) 
in this PR
   
   `SchemaValidator`, `InsertBaseStatement`, `InsertRowStatement`, 
`InsertTabletStatement`, `InsertValues`, `AstUtil`, `IoTDBInsertTypeCastIT`.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to