PDGGK opened a new pull request, #110: URL: https://github.com/apache/iotdb-extras/pull/110
## What this adds A new Maven module **`iotdb-thingsboard-table`** that implements ThingsBoard's historical telemetry DAO SPI on top of **Apache IoTDB 2.0.8 Table Mode** (relational SQL via `ITableSession` / Tablet writes). This is the **first PR of a staged series**; it delivers the write + raw-read foundation, packaged so it activates only on an explicit opt-in and otherwise stays completely inert. ### DAO implementation - **`IoTDBTableBaseDao`** — `ITableSessionPool` wiring (Spring constructor injection), `iotdb.*` configuration binding, and `getEntry()` mapping of the five typed columns (`bool_v/long_v/double_v/str_v/json_v`) with fail-fast on the single-typed-column schema invariant. - **`IoTDBTableTimeseriesDao.save()`** — an async bounded-queue batch writer: rows are mapped to a sparse IoTDB Tablet (TAG columns declared low-cardinality-first — `entity_type, tenant_id, key, entity_id` — plus one populated typed FIELD per `DataType`), flushed by size/linger, with retry+backoff on connection errors, reject-on-full back-pressure (fails the future rather than blocking the caller, with reject counters and a rate-limited WARN — at most one per 10s with cumulative counts — so dropped points are never silent), and a graceful shutdown drain. - **`findAllAsync` (raw)** + **`remove`** + **`savePartition`** — the non-aggregated read path (half-open ranges, escaped key/order, rows mapped back to `BasicTsKvEntry`), the delete path, and the partition no-op, all on a bounded read thread pool. Per the ThingsBoard `AbstractChunkedAggregationTimeseriesDao` contract, a query is routed to raw when `aggregation == NONE || interval < 1`. - Aggregation, latest telemetry and attributes arrive in later PRs. The not-yet-implemented **positive-interval aggregation** path in `IoTDBTableTimeseriesDao` throws `UnsupportedOperationException` and is unreachable by default thanks to the explicit raw-only opt-in. `IoTDBTableLatestDao` / `IoTDBTableAttributesDao` ship only as **unregistered inert skeletons** (not Spring beans, no ThingsBoard interface binding), so no configuration selector can route traffic to a non-working DAO. ### Activation & runtime (inert by default, opt-in foundation) - **Spring auto-configuration with classpath isolation** — `IoTDBTableConfiguration` is an `@AutoConfiguration` registered via `META-INF/spring/...AutoConfiguration.imports` + `META-INF/spring.factories`. The outer class carries a **string-based `@ConditionalOnClass(name="org.thingsboard.server.dao.timeseries.TimeseriesDao")`** and contains no bean method or annotation that force-loads a ThingsBoard type, so on a non-ThingsBoard classpath Spring evaluates the condition from ASM metadata and skips the module without a `NoClassDefFoundError`. All ThingsBoard-referencing beans live in a nested `@Configuration`; `@ConditionalOnMissingBean(type=...)` uses the string form. - **Inert until explicitly opted in** — because this PR delivers only the write + raw-read path (aggregation lands in a later PR), the live `TimeseriesDao`, session pool, writer and schema bootstrap activate **only when BOTH `database.ts.type=iotdb-table` AND `iotdb.ts.experimental-raw-only=true`** are set. A normal deployment (selector alone, or neither) gets nothing, so the not-yet-implemented aggregation path is never reachable through the public SPI. Activation uses a small case-insensitive `Condition`, not SpEL. - **Module-owned session pool** — the module registers its session pool under a dedicated bean name and injects it everywhere by qualifier, so a host-provided `ITableSessionPool` can never silently substitute for it. - **No silent half-activation** — if the backend is enabled but the host already provides a conflicting non-IoTDB `TimeseriesDao`, a `BeanFactoryPostProcessor` fails startup with a clear message (trusting only resolvable bean types, fail-closed) rather than leaving the pool/bootstrap running while the DAO is shadowed. - **`IoTDBTableSchemaBootstrap`** — creates the IoTDB database/table on first activation (same opt-in guard + `@ConditionalOnProperty(iotdb.schema.bootstrap, matchIfMissing=true)`), so the first write does not fail on a missing table. The configured database name is validated before it is spliced into DDL. - **`iotdb.*` config is bound/validated only when the backend is selected** (`@EnableConfigurationProperties` sits on the conditional nested config), so an unrelated host with stray `iotdb.*` properties is unaffected. ## ThingsBoard compile surface (Strategy F) ThingsBoard's `dao`/`common` artifacts are not published to Maven Central, so the SPI/value types the module compiles against are provided as a **compile-only** source surface under `src/provided/java` and **excluded from the built jar** (`org/thingsboard/**`, `org/apache/commons/**`); at runtime the real ThingsBoard classpath provides them. The surface is kept in sync with ThingsBoard **v4.3.1.2** (each stub file carries a provenance header with the verified version + date), and `StrategyFContractTest` pins the exact `TimeseriesDao` SPI signatures the DAO depends on so any accidental drift fails the build. Integration tests run against `target/classes` (which retains the compile surface). ## Reactor / build impact - The module is added to the root reactor through a **profile activated by `<jdk>[17,)</jdk>`** (it uses Java-17 language features), so existing JDK 8/11 root builds skip it and only 17/21 jobs build it — no change to current CI on older JDKs. - **The parent `tsfile.version` is left untouched at `2.1.1`.** The iotdb-session 2.0.8 Tablet write API needs tsfile `2.3.0`, so the bump is a **module-local property override** in `iotdb-thingsboard-table/pom.xml` — only this module resolves the newer tsfile; every other connector keeps `2.1.1`, so there is zero blast radius on the rest of the reactor. - The module's Testcontainers integration tests run only under an explicit `-Piotdb-table-it` profile, so a default `mvn install` runs unit tests only and does not require Docker. ## Tests 73 unit tests (type mapping, batch flush/linger, back-pressure, retry, shutdown drain + forced-stop settle guarantees, reject-WARN rate limiting, raw read SQL/mapping, half-open delete, blank-key fail-fast, read-pool reject/drain, auto-config discovery + classpath isolation, named-pool ownership, conflict-guard fail-fast, schema bootstrap + DDL column-order pins, Strategy-F contract) plus 9 Testcontainers integration tests against a real `apache/iotdb:2.0.8-standalone` container (round-trip all five types, order/limit, half-open end, scoped delete, key escaping, fresh-database bootstrap through a database-bound pool). `mvn apache-rat:check` is clean. ## Known limitation (documented, in the active path) This affects the write + raw-read surface delivered here, not a deferred placeholder: if the **same** `(tenant, entity, key, ts)` point has its data type changed across two separate flushes, the two typed columns coexist on that row, and the raw read **fails fast** on the single-typed-column invariant (`IoTDBTableBaseDao.getEntry`) rather than silently returning a wrong value. Within a single flush the writer already de-duplicates such a type change. Full delete-then-insert reconciliation across flushes is deferred to a later PR in the series. The behaviour is pinned by a unit test and an integration test and documented in the module README. ## Context This module is the first deliverable of a Google Summer of Code 2026 project to add an Apache IoTDB 2.x Table Mode storage backend to ThingsBoard. The design (schema, current-state analysis, Spring activation, and the staged-PR plan) was shared and discussed earlier on the `[email protected]` list. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
