PDGGK opened a new pull request, #112: URL: https://github.com/apache/iotdb-extras/pull/112
This is the second PR of the GSoC series (follows #110, the timeseries `TimeseriesDao`, now merged). It adds the latest-telemetry and attributes storage paths for running ThingsBoard on Apache IoTDB 2.x Table Mode. ## What this adds **1. `IoTDBTableLatestDao`** — the ThingsBoard `TimeseriesLatestDao` SPI on Table Mode. - Reads are **derived-latest** over the existing `telemetry` table (`ORDER BY time DESC LIMIT 1` for a single key; `LAST_BY(col, time) + MAX(time) GROUP BY key` for all keys), so the normal full-save path needs no separate latest table. - Plus a **minimal latest overlay** (`telemetry_latest`) that captures the latest-only write/delete cases the single-table derived read cannot satisfy (e.g. the EntityView `LATEST_AND_WS` copy path calls `saveLatest` with no paired `save()`). `findLatest`/`findAllLatest` merge the derived latest with the overlay by max timestamp per key. The overlay table is created only under the latest selector. **2. `IoTDBTableAttributesDao`** — the ThingsBoard `AttributesDao` SPI on Table Mode, against a dedicated `entity_attributes` table. - This is a **stretch / opt-in** path: it is **inert by default**. It activates only when `database.attributes.type=iotdb-table` is set — a selector that no current ThingsBoard release exposes — so a normal deployment never instantiates it and attributes keep flowing to the host entity DB. No `@Repository`; an `AttributesDaoConflictGuard` fails startup if a conflicting host `AttributesDao` is present; a constructor gate requires an explicit `iotdb.attributes.cluster_mode` acknowledgement. Both follow the same activation isolation + compile-only ThingsBoard stub surface (excluded from the built jar) as #110, and reuse the module's single named session pool (deduped via `@ConditionalOnMissingBean(name=...)`); each DAO can be activated independently. ## Testing Verified with the build cache off: **141 unit tests + 31 Testcontainers integration tests** (Latest 11 / Timeseries 9 / Attributes 11) against `apache/iotdb:2.0.8-standalone`; apache-rat, checkstyle, spotless, dependency-analyze, and enforcer all clean. The compile-only SPI stubs are pinned to the real ThingsBoard v4.3.1.2 signatures by `StrategyFContractTest`. ## Points I'd appreciate review on - **Latest overlay shape.** To close the latest-only data-loss gap, `saveLatest` writes the overlay on every call: the async batch writer means the DAO cannot tell a latest-only `saveLatest` from a full-save's paired `saveLatest`, so `telemetry_latest` ends up being a per-key latest store (one row per key, delete-then-insert), merged with the derived read. It is therefore effectively a latest table (one extra write per latest update). It closes the gaps; I'd welcome confirmation the shape is acceptable, or a preference for a narrower variant. - **Documented Phase-1 residuals** (in the `IoTDBTableLatestDao` javadoc): `version` is always `null` (IoTDB has no per-series sequence — same as the Cassandra backend; the EDQS latest-index `onUpdate` is suppressed for latest-only writes); for a purely telemetry-derived (full-save) latest, `removeLatest` and a concurrent historical delete are separate futures, so a re-read can transiently disagree until the historical delete lands; `telemetry_latest` is `TTL='INF'` with no entity-level cleanup; a same-timestamp cross-store type change resolves to the overlay value. - **`AttributesDao.findAllKeysByDeviceProfileId`** with a non-null `DeviceProfileId` returns an empty list — matching the official non-relational backend (`CassandraBaseTimeseriesLatestDao.findAllKeysByDeviceProfileId` also returns `Collections.emptyList()`); the sole caller is a TENANT_ADMIN config-time UI key-enumeration endpoint (`GET /api/deviceProfile/devices/keys/attributes`) that tolerates empty. The rationale is documented in the DAO javadoc + README; a real device→profile lookup is a future optional enhancement. Happy to iterate. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
