CritasWang opened a new pull request, #19150:
URL: https://github.com/apache/iotdb/pull/19150
## Summary
Tree-model `export-schema` dropped tags and attributes entirely, and
`import-schema` had no way to restore them even though `createMultiTimeseries`
/ `createAlignedTimeseries` accept both. This PR makes a full export → import
cycle preserve them exactly, including every special character.
It is two logically separate pieces, kept as three commits so they can be
reviewed (or split) independently:
| Commit | Scope |
| --- | --- |
| `[Server] Render tags and attributes as valid JSON in SHOW TIMESERIES` |
prerequisite defect fix |
| `[Tools] Export and import tags and attributes in tree-model schema files`
| the feature |
| `[Tools] Clear the aliased lists at every import batch boundary` |
pre-existing bug found while testing the feature |
## 1. Why the server change is required
The client has **no** API that returns tags as a `Map` —
`Session`/`SessionPool` only accept `tagsList` on write. Reading them is only
possible through the `Tags` / `Attributes` columns of `SHOW TIMESERIES`, and
`TimeSeriesSchemaSource.mapToString` concatenated keys and values **without
escaping**, so that column was not parseable data:
| tag value | before | consequence |
| --- | --- | --- |
| `a"b` | `{"k":"a"b"}` | invalid JSON, parse fails |
| `a\b` | `{"k":"a\b"}` | parses "successfully" but yields `a<backspace>` —
silent corruption |
| `a<TAB>b` | `{"k":"a<TAB>b"}` | raw control character, invalid JSON |
The fix escapes per RFC 8259 and keeps non-ASCII characters readable; it
also removes the duplicate implementation in `LogicalViewSchemaSource`. Output
is **byte-identical for values that need no escaping**, so ordinary schemas are
unaffected.
## 2. File format
Export now writes seven columns:
```
Timeseries,Alias,DataType,Encoding,Compression,Tags,Attributes
root.t.d1.s1,,INT64,TS_2DIFF,SNAPPY,"{""room"":""101"",""type"":""sensor""}","{""owner"":""team
a"",""note"":""a\""b""}"
root.t.d1.s2,aliasA,INT64,TS_2DIFF,SNAPPY,"{""desc"":""a,b"",""p"":""a\\b""}",
root.t.d1.s3,,INT64,TS_2DIFF,SNAPPY,"{""q"":""`x`"",""multi"":""[ C ]""}",
root.t.d1.s4,,INT64,TS_2DIFF,SNAPPY,,
```
- `Tags` / `Attributes` are compact JSON objects with **sorted keys** and
HTML escaping disabled; an absent set is an **empty cell**.
- The CSV layer uses **RFC4180** (quote `"`, no escape character, LF). JSON
cells are therefore quoted with their inner quotes doubled, which is what makes
them round-trip.
## 3. Compatibility
- Files with the previous five-column header are still read with the legacy
settings (`quote='`'`, `escape='\'`), so existing exports keep importing
unchanged.
- The new format is detected from the header (`Tags` or `Attributes`
present), and such a header is rejected unless all seven expected columns are
there.
- **User-visible change to note in release notes:** tag/attribute values
containing `"`, `\` or control characters are now rendered escaped in `SHOW
TIMESERIES` (previously the output was not valid JSON at all). Values without
such characters are unchanged.
- External scripts that split that column on `","` would need adjusting.
## 4. Testing
Unit tests (real classes, no mocks except the session pool):
- `TimeSeriesSchemaSourceMapToStringTest` (18 tests): every escaping case,
round-trip through a standard JSON parser, and byte-identity with the old
algorithm for values that need no escaping.
- `SchemaTagsFidelityTest` (14) and `SchemaTagsImportTest` (6): JSON cell
encoding (sorted keys, no HTML escaping, empty cell), RFC4180 bytes, CSV
round-trip for `"` `\` `,` tab newline CJK backtick, legacy detection, and the
create-call arguments for both the normal and the aligned path. The batch test
was validated by reverting the fix and observing it fail (`expected:<4> but
was:<2>`).
Full local runs: `mvn test -pl iotdb-client/cli` → 100/100; datanode test
for the escaping change → 18/18; both locales (`-P with-zh-locale`) compile.
End-to-end on a real standalone cluster (built from this branch): created 18
time series covering `"`, `\`, a literal `\n`, tab, newline, CJK+emoji, literal
`null`, empty string, double spaces, backtick, a key with a space, multiple
tags, attributes-only, neither, and an alias; exported, dropped the database,
imported, and compared the server's rendering field by field — **18/18
identical**. Editing one tag in the exported file with a spreadsheet tool and
re-importing also worked, leaving the other rows untouched.
`SchemaTagsRoundTripIT` was added for CI. **It has not been executed
locally** (`iotdb-core/ainode/dist/ainode/`, which the integration-test
assembly references, is absent in this environment, and the test node template
needs a full reactor build). It compiles, is spotless-clean, and passes the
repository's IT-selection check, but its first real run will be in CI.
## 5. Not addressed here
Logical views, schema templates and device alignment are still not exported
(they need different creation statements or device-level information); the
import batch fix above is included because misaligned tags would otherwise
corrupt exactly this feature's data.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]