CritasWang opened a new pull request, #19150:
URL: https://github.com/apache/iotdb/pull/19150

   ## Summary
   
   Tree-model `export-schema` dropped tags and attributes entirely, and 
`import-schema` had no way to restore them even though `createMultiTimeseries` 
/ `createAlignedTimeseries` accept both. This PR makes a full export → import 
cycle preserve them exactly, including every special character.
   
   It is two logically separate pieces, kept as three commits so they can be 
reviewed (or split) independently:
   
   | Commit | Scope |
   | --- | --- |
   | `[Server] Render tags and attributes as valid JSON in SHOW TIMESERIES` | 
prerequisite defect fix |
   | `[Tools] Export and import tags and attributes in tree-model schema files` 
| the feature |
   | `[Tools] Clear the aliased lists at every import batch boundary` | 
pre-existing bug found while testing the feature |
   
   ## 1. Why the server change is required
   
   The client has **no** API that returns tags as a `Map` — 
`Session`/`SessionPool` only accept `tagsList` on write. Reading them is only 
possible through the `Tags` / `Attributes` columns of `SHOW TIMESERIES`, and 
`TimeSeriesSchemaSource.mapToString` concatenated keys and values **without 
escaping**, so that column was not parseable data:
   
   | tag value | before | consequence |
   | --- | --- | --- |
   | `a"b` | `{"k":"a"b"}` | invalid JSON, parse fails |
   | `a\b` | `{"k":"a\b"}` | parses "successfully" but yields `a<backspace>` — 
silent corruption |
   | `a<TAB>b` | `{"k":"a<TAB>b"}` | raw control character, invalid JSON |
   
   The fix escapes per RFC 8259 and keeps non-ASCII characters readable; it 
also removes the duplicate implementation in `LogicalViewSchemaSource`. Output 
is **byte-identical for values that need no escaping**, so ordinary schemas are 
unaffected.
   
   ## 2. File format
   
   Export now writes seven columns:
   
   ```
   Timeseries,Alias,DataType,Encoding,Compression,Tags,Attributes
   
root.t.d1.s1,,INT64,TS_2DIFF,SNAPPY,"{""room"":""101"",""type"":""sensor""}","{""owner"":""team
 a"",""note"":""a\""b""}"
   
root.t.d1.s2,aliasA,INT64,TS_2DIFF,SNAPPY,"{""desc"":""a,b"",""p"":""a\\b""}",
   root.t.d1.s3,,INT64,TS_2DIFF,SNAPPY,"{""q"":""`x`"",""multi"":""[  C  ]""}",
   root.t.d1.s4,,INT64,TS_2DIFF,SNAPPY,,
   ```
   
   - `Tags` / `Attributes` are compact JSON objects with **sorted keys** and 
HTML escaping disabled; an absent set is an **empty cell**.
   - The CSV layer uses **RFC4180** (quote `"`, no escape character, LF). JSON 
cells are therefore quoted with their inner quotes doubled, which is what makes 
them round-trip.
   
   ## 3. Compatibility
   
   - Files with the previous five-column header are still read with the legacy 
settings (`quote='`'`, `escape='\'`), so existing exports keep importing 
unchanged.
   - The new format is detected from the header (`Tags` or `Attributes` 
present), and such a header is rejected unless all seven expected columns are 
there.
   - **User-visible change to note in release notes:** tag/attribute values 
containing `"`, `\` or control characters are now rendered escaped in `SHOW 
TIMESERIES` (previously the output was not valid JSON at all). Values without 
such characters are unchanged.
   - External scripts that split that column on `","` would need adjusting.
   
   ## 4. Testing
   
   Unit tests (real classes, no mocks except the session pool):
   
   - `TimeSeriesSchemaSourceMapToStringTest` (18 tests): every escaping case, 
round-trip through a standard JSON parser, and byte-identity with the old 
algorithm for values that need no escaping.
   - `SchemaTagsFidelityTest` (14) and `SchemaTagsImportTest` (6): JSON cell 
encoding (sorted keys, no HTML escaping, empty cell), RFC4180 bytes, CSV 
round-trip for `"` `\` `,` tab newline CJK backtick, legacy detection, and the 
create-call arguments for both the normal and the aligned path. The batch test 
was validated by reverting the fix and observing it fail (`expected:<4> but 
was:<2>`).
   
   Full local runs: `mvn test -pl iotdb-client/cli` → 100/100; datanode test 
for the escaping change → 18/18; both locales (`-P with-zh-locale`) compile.
   
   End-to-end on a real standalone cluster (built from this branch): created 18 
time series covering `"`, `\`, a literal `\n`, tab, newline, CJK+emoji, literal 
`null`, empty string, double spaces, backtick, a key with a space, multiple 
tags, attributes-only, neither, and an alias; exported, dropped the database, 
imported, and compared the server's rendering field by field — **18/18 
identical**. Editing one tag in the exported file with a spreadsheet tool and 
re-importing also worked, leaving the other rows untouched.
   
   `SchemaTagsRoundTripIT` was added for CI. **It has not been executed 
locally** (`iotdb-core/ainode/dist/ainode/`, which the integration-test 
assembly references, is absent in this environment, and the test node template 
needs a full reactor build). It compiles, is spotless-clean, and passes the 
repository's IT-selection check, but its first real run will be in CI.
   
   ## 5. Not addressed here
   
   Logical views, schema templates and device alignment are still not exported 
(they need different creation statements or device-level information); the 
import batch fix above is included because misaligned tags would otherwise 
corrupt exactly this feature's data.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to