HTHou commented on code in PR #19150:
URL: https://github.com/apache/iotdb/pull/19150#discussion_r4547326661


##########
iotdb-client/cli/src/main/java/org/apache/iotdb/tool/schema/AbstractSchemaTool.java:
##########
@@ -218,32 +228,71 @@ public static Boolean writeCsvFile(List<List<Object>> 
records, String filePath)
     }
   }
 
+  /**
+   * The RFC4180 CSV layout of the schema files, i.e. the delimiter is {@code 
','}, the quote char
+   * is {@code '"'} and no escape char is configured. A JSON object cell is 
therefore always quoted
+   * and its inner quotes are doubled, which is exactly what RFC4180 asks for.
+   */
+  static CSVFormat.Builder createRfc4180CsvFormat() {
+    return CSVFormat.Builder.create(CSVFormat.DEFAULT)
+        .setQuote('"')
+        .setQuoteMode(QuoteMode.MINIMAL)
+        .setRecordSeparator('\n');
+  }
+
+  /**
+   * Serializes tags or attributes into the compact JSON object cell of the 
schema file. Keys are
+   * sorted so that repeated exports of the same schema are identical, and 
HTML escaping is disabled
+   * so that printable characters such as {@code '}, {@code =}, {@code <} and 
{@code &} stay
+   * readable.
+   *
+   * @return an empty string when there is nothing to write, never {@code {}} 
or {@code null}
+   */
+  static String toJsonCell(Map<String, String> values) {
+    if (values == null || values.isEmpty()) {
+      return "";
+    }
+    return JSON_GSON.toJson(new TreeMap<>(values));
+  }
+
+  /**
+   * Parses a Tags / Attributes cell written by {@link #toJsonCell(Map)}.
+   *
+   * @return {@code null} when the cell carries no value, never an empty map
+   * @throws com.google.gson.JsonParseException when the cell is not a JSON 
object
+   */
+  static Map<String, String> parseJsonCell(String cell) {
+    if (cell == null) {
+      return null;
+    }
+    String trimmed = cell.trim();
+    if (trimmed.isEmpty() || "null".equals(trimmed)) {
+      return null;
+    }
+    Map<String, String> parsed = JSON_GSON.fromJson(trimmed, JSON_CELL_TYPE);

Review Comment:
   **[P2] Reject null metadata values before batching**
   
   Gson accepts a cell such as `{"room":null}` as a map containing a null 
value. Thrift then throws `NullPointerException` in `writeString` while 
serializing the map, so the entire `createMultiTimeseries` batch fails, 
including valid rows.
   
   Please validate that each value in Tags and Attributes is a JSON string 
before adding the row to the batch. The existing failed-records path can then 
capture the invalid row.



##########
iotdb-client/cli/src/main/java/org/apache/iotdb/tool/schema/ImportSchemaTree.java:
##########
@@ -268,31 +365,56 @@ private static void writeScheme(
                     compressionTypeRaw));
             failedRecords.add(recordObj.stream().collect(Collectors.toList()));
             failed = true;
+          } else if (!tagsParsed) {
+            failedRecords.add(recordObj.stream().collect(Collectors.toList()));
+            failed = true;
           } else {
             if (StringUtils.isBlank(alias)) {
               paths.add(path);
               dataTypes.add(dataType);
               encodings.add(encodingType);
               compressors.add(compressionType);
+              if (v2) {
+                tags.add(recordTags);
+                attributes.add(recordAttributes);
+              }
             } else {
               pathsWithAlias.add(path);
               dataTypesWithAlias.add(dataType);
               encodingsWithAlias.add(encodingType);
               compressorsWithAlias.add(compressionType);
               measurementAlias.add(alias);
+              if (v2) {
+                tagsWithAlias.add(recordTags);
+                attributesWithAlias.add(recordAttributes);
+              }
             }
             pointSize.getAndIncrement();
           }
           if (!failed && aligned) {
             String deviceId = path.substring(0, path.lastIndexOf("."));
             paths.add(0, path.substring(deviceId.length() + 1));
+            if (v2) {
+              // keep the maps aligned with the measurements list, whose head 
now holds this one
+              tags.add(0, recordTags);

Review Comment:
   **[P2] Exercise aligned import through the CLI and server**
   
   The CLI rejects `-aligned true` with `Unrecognized option`. The new test 
sets `aligned` directly and expects the request to contain both `s1` and 
`root.sg.d.s1` alongside one data type. The server rejects the full path 
because each measurement must be a single node name.
   
   Please register the CLI option and send one measurement with matching type, 
encoding, compression, alias, tags, and attributes per row. An end-to-end test 
would catch both failures.



##########
iotdb-client/cli/src/main/java/org/apache/iotdb/tool/schema/ImportSchemaTree.java:
##########
@@ -131,18 +134,68 @@ protected void importSchemaFromCsvFile(File file) {
     }
   }
 
-  private static CSVParser readCsvFile(String path) throws IOException {
-    return CSVFormat.Builder.create(CSVFormat.DEFAULT)
-        .setHeader()
-        .setSkipHeaderRecord(true)
-        .setQuote('`')
-        .setEscape('\\')
-        .setIgnoreEmptyLines(true)
+  static CSVParser readCsvFile(String path) throws IOException {
+    CSVFormat.Builder builder =
+        CSVFormat.Builder.create(CSVFormat.DEFAULT)
+            .setHeader()
+            .setSkipHeaderRecord(true)
+            .setIgnoreEmptyLines(true);
+    if (isV2Format(readHeaderNames(path))) {
+      builder.setQuote('"');
+    } else {
+      builder.setQuote('`').setEscape('\\');
+    }
+    return builder
         .build()
-        .parse(new InputStreamReader(new FileInputStream(path)));
+        .parse(new InputStreamReader(new FileInputStream(path), 
StandardCharsets.UTF_8));
+  }
+
+  /**
+   * Reads the column names of a schema file without parsing it, so that the 
CSV settings of the
+   * file can be selected before it is read.
+   */
+  static List<String> readHeaderNames(String path) throws IOException {
+    try (BufferedReader reader =
+        new BufferedReader(
+            new InputStreamReader(new FileInputStream(path), 
StandardCharsets.UTF_8))) {
+      String headerLine = reader.readLine();

Review Comment:
   **[P3] Use consistent header handling for format detection and parsing**
   
   The probe reads the first physical line. `CSVParser` skips leading empty 
lines, so a leading blank line sends a V2 file through the legacy parser and 
splits quoted Tags JSON containing a comma. A UTF-8 BOM also becomes part of 
the `Timeseries` header and causes validation to reject the file.
   
   Please apply the same rules for leading blank lines and an optional BOM 
during detection and parsing.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to