CritasWang commented on code in PR #18828:
URL: https://github.com/apache/iotdb/pull/18828#discussion_r4343931895


##########
iotdb-client/cli/src/main/java/org/apache/iotdb/tool/data/ImportDataTable.java:
##########
@@ -370,36 +383,39 @@ protected void writeData(
   }
 
   private static void writeAndEmptyDataSet(Tablet tablet, int retryTime) {
-    try (ITableSession session = sessionPool.getSession()) {
-      session.insert(tablet);
-    } catch (IoTDBConnectionException e) {
-      if (retryTime > 0) {
-        writeAndEmptyDataSet(tablet, --retryTime);
-      }
-    } catch (StatementExecutionException e) {
-      ioTPrinter.println(Constants.INSERT_CSV_MEET_ERROR_MSG + e.getMessage());
-      System.exit(1);
-    }
+    executeWithRetry(
+        () -> {
+          try (ITableSession session = sessionPool.getSession()) {
+            session.insert(tablet);
+          }
+        },
+        retryTime);
   }
 
   private void parseHeaders(
       List<String> headerNames,
       Map<String, TSDataType> headerTypeMap,
       Map<String, String> headerNameMap) {
-    String regex = "(?<=\\()\\S+(?=\\))";
+    String regex = "\\(([^()]+)\\)$";
     Pattern pattern = Pattern.compile(regex);
     for (String headerName : headerNames) {
-      headerName = headerName.toLowerCase();
+      String normalizedHeaderName = headerName.toLowerCase(Locale.ROOT);
+      // A complete table column name takes precedence over a possible type 
suffix.
+      if (dataTypes.containsKey(normalizedHeaderName)) {

Review Comment:
   With this "complete name first" rule, a typed CSV round trip now silently 
drops a column when the table has both `x` and `x(<type>)`.
   
   For a table with columns `value DOUBLE` and `value(double) DOUBLE`, `-dt 
true` exports the header `time,value(DOUBLE),value(double)(DOUBLE)`. On import:
   - `value(DOUBLE)` lowercases to `value(double)`, which is a real column, so 
it is mapped to `value(double)`.
   - `value(double)(DOUBLE)` goes through the suffix branch, also resolves to 
`value(double)`, and `headerNameMap.put` overwrites the first mapping.
   
   The `value` column therefore receives no data. No `.failed` file is written 
and the run exits 0. I reproduced this with a probe in `DataToolRegressionTest` 
(export via `ExportDataTable.exportBySql`, then `ImportDataTable.writeData`). 
The tablet contained `value=null, value(double)=2.5` instead of `value=1.5, 
value(double)=2.5`. With the reverse header order, `value`'s data would end up 
in `value(double)` instead.
   
   Before this commit the typed case worked and the untyped `value(double)` 
case broke. The header is ambiguous by nature, so I'd fail closed rather than 
pick one interpretation:
   1. First map headers that exactly match a column.
   2. Strip the type suffix only from headers that are still unmatched.
   3. If two headers resolve to the same column, reject the file 
(`markFileFailed`) instead of overwriting the mapping.
   
   With that, both single-column cases still round-trip, the untyped two-column 
case works, and the typed two-column case fails loudly instead of losing data. 
A regression test for this two-column case would be good.



##########
iotdb-client/cli/src/main/java/org/apache/iotdb/tool/data/ExportDataTable.java:
##########
@@ -131,31 +147,108 @@ public void exportBySql(String sql, int index) {
       }
     }
     for (int i = 0; i < exportSql.size(); i++) {
-      String path = targetDirectory + targetFile + i;
-      String table = tables.get(i);
-      try (SessionDataSet sessionDataSet =
-          tableSession.executeQueryStatement(exportSql.get(i), timeout)) {
-        if (Constants.SQL_SUFFIXS.equalsIgnoreCase(exportType)) {
-          exportToSqlFile(sessionDataSet, table, path);
-        } else if (Constants.TSFILE_SUFFIXS.equalsIgnoreCase(exportType)) {
-          long start = System.currentTimeMillis();
-          boolean isComplete = exportToTsFile(sessionDataSet, path + 
".tsfile", table);
-          if (isComplete) {
-            long end = System.currentTimeMillis();
-            ioTPrinter.println("Export completely!cost: " + (end - start) + " 
ms.");
+      String path = targetDirectory + targetFile + (customQuery ? index : i);
+      try {
+        String queryTable =
+            customQuery
+                ? (Constants.SQL_SUFFIXS.equalsIgnoreCase(exportType)
+                        || 
Constants.TSFILE_SUFFIXS.equalsIgnoreCase(exportType)
+                    ? getQueryTable(sql)
+                    : null)
+                : tables.get(i);
+        try (SessionDataSet sessionDataSet =
+            tableSession.executeQueryStatement(exportSql.get(i), timeout)) {
+          if (Constants.SQL_SUFFIXS.equalsIgnoreCase(exportType)) {
+            exportToSqlFile(sessionDataSet, queryTable, path);
+          } else if (Constants.TSFILE_SUFFIXS.equalsIgnoreCase(exportType)) {
+            long start = System.currentTimeMillis();
+            boolean isComplete = exportToTsFile(sessionDataSet, path + 
".tsfile", queryTable);
+            if (isComplete) {
+              long end = System.currentTimeMillis();
+              ioTPrinter.println("Export completely!cost: " + (end - start) + 
" ms.");
+            }
+          } else {
+            exportToCsvFile(sessionDataSet, path);
           }
-        } else {
-          exportToCsvFile(sessionDataSet, path);
+          sessionDataSet.closeOperationHandle();
+          ioTPrinter.println(Constants.EXPORT_COMPLETELY);
         }
-        sessionDataSet.closeOperationHandle();
-        ioTPrinter.println(Constants.EXPORT_COMPLETELY);
       } catch (StatementExecutionException
           | IoTDBConnectionException
           | IOException
           | WriteProcessException e) {
         ioTPrinter.println("Cannot dump result because: " + e.getMessage());
+        success = false;
       }
     }
+    return success;
+  }
+
+  private static String getQueryTable(String sql) throws 
StatementExecutionException {
+    String message =
+        String.format(
+            CliMessages
+                
.EXCEPTION_SQL_TSFILE_EXPORT_REQUIRES_A_QUERY_FROM_A_SINGLE_TABLE_IN_DATABASE_ARG_C9EB3768,
+            database);
+    RelationalSqlLexer lexer =
+        new RelationalSqlLexer(new 
CaseInsensitiveStream(CharStreams.fromString(sql)));
+    RelationalSqlParser parser = new RelationalSqlParser(new 
CommonTokenStream(lexer));
+    lexer.removeErrorListeners();
+    lexer.addErrorListener(
+        new BaseErrorListener() {
+          @Override
+          public void syntaxError(
+              Recognizer<?, ?> recognizer,
+              Object offendingSymbol,
+              int line,
+              int charPositionInLine,
+              String error,
+              RecognitionException cause) {
+            throw new ParseCancellationException(message);
+          }
+        });
+    parser.removeErrorListeners();
+    parser.setErrorHandler(new BailErrorStrategy());
+    List<RelationalSqlParser.QualifiedNameContext> sources = new ArrayList<>();
+    try {
+      new RelationalSqlBaseVisitor<Void>() {
+        @Override
+        public Void visitTableName(RelationalSqlParser.TableNameContext 
context) {
+          sources.add(context.qualifiedName());
+          return null;
+        }
+      }.visit(parser.singleStatement());
+    } catch (ParseCancellationException e) {
+      throw new StatementExecutionException(message);
+    }
+    // Multiple source relations do not have an unambiguous destination table.
+    if (sources.size() != 1) {
+      throw new StatementExecutionException(message);
+    }
+    List<RelationalSqlParser.IdentifierContext> names = 
sources.get(0).identifier();
+    if (names.size() > 2
+        || (names.size() == 2 && 
!identifierValue(names.get(0)).equalsIgnoreCase(database))) {
+      throw new StatementExecutionException(message);
+    }
+    String name = identifierValue(names.get(names.size() - 1));
+    if (!tables.contains(name)) {

Review Comment:
   Minor: when `-table` is also given, `init()` narrows `tables` to that single 
table. As a result, `-table t1 -q "select * from t2" -ft sql` (or `-ft tsfile`) 
fails with "requires a query from a single table in database ...", even though 
`t2` exists. The new `-q` help text says the query "Overrides -table", so 
either validate against the full `show tables` result here, or adjust the help 
text.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to