JackieTien97 commented on code in PR #19237:
URL: https://github.com/apache/iotdb/pull/19237#discussion_r4561915369


##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/parser/StatementGenerator.java:
##########
@@ -385,6 +389,19 @@ public static InsertTabletStatement 
createStatement(TSInsertTabletReq insertTabl
                 
ColumnCategory.values()[insertTabletReq.columnCategories.get(i).intValue()]);
       }
       insertStatement.setColumnCategories(columnCategories);
+      if (insertTabletReq.isSetDatabase()) {
+        if (insertTabletReq.getDatabase().isEmpty()) {
+          throw new IllegalArgumentException(

Review Comment:
   Malformed names come back with inconsistent status codes:
   
   | `database` | result |
   |---|---|
   | `""` | 301 `[EXECUTE_STATEMENT_ERROR(301)] Exception occurred: 
insertTablet failed. database is not specified for insert:` |
   | `" "` | 500 `Unknown database ` |
   | `"a.b"` | 500 `Unknown database a.b` |
   | `null` with no session database | 701 `database is not specified` |
   
   The DataNode also logs each 301/500 response as a WARN with a full stack 
trace, on every call. For `""` the cause is that 
`ErrorHandlingUtils.onQueryException` doesn't recognize 
`IllegalArgumentException`, so the exception falls through to 
`onNpeOrUnexpectedException` (DN log: `StatementGenerator.java:394`). In 
addition, `DATABASE_IS_NOT_SPECIFIED_FOR_INSERT` is a prefix that is supposed 
to be followed by the statement, as in the relational `Analyzer`. Used here, 
the message ends in a dangling colon (zh: "未指定插入操作的数据库:").
   
   `CREATE DATABASE` already validates names with 
`TableConfigTaskVisitor.validateDatabaseName`. It rejects `""`, blanks, `.`, 
characters outside `DATABASE_PATTERN`, and names longer than 64. It throws a 
`SemanticException` that wraps `IllegalPathException`, which comes back as 509 
and is logged at INFO without a stack trace. Calling it after lower-casing (see 
the comment above) gives every malformed name a clean user error.



##########
iotdb-protocol/thrift-datanode/src/main/thrift/client.thrift:
##########
@@ -274,6 +274,7 @@ struct TSInsertTabletReq {
   11: optional bool isCompressed
   12: optional list<byte> encodingTypes
   13: optional byte compressType
+  14: optional string database

Review Comment:
   **Blocking: an older server ignores this field and writes into the session 
database.**
   
   Thrift servers skip unknown optional fields, so a server without this change 
never sees `database` and falls back to the session's current database. I ran 
the PR client against a server built from 207503295e:
   
   ```java
   // session opened with database("cdb1"); cdb1.t1 and cdb2.t1 both exist
   session.insert(tablet("t1", 3 rows), "cdb2");          // returns OK
   session.insert(tablet("only_in_cdb2", 2 rows), "cdb2"); // returns OK
   ```
   
   Result: `cdb1.t1` gets the 3 rows, `cdb2.t1` stays at 0, and table 
`only_in_cdb2` is auto-created in `cdb1`. With no session database the call 
fails with 701, but whenever a session database is set the misrouting is silent.
   
   The description acknowledges this fallback ("an older server can ignore the 
unknown optional field and fall back to its session database"). Documenting it 
isn't enough, though: the call reports success while the data lands in a 
different database, and the caller has no way to detect that. A rolling upgrade 
is a realistic way to hit this, since a redirect can send the request to a node 
that hasn't been upgraded yet.
   
   Suggestion: in `openSession`, have the server advertise a capability in 
`TSOpenSessionResp.configuration`, next to the existing time-precision entry. 
The client records it per `SessionConnection`, because redirect connections are 
opened separately. When `database != null` and the connection lacks the 
capability, the client throws before it sends the request.



##########
integration-test/src/test/java/org/apache/iotdb/relational/it/session/IoTDBSessionRelationalIT.java:
##########
@@ -177,6 +177,63 @@ public static void main(String[] args)
     }
   }
 
+  @Test
+  public void testInsertTabletIntoSpecifiedDatabase()
+      throws IoTDBConnectionException, StatementExecutionException {
+    try (ITableSession session = 
EnvFactory.getEnv().getTableSessionConnection()) {
+      session.executeNonQueryStatement(
+          "CREATE TABLE db1.tablet_database (tag1 STRING TAG, m1 DOUBLE 
FIELD)");
+      session.executeNonQueryStatement(
+          "CREATE TABLE db2.tablet_database (tag1 STRING TAG, m1 DOUBLE 
FIELD)");
+
+      List<String> columnNames = Arrays.asList("tag1", "m1");
+      List<TSDataType> dataTypes = Arrays.asList(TSDataType.STRING, 
TSDataType.DOUBLE);
+      List<ColumnCategory> columnCategories =
+          Arrays.asList(ColumnCategory.TAG, ColumnCategory.FIELD);
+      Tablet tablet = new Tablet("tablet_database", columnNames, dataTypes, 
columnCategories, 2);
+      for (int row = 0; row < 2; row++) {
+        tablet.addTimestamp(row, row);
+        tablet.addValue("tag1", row, "tag:" + row);
+        tablet.addValue("m1", row, row * 1.0);
+      }
+
+      session.insert(tablet, "db1");
+      session.executeNonQueryStatement("USE db1");
+      session.insert(tablet, "db2");
+      Tablet currentDatabaseTablet =
+          new Tablet("tablet_database", columnNames, dataTypes, 
columnCategories, 1);
+      currentDatabaseTablet.addTimestamp(0, 2);
+      currentDatabaseTablet.addValue("tag1", 0, "tag:2");
+      currentDatabaseTablet.addValue("m1", 0, 2.0);
+      session.insert(currentDatabaseTablet, null);
+
+      try (SessionDataSet dataSet =
+          session.executeQueryStatement("SELECT * FROM db1.tablet_database")) {
+        assertTrue(dataSet.hasNext());
+        dataSet.next();
+        assertTrue(dataSet.hasNext());
+        dataSet.next();
+        assertTrue(dataSet.hasNext());
+        dataSet.next();
+        assertFalse(dataSet.hasNext());
+      }
+      try (SessionDataSet dataSet =
+          session.executeQueryStatement("SELECT * FROM db2.tablet_database")) {
+        assertTrue(dataSet.hasNext());
+        dataSet.next();
+        assertTrue(dataSet.hasNext());
+        dataSet.next();
+        assertFalse(dataSet.hasNext());
+      }
+      try (SessionDataSet dataSet = session.executeQueryStatement("SHOW 
CURRENT_DATABASE")) {
+        assertTrue(dataSet.hasNext());
+        assertEquals("db1", dataSet.next().getField(0).getStringValue());
+      }
+
+      assertThrows(StatementExecutionException.class, () -> 
session.insert(tablet, ""));

Review Comment:
   Please assert the status code here, not just the exception type. `""` 
currently returns 301, the generic failure code, so this assertion would still 
pass if the call failed for an unrelated reason.
   
   Missing cases:
   - **Tree-prefixed names.** Try `"root.xxx"`, `"ROOT.xxx"` and `"Root.xxx"` 
while both tree database `root.xxx` and table database `xxx` exist. Assert the 
error code, and assert that no device appears under `root.xxx.**` (see the 
comment on `StatementGenerator`).
   - **The grouped path with an explicit database.** No test runs 
`insertRelationalTabletByGroup` with a database set. The new unit test uses a 
single device. In the IT, each explicit-database insert starts with an empty 
route map for that database, so it always goes to the default connection. 
Extending `SessionCacheLeaderTest` with a multi-device tablet whose devices 
have different cached leaders, written to an explicit database that differs 
from the session's, would cover the grouping, the redirect bookkeeping and the 
per-group requests carrying the database.



##########
iotdb-client/isession/src/main/java/org/apache/iotdb/isession/ISession.java:
##########
@@ -70,7 +70,15 @@ void open(
       boolean enableRPCCompression,
       int connectionTimeoutInMs,
       Map<String, TEndPoint> deviceIdToEndpoint,
-      Map<IDeviceID, TEndPoint> tabletModelDeviceIdToEndpoint,
+      Map<IDeviceID, TEndPoint> tableModelDeviceIdToEndpoint,
+      INodeSupplier nodeSupplier)
+      throws IoTDBConnectionException;
+
+  void openWithTableModelDatabaseCache(

Review Comment:
   This adds an internal cache-sharing hook to `ISession`, a public interface 
that users program against, and its nested-map parameter exposes the cache 
layout. The only callers are in `SessionPool`. The method has to be on the 
interface only because `getSession()` declares `ISession session = 
queue.poll();`. Opening the new connection through a local `Session` would 
avoid that:
   
   ```java
   Session newSession = constructNewSession();
   newSession.openWithTableModelDatabaseCache(...);
   session = newSession;
   ```
   
   `tryConstructNewSession()` already uses a `Session`.
   
   Separately, the new abstract methods here and the new 
`ITableSession.insert(Tablet, String)` break every third-party implementation 
of these interfaces at compile time. `ISession` already uses `default` methods 
for this kind of evolution (e.g. `isEnableCacheLeader`). `insert(Tablet, 
String)` could be a `default` method that delegates to `insert(tablet)` when 
`database == null` and throws `UnsupportedOperationException` otherwise.



##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/parser/StatementGenerator.java:
##########
@@ -385,6 +389,19 @@ public static InsertTabletStatement 
createStatement(TSInsertTabletReq insertTabl
                 
ColumnCategory.values()[insertTabletReq.columnCategories.get(i).intValue()]);
       }
       insertStatement.setColumnCategories(columnCategories);
+      if (insertTabletReq.isSetDatabase()) {
+        if (insertTabletReq.getDatabase().isEmpty()) {
+          throw new IllegalArgumentException(
+              DataNodeQueryMessages.DATABASE_IS_NOT_SPECIFIED_FOR_INSERT);
+        }
+        if (!PathUtils.isTableModelDatabase(insertTabletReq.getDatabase())) {

Review Comment:
   **Blocking: the tree-model check is case-sensitive and runs before 
lower-casing, so `ROOT.xxx` gets through and the rows are written into the tree 
database.**
   
   `PathUtils.isTableModelDatabase` is just `!startsWith("root.")`, so 
`"ROOT.treedb"` and `"Root.treedb"` pass. The name is then lower-cased to 
`root.treedb`. Schema validation and the privilege check resolve the table 
through `unQualifyDatabaseName(...)`, which gives table `treedb.tt`, while 
schema/data partitioning and device creation use `root.treedb`. The other entry 
points don't have this gap: `openSession` lower-cases before 
`IClientSession.setDatabaseName` checks the name, and SQL rejects a 
`"ROOT.xxx".t` prefix with 701 (`The tree model database shall not be specified 
in table model.`).
   
   Repro on this head. Tree database `root.treedb` has device `d1`, and table 
database `treedb` has table `tt(tag1 TAG, m1 FIELD)`:
   
   ```java
   session.insert(tablet("tt", tags d0, d1; 2 rows), "ROOT.treedb"); // OK
   session.insert(tablet("tt", 1 row), "Root.treedb");               // OK
   session.insert(tablet("tt", 1 row), "treedb");                    // OK 
(control)
   ```
   
   | | before | after |
   |---|---|---|
   | table `SELECT count(*) FROM treedb.tt` | 0 | 1 (only the control row) |
   | tree `COUNT DEVICES root.treedb.**` | 1 | 3: new aligned devices 
`root.treedb.tt.d0` and `root.treedb.tt.d1` |
   | tree `COUNT TIMESERIES root.treedb.**` | 1 | 1 |
   
   The rows are written: after `FLUSH` they are in a TsFile under 
`sequence/root.treedb/...`. Neither model can read them, though. Tree `SELECT * 
FROM root.treedb.tt.d0` returns 0 rows and `SHOW TIMESERIES root.treedb.tt.**` 
is empty.
   
   This also gets around the tree privilege. Take a user that has only `GRANT 
INSERT ON DATABASE treedb`:
   - tree `INSERT INTO root.treedb.d9(time, s1) VALUES (1, 1)` → 803, needs 
`WRITE_DATA on [root.treedb.d9.s1]`
   - `insert(tablet, "root.treedb")` → 701
   - `insert(tablet, "ROOT.treedb")` → **OK**, and device `root.treedb.tt.pv2` 
is created in the tree database
   
   If there is no matching table database, the same input surfaces internal 
errors instead of a clean 701:
   - `"ROOT.db1"` → 500 `Create SchemaPartition failed because the database: 
root.db1 is not exists`
   - `"ROOT.db1"` with a table that doesn't exist yet → 301 
`org.apache.iotdb.confignode.exception.DatabaseNotExistsException: Database: 
root.db1 doesn't exist.`
   - `"ROOT.treedb"` before table database `treedb` existed → 500 `Unknown 
database treedb`
   
   Also, the reused `DATABASE_NAME_IS_TREE_MODEL_DATABASE` ends with "...which 
is not allowed to set in the client session." (zh: "...不允许设置到客户端会话中。"). This 
API deliberately leaves the session untouched. 
`THE_TREE_MODEL_DATABASE_SHALL_NOT_BE_SPECIFIED`, which the SQL path uses for 
the same case, fits better.
   
   Suggested shape:
   
   ```java
   final String database = 
insertTabletReq.getDatabase().toLowerCase(Locale.ENGLISH);
   if (!PathUtils.isTableModelDatabase(database)) {
     throw new SemanticException(
         DataNodeQueryMessages.THE_TREE_MODEL_DATABASE_SHALL_NOT_BE_SPECIFIED);
   }
   TableConfigTaskVisitor.validateDatabaseName(database); // see the comment on 
the empty-name check
   insertStatement.setDatabaseName(database);
   ```



##########
iotdb-client/session/src/main/java/org/apache/iotdb/session/Session.java:
##########
@@ -3101,6 +3166,12 @@ private void 
insertRelationalTabletByGroup(Map<SessionConnection, Tablet> relati
     }
   }
 
+  private static void setRelationalRequestDatabase(TSInsertTabletReq request, 
String database) {

Review Comment:
   Nit: the null check is redundant. The generated `setDatabase(null)` leaves 
the field unset, because `isSetDatabase()` is `database != null`. More 
importantly, the database is attached at three separate call sites, each right 
after `setWriteToTable(true)`, so a future send path can easily forget it. A 
single builder for relational tablet requests, taking the tablet and the 
database, would keep this in one place. That builder is also the natural home 
for the server-capability check suggested on `client.thrift`.



##########
iotdb-client/session/src/main/java/org/apache/iotdb/session/Session.java:
##########
@@ -1524,11 +1556,16 @@ private void handleRedirection(IDeviceID deviceId, 
TEndPoint endpoint) {
                 }
               });
       if (connection == null) {
-        tableModelDeviceIdToEndpoint.remove(deviceId);
+        databaseToEndpoint.remove(deviceId, endpoint);
       }
     }
   }
 
+  private String resolveTableModelRouteDatabase(String database) {
+    String routeDatabase = database == null ? this.database : database;

Review Comment:
   Nit: the route-cache key is the raw string, while the server lower-cases it, 
so `"DB1"` and `"db1"` get separate route maps for the same database. 
`getTableModelDatabaseRoutes` also calls `computeIfAbsent` before the server 
has validated the name. Every invalid name (`""`, `"ROOT.x"`, typos) therefore 
leaves an empty entry in a map that a `SessionPool` shares across sessions. 
Lower-casing here, as the server does, and only creating the entry when a 
redirect is actually recorded would avoid both problems.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to