JackieTien97 commented on code in PR #19234:
URL: https://github.com/apache/iotdb/pull/19234#discussion_r4549812090
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/protocol/thrift/impl/DataNodeInternalRPCServiceImpl.java:
##########
@@ -1887,6 +1888,17 @@ public TSStatus executeCQ(TExecuteCQ req) {
}
executedSQL = String.join(" ",
s.constructFormattedSQL().split("\n")).replaceAll(" +", " ");
+ // Refresh the owner's READ_DATA scope for each CQ execution.
+ TSStatus status =
+ AuthorityChecker.checkAuthority(
+ s,
+ new TreeAccessCheckContext(
+ session.getUserId(), session.getUsername(),
session.getClientAddress())
+ .setSqlString(executedSQL));
+ if (status.getCode() != TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+ return status;
Review Comment:
On failure this returns the status from `TreeAccessCheckVisitor.visitQuery`
unchanged. For an `AuthException`, that status is `new
TSStatus(e.getCode().getStatusCode())`, which drops `e.getMessage()`, and
`executeCQ` logs nothing on the DataNode.
Reproduced on this branch: after dropping the CQ owner, the ConfigNode logs
`Execute CQ cq_wild failed, TSStatus is TSStatus(code:301)` every 2 s with no
message. The DataNode log only shows `Session-88-cq_probe_user:cq_wild is
closing`, so an operator can't tell why the CQ is stuck.
Could we add context (CQ id, owner) to a non-success status here and log a
WARN (new i18n constant in both `en` and `zh`), or have `visitQuery` keep
`e.getMessage()`?
##########
integration-test/src/test/java/org/apache/iotdb/db/it/cq/IoTDBCQExecIT.java:
##########
@@ -52,6 +61,105 @@ public static void tearDown() throws Exception {
EnvFactory.getEnv().cleanClusterEnvironment();
}
+ @Test
+ public void testCQExecutionWithReadDataPermission() throws Exception {
+ testCQExecutionWithReadDataPermission(false);
+ }
+
+ @Test
+ public void testCQExecutionWithReadDataPermissionAlignByDevice() throws
Exception {
+ testCQExecutionWithReadDataPermission(true);
+ }
+
+ private void testCQExecutionWithReadDataPermission(boolean alignByDevice)
throws Exception {
+ String cqId = alignByDevice ? "cq_read_auth_aligned" : "cq_read_auth";
+ String database = "root." + cqId;
+ String username = cqId + "_user";
+ String password = "CQRead!Test2026";
+ String queryBody =
+ "SELECT * INTO "
+ + database
+ + ".target(::) FROM "
+ + database
+ + ".source"
+ + (alignByDevice ? " ALIGN BY DEVICE" : "");
+ String resultQuery = "SELECT s1, s2 FROM " + database + ".target";
+ String resultHeader = "Time," + database + ".target.s1," + database +
".target.s2,";
+
+ IClientManager<TEndPoint, SyncDataNodeInternalServiceClient> clients =
+ new IClientManager.Factory<TEndPoint,
SyncDataNodeInternalServiceClient>()
+ .createClientManager(
+ new
ClientPoolFactory.SyncDataNodeInternalServiceClientPoolFactory());
+ try (Connection connection = EnvFactory.getEnv().getConnection();
+ Statement statement = connection.createStatement()) {
+ statement.execute("CREATE USER " + username + " '" + password + "'");
Review Comment:
`CREATE USER` and `CREATE DATABASE` run before the `try` whose `finally`
drops the user, so the user leaks into the shared cluster if `CREATE DATABASE`
fails. Inside the `finally`, a failing `DELETE DATABASE` also skips `DROP
USER`, and any exception thrown during cleanup replaces the original assertion
failure. Moving `CREATE USER` into the `try`, or guarding each cleanup
statement, would avoid both.
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/protocol/thrift/impl/DataNodeInternalRPCServiceImpl.java:
##########
@@ -1887,6 +1888,17 @@ public TSStatus executeCQ(TExecuteCQ req) {
}
executedSQL = String.join(" ",
s.constructFormattedSQL().split("\n")).replaceAll(" +", " ");
+ // Refresh the owner's READ_DATA scope for each CQ execution.
+ TSStatus status =
+ AuthorityChecker.checkAuthority(
Review Comment:
With this check, a CQ whose INTO clause lists explicit target columns stops
making progress when the owner lacks READ_DATA on part of the source.
The authorized scope silently drops the unreadable source series, so
`IntoComponent.validate` rejects the query with 701 (source column count !=
target path count). `CQScheduleTask` does not advance `executionTime` after a
failure, so the scheduler retries the same window indefinitely.
Reproduced on this branch with READ_DATA granted only on `root.cqp.src.s1`:
```sql
CREATE CQ cq_explicit RESAMPLE EVERY 2s BOUNDARY <now + 4s> RANGE 1h
BEGIN SELECT s1, s2 INTO root.cqp.dst_explicit(m1, m2) FROM root.cqp.src END
```
Every 2 s the ConfigNode logs the same time range `[1794476375736,
1794479975736)`:
```
Execute CQ cq_explicit failed, TSStatus is TSStatus(code:701, message:select
into: the number of source columns and the number of target paths should be the
same., needRetry:false)
```
`root.cqp.dst_explicit` stays empty (even `s1` is not copied), and `SHOW
CQS` still reports `ACTIVE`. On the same cluster, `SELECT * INTO
root.cqp.dst_wild(::) FROM root.cqp.src` copies `s1` and advances normally.
So the two CQ shapes now diverge: wildcard CQs permanently skip unauthorized
series for windows that run while access is missing, while explicit-target CQs
stop entirely and, once access is restored, resume from the stalled window
(catching up window by window under BLOCKED). The error also doesn't mention
permissions. Could we document this in the PR description and pin it with an
IT, or return an explicit NO_PERMISSION when an explicitly referenced source
path is not authorized?
##########
integration-test/src/test/java/org/apache/iotdb/db/it/cq/IoTDBCQExecIT.java:
##########
@@ -52,6 +61,105 @@ public static void tearDown() throws Exception {
EnvFactory.getEnv().cleanClusterEnvironment();
}
+ @Test
+ public void testCQExecutionWithReadDataPermission() throws Exception {
+ testCQExecutionWithReadDataPermission(false);
+ }
+
+ @Test
+ public void testCQExecutionWithReadDataPermissionAlignByDevice() throws
Exception {
+ testCQExecutionWithReadDataPermission(true);
+ }
+
+ private void testCQExecutionWithReadDataPermission(boolean alignByDevice)
throws Exception {
+ String cqId = alignByDevice ? "cq_read_auth_aligned" : "cq_read_auth";
+ String database = "root." + cqId;
+ String username = cqId + "_user";
+ String password = "CQRead!Test2026";
+ String queryBody =
+ "SELECT * INTO "
+ + database
+ + ".target(::) FROM "
+ + database
+ + ".source"
+ + (alignByDevice ? " ALIGN BY DEVICE" : "");
+ String resultQuery = "SELECT s1, s2 FROM " + database + ".target";
+ String resultHeader = "Time," + database + ".target.s1," + database +
".target.s2,";
+
+ IClientManager<TEndPoint, SyncDataNodeInternalServiceClient> clients =
+ new IClientManager.Factory<TEndPoint,
SyncDataNodeInternalServiceClient>()
+ .createClientManager(
+ new
ClientPoolFactory.SyncDataNodeInternalServiceClientPoolFactory());
+ try (Connection connection = EnvFactory.getEnv().getConnection();
+ Statement statement = connection.createStatement()) {
+ statement.execute("CREATE USER " + username + " '" + password + "'");
+ statement.execute("CREATE DATABASE " + database);
+ try {
+ statement.execute("CREATE TIMESERIES " + database + ".source.s1 WITH
DATATYPE=INT64");
+ statement.execute("CREATE TIMESERIES " + database + ".source.s2 WITH
DATATYPE=INT64");
+ statement.execute("CREATE TIMESERIES " + database + ".target.s1 WITH
DATATYPE=INT64");
+ statement.execute("CREATE TIMESERIES " + database + ".target.s2 WITH
DATATYPE=INT64");
+ statement.execute(
+ "INSERT INTO "
+ + database
+ + ".source(time, s1, s2) VALUES (1, 11, 21), (1001, 12, 22),
(2001, 13, 23)");
+ statement.execute("GRANT SYSTEM ON root.** TO USER " + username);
+ statement.execute("GRANT WRITE_DATA ON " + database + ".target.** TO
USER " + username);
+ statement.execute("GRANT READ_DATA ON " + database + ".source.s1 TO
USER " + username);
+
+ try (Connection userConnection =
EnvFactory.getEnv().getConnection(username, password);
+ Statement userStatement = userConnection.createStatement()) {
+ // Keep the scheduler idle and execute deterministic windows through
its DataNode RPC.
+ userStatement.execute(
+ "CREATE CQ "
+ + cqId
+ + " RESAMPLE EVERY 1s BOUNDARY "
+ + (System.currentTimeMillis() + TimeUnit.DAYS.toMillis(1))
+ + " BEGIN "
+ + queryBody
+ + " END");
+ }
+ try {
+ TExecuteCQ request = new TExecuteCQ(queryBody, 0, 1000, 10000,
"UTC", cqId, username);
+ executeCQOnAllDataNodes(clients, request);
Review Comment:
All three `executeCQ` calls expect success, so the new `return status`
branch is never exercised. If that branch regressed (for example, execution
continued after a failed check), both tests would still pass.
Could we add cases for:
1. a dropped owner: `executeCQ` returns a non-success status and writes
nothing;
2. an explicit-target CQ (`SELECT s1, s2 INTO target(m1, m2)`) under partial
READ_DATA, asserting whichever behavior we settle on (see the comment on
`DataNodeInternalRPCServiceImpl`).
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/protocol/thrift/impl/DataNodeInternalRPCServiceImpl.java:
##########
@@ -1887,6 +1888,17 @@ public TSStatus executeCQ(TExecuteCQ req) {
}
executedSQL = String.join(" ",
s.constructFormattedSQL().split("\n")).replaceAll(" +", " ");
+ // Refresh the owner's READ_DATA scope for each CQ execution.
+ TSStatus status =
+ AuthorityChecker.checkAuthority(
+ s,
+ new TreeAccessCheckContext(
+ session.getUserId(), session.getUsername(),
session.getClientAddress())
Review Comment:
`session.getUserId()` is the placeholder `-1` passed to `supplySession(...)`
above (`// TODO: User the real userId`). The new access-check context, and the
audit records it produces, therefore attribute every CQ execution to user ID -1.
The Apache build shows no effect because `DNAuditLogger.log` is a no-op, but
editions that implement audit logging would record the wrong ID.
`AccessControlImpl` also makes many `SUPER_USER_ID == getUserId()` decisions;
if this path ever reaches them, a root-owned CQ would be treated as an ordinary
user.
The REST `AuthorizationHandler` already resolves the ID with
`AuthorityChecker.getUserId(userName).orElse(-1L)`. Doing the same in
`supplySession` would resolve the TODO and also give the `SessionInfo` used for
the INTO writes the real ID.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]