PDGGK commented on PR #19107: URL: https://github.com/apache/iotdb/pull/19107#issuecomment-6717101251
Thanks for working on this. I built 3f6dd87 and today's master (d8800f1) and ran each as a standalone server (one ConfigNode, one DataNode). The functions come from a small jar. `lazy_f` and the UDF in the slow query below call a helper class while evaluating rows, so the JVM loads that class only when it is first used. **Before and after** | steps | master | this PR | |---|---|---| | `CREATE FUNCTION lazy_f` (table), then `CREATE FUNCTION tree_f` (tree, same jar), then `SELECT lazy_f(v) FROM db.t` | `301: lazy.Helper` | correct values | | the same two functions, a restart, then the same `SELECT` | `301: lazy.Helper` | correct values | | a `SELECT` whose UDF sleeps 1 s on each of its first 4 rows and then uses another helper class, with `CREATE FUNCTION other_f` (table) run 1.5 s after the query starts | `301: lazy.LateHelper` | all 10 rows | Without `tree_f`, or without the `CREATE FUNCTION` during the query, both builds return the right values. In the last case the `CREATE FUNCTION` returned about 2.7 s before the query finished, so it did not wait for the query. The first two rows are the case Jackie described: on master `doRegister` reloads only the classes of the model being registered, so `lazy_f` stays on a loader that is already closed. With this PR both models are reloaded. With the PR these worked too: `SELECT sum(lazy_f(v))`, `EXPLAIN ANALYZE`, `INSERT INTO db.t2 SELECT … lazy_f(v) …`, and `tree_f` in the tree model, including `ALIGN BY DEVICE`. **`UPDATE` still runs without a scope** ```sql UPDATE db.t SET attr1 = str_f(device_id) WHERE device_id = 'd2' ``` With this PR it fails with: ``` 305: internal error. statemachine throws a runtime exception: java.lang.IllegalStateException: Cannot pin UDF ClassLoader for class lazy.StrF because no UDF query scope is active. ``` The attribute update builds its transformers in `SchemaRegionMemoryImpl#constructDevicePredicateUpdater`, which runs from `SchemaRegionStateMachine#write`, so `reflect()` is called with no scope. This is the schema-region `UPDATE` path Jackie listed. On master the same statement fails too, for another reason (`IoTDBLocalFactory must not be null for UDF execution`), so it isn't a regression today, but with this PR the statement would still fail after that one is fixed. That other error isn't from this PR. A UDF on a TAG column in `WHERE` fails the same way on both builds: `SELECT count(*) FROM db.t WHERE str_f(device_id) = 'd1'`, and `SHOW DEVICES` / `COUNT DEVICES` with that condition, give `outerQueryDeadlineMs must be positive for UDF execution` or `IoTDBLocalFactory must not be null for UDF execution`, from `TableOperatorGenerator.IoTDBLocalFactory`. So I couldn't use those statements to check the scopes on the schema fragment path. I only ran a single DataNode, so the RPC and consensus paths aren't covered by this. The red CI jobs on this head look like infrastructure: the three failed IT jobs report that the self-hosted runner lost communication, and `Windows (1)` hit the 6-hour limit. `IoTDBSQLFunctionManagementIT` ran in one of them before that and passed all 11 tests. The jar and the script are small, so I can post them here if they would help. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
