PDGGK commented on PR #19107:
URL: https://github.com/apache/iotdb/pull/19107#issuecomment-6717101251

   Thanks for working on this. I built 3f6dd87 and today's master (d8800f1) and 
ran each as a standalone server (one ConfigNode, one DataNode). The functions 
come from a small jar. `lazy_f` and the UDF in the slow query below call a 
helper class while evaluating rows, so the JVM loads that class only when it is 
first used.
   
   **Before and after**
   
   | steps | master | this PR |
   |---|---|---|
   | `CREATE FUNCTION lazy_f` (table), then `CREATE FUNCTION tree_f` (tree, 
same jar), then `SELECT lazy_f(v) FROM db.t` | `301: lazy.Helper` | correct 
values |
   | the same two functions, a restart, then the same `SELECT` | `301: 
lazy.Helper` | correct values |
   | a `SELECT` whose UDF sleeps 1 s on each of its first 4 rows and then uses 
another helper class, with `CREATE FUNCTION other_f` (table) run 1.5 s after 
the query starts | `301: lazy.LateHelper` | all 10 rows |
   
   Without `tree_f`, or without the `CREATE FUNCTION` during the query, both 
builds return the right values. In the last case the `CREATE FUNCTION` returned 
about 2.7 s before the query finished, so it did not wait for the query.
   
   The first two rows are the case Jackie described: on master `doRegister` 
reloads only the classes of the model being registered, so `lazy_f` stays on a 
loader that is already closed. With this PR both models are reloaded.
   
   With the PR these worked too: `SELECT sum(lazy_f(v))`, `EXPLAIN ANALYZE`, 
`INSERT INTO db.t2 SELECT … lazy_f(v) …`, and `tree_f` in the tree model, 
including `ALIGN BY DEVICE`.
   
   **`UPDATE` still runs without a scope**
   
   ```sql
   UPDATE db.t SET attr1 = str_f(device_id) WHERE device_id = 'd2'
   ```
   
   With this PR it fails with:
   
   ```
   305: internal error. statemachine throws a runtime exception: 
java.lang.IllegalStateException: Cannot pin UDF ClassLoader for class lazy.StrF 
because no UDF query scope is active.
   ```
   
   The attribute update builds its transformers in 
`SchemaRegionMemoryImpl#constructDevicePredicateUpdater`, which runs from 
`SchemaRegionStateMachine#write`, so `reflect()` is called with no scope. This 
is the schema-region `UPDATE` path Jackie listed. On master the same statement 
fails too, for another reason (`IoTDBLocalFactory must not be null for UDF 
execution`), so it isn't a regression today, but with this PR the statement 
would still fail after that one is fixed.
   
   That other error isn't from this PR. A UDF on a TAG column in `WHERE` fails 
the same way on both builds: `SELECT count(*) FROM db.t WHERE str_f(device_id) 
= 'd1'`, and `SHOW DEVICES` / `COUNT DEVICES` with that condition, give 
`outerQueryDeadlineMs must be positive for UDF execution` or `IoTDBLocalFactory 
must not be null for UDF execution`, from 
`TableOperatorGenerator.IoTDBLocalFactory`. So I couldn't use those statements 
to check the scopes on the schema fragment path.
   
   I only ran a single DataNode, so the RPC and consensus paths aren't covered 
by this. The red CI jobs on this head look like infrastructure: the three 
failed IT jobs report that the self-hosted runner lost communication, and 
`Windows (1)` hit the 6-hour limit. `IoTDBSQLFunctionManagementIT` ran in one 
of them before that and passed all 11 tests.
   
   The jar and the script are small, so I can post them here if they would help.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to