aglinxinyuan opened a new issue, #8635: URL: https://github.com/apache/texera/issues/8635
### Problem Inside a control block (LoopStart/LoopEnd), the iteration state — the loop's variables such as `K` — reaches every operator as a state message, but no built-in operator can *use* it. An operator's properties are read once when its executor is initialized, before any message has arrived, and the executor is built from a frozen descriptor JSON with no substitution stage. Today the only operator that reads a state key is `If`; a `Filter` or an sklearn operator inside a loop cannot take its threshold or cluster count from the loop. ### Proposal Let a property of any operator inside a block refer to a loop variable as `$K`, with no change to the operator's source: 1. **Compile phase (unchanged wire format).** The frontend sends `$K` as the property value. When the logical plan is parsed, a rewriter records each `$name` reference on the operator (`stateReferences`: JSON pointer → variable name) and, for non-string fields, replaces the value with a typed placeholder so the descriptor still deserializes. 2. **Shared registration.** When a state message arrives, the worker registers the state's key–value map in the operator base (`OperatorExecutor`) before calling `processState` — the same for every operator, UDF or built-in. 3. **Late binding.** An executor whose descriptor carries `stateReferences` is created as a late-bound wrapper: it starts with the placeholders unbound, binds them from the registered state at the first tuple (patching the descriptor and instantiating the real executor), and reuses the bound values for every later tuple. A tuple that arrives before the state, or a name missing from the state, is an error. The Python operator base gets the same registration hook for symmetry; UDF-side sugar and the frontend are separate issues. ### Acceptance - A `Filter` inside a loop with predicate value `$i` filters with the current iteration's `i`, verified end to end in `LoopIntegrationSpec`. - Unit tests for the plan rewriter (string, integer and nested-pointer references) and for the late-bound executor (bind at first tuple, reuse, error when unbound / unknown name). - No per-operator code changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
