aglinxinyuan opened a new pull request, #8640:
URL: https://github.com/apache/texera/pull/8640
### What changes were proposed in this PR?
Inside a control block (`Loop Start ... Loop End`), any primitive property
of an operator (string, integer, number, boolean) can now hold a loop-variable
reference such as `$K`. The frontend sends the literal string `"$K"` unchanged.
The backend marks it as a placeholder when it compiles the plan and binds it
when the iteration's state message reaches the worker (#8639). Operators
outside every block behave as before.
```
Loop Start (K = 2) -> Limit(limit: "$K") -> Loop End
^ panel offers $K; canvas valid; sent as "$K"
rename K to i in the Loop Start -> Limit flagged "$K is not a variable of
an enclosing block"
delete a link upstream -> Limit outside the block -> "must be
integer"
```
| Piece | What it does |
|---|---|
| Block membership (`getEnclosingLoopStarts`, on `WorkflowGraph`) | Finds
the Loop Starts whose block encloses an operator, outermost first. It walks
downstream to an unmatched Loop End and upstream to an unmatched Loop Start,
counting open blocks, so blocks in a row do not leak into each other and nested
blocks give several starts. The open count is capped, so the walk ends even on
a cycle through a control operator. |
| Loop variables (`extractLoopVariables`, `loopVariablesInScope`) | Reads
the variables from each enclosing Loop Start's `initialization` code, following
Python's module-level binding: assignments at any depth outside a `def`/`class`
body, for-loop targets, and one-line compound bodies such as `try: k = 1`. |
| Property panel | Inside a block, each plain primitive field becomes a text
box that offers `$name` for every variable in scope. Enums, arrays and custom
widgets keep their own controls. A `$name` that no enclosing block declares is
flagged. |
| Canvas validation | Inside a block, the schema `type` error is waived
exactly where a reference sits in a primitive-typed property. Every reference
must name a declared variable. Adding or removing any link, or editing any Loop
Start, re-validates every operator that holds a reference, so the Run button
follows edits two hops away. |
| Open panel | Rebuilt on those same edits when the operator's scope
changed. |
Panel behavior for a field inside a block:
| Field type | Typed text | Stored value |
|---|---|---|
| integer | `7` | `7` (number) |
| integer / number / boolean | `$K` | `"$K"` |
| number | `0.5`, typed one key at a time | `0.5`; partial text such as `7.`
stays text until complete |
| boolean | `true` / `FALSE` | `true` / `false`; other text is flagged |
| string | `hello` / `$K` | `"hello"` / `"$K"` |
| any | `$foo` (undeclared) | `"$foo"`, flagged; operator marked invalid |
**Merge order.** This PR depends on #8639 and must merge after it. Until
then, a `"$K"` in a non-string property fails deserialization when the workflow
is compiled.
**Scope limits and follow-ups**
- Operators whose code is generated before the loop runs (Python UDFs and
`PythonOperatorDescriptor` subclasses such as the sklearn operators) accept
`$K` in the panel. The backend reports a compile error for them, since nothing
can bind their values yet.
- An operator with an input fed from outside the block (for example a join
whose other side comes from outside) is inside the block for the panel, so the
panel offers `$K`. The backend reports a compile error for it, because that
input's tuples could arrive before the loop state.
- The Form View builds its fields with its own formly mapping and does not
offer `$K` yet.
- Inside a block, a boolean property renders as a text box (`true` / `false`
/ `$name`) rather than a checkbox.
- Inside a block, a whole-string `$name` in a string property is always a
reference. There is no escape for a literal `$name` there yet.
**Screenshots.** Before/after screenshots of the property panel inside a
block are still to be attached: the workspace requires signing in, which was
not possible from the authoring environment.
### Any related issues, documentation, discussions?
Closes #8636
### How was this PR tested?
New and updated Vitest specs, run from `frontend/`:
| Spec | Covers |
|---|---|
| `loop-block.util.spec.ts` | Straight, branching, nested, sequential and
incomplete blocks; a cycle in the body; three cycles through a control
operator, which exhausted the heap before the cap. |
| `loop-variable.util.spec.ts` | Assignment forms; if/else, try/except,
while and for bodies; for targets; one-line compound bodies; def/class bodies
and with/import/except-as names excluded; Python keywords never taken for
names. |
| `loop-variable-field.util.spec.ts` | Complete vs partial numerals (`7.`,
`-0.`, `1e-` kept as text); the parser sets the control without redrawing the
box; the string `""` default; the boolean validator rejects `maybe`. |
| `loop-variable-input.component.spec.ts` (new) | Rendered through a real
formly form: typing `1.05`, `-0.5` and `7.5` one key at a time stores the right
number and keeps the box as typed; `$K` and integers; formly attributes reach
the input. |
| `validation-workflow.service.spec.ts` | The waiver applies only to
primitive types. Canvas state follows graph edits: a Loop Start variable
renamed and restored, an upstream link deleted, a block closed by a link
between two other operators, an operator inserted into the body. |
| `operator-property-edit-frame.component.spec.ts` | Primitive fields
converted inside a block; file picker, enum and array keep their controls; the
form rebuilds when a link or the Loop Start's variables change its scope, and
only then. |
Each new behavior spec was checked red against the pre-fix code before the
final run.
```
yarn ng test --watch=false --include=<the 6 loop specs> #
185 passed
yarn ng test --watch=false #
233 files, 6300 passed, 1 skipped (existing it.skip)
yarn exec tsc --noEmit -p tsconfig.json (and src/tsconfig.spec.json) #
clean
yarn exec ngc -p src/tsconfig.app.json --noEmit #
AOT + strictTemplates clean
yarn format:fix && yarn eslint <changed files> #
clean
```
`yarn ng build` did not complete locally: the worktree's `node_modules` is a
symlink, so the custom webpack rules for monaco assets never match. The
template compile above is clean, and CI's frontend build covers the bundle.
Manual steps for the reviewer (Chrome, desktop): link `Loop Start -> Limit
-> Loop End` and set the Loop Start initialization to `K = 2`. Open `Limit`:
its integer field is a text box, typing `$` offers `$K`, and choosing it keeps
the operator valid. Typing `$foo` shows "$foo is not a variable of an enclosing
block". Delete a link so `Limit` leaves the block: the field is a number box
again and `$K` is a type error.
### Was this PR authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Claude Fable 5.1, Claude Opus 5.5)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
