GitHub user Xiao-zhen-Liu edited a comment on the discussion: Design and merge 
plan: operator output port result cache (MVP)

@chenlica @Yicong-Huang Following our October 1 discussion (where #8731 to 
#8734 were put on hold to examine @Yicong-Huang's proposal and how the design 
extends past full reuse), here is a summary of what we found and the two 
choices to settle.

### 1. Any reuse method needs a workflow-wide backward pass

At the October 1 meeting, @Yicong-Huang asked whether reuse can be decided 
inside the cost estimator and resource allocator by assigning 0 workers to 
operators with saved results, without a separate pass before scheduling 
(#8731). When `B` has a saved result and upstream `A` does not, checking per 
operator wastes compute on `A` (left), while propagating only within one region 
`R1` skips `A` even when a second region `R2` still needs `A`'s output (right):

![Per-operator check causes wasted work; per-region pass leaves downstream E 
unable to 
run](https://gist.githubusercontent.com/Xiao-zhen-Liu/0ab5fc9832b2688c2fa9aa46ec9f8f4a/raw/a269f32831b1723ae2f536d1a45a85967baa5ab0/fig01-global-pass.png)

### 2. Where that global pass lives, and why it separates cleanly from Ramen 
(#3605)

In `CostBasedScheduleGenerator`, regions are formed across the whole workflow 
first, and `CostEstimator` and `ResourceAllocator` (including planned Ramen, 
#3605) are called afterward on **one region at a time**. Running the backward 
pass before region formation lets Ramen allocate normal running regions with no 
changes to Ramen:

![Workflow-wide propagation before createRegions vs. per-region CostEstimator 
and Ramen 
(#3605)](https://gist.githubusercontent.com/Xiao-zhen-Liu/0ab5fc9832b2688c2fa9aa46ec9f8f4a/raw/a269f32831b1723ae2f536d1a45a85967baa5ab0/fig02-pipeline-and-ramen.png)

### 3. Why skipped operators form Level 0 skip regions instead of staying in 
running regions

Even after a workflow-wide pass decides what skips, keeping 0-worker `B` inside 
a running region creates a false 60 s completion barrier (and breaks 
multi-output `Split` when only one port is saved, #8731, #8732). Putting 
skipped operators in Level 0 skip regions (#8733) lets `A` and `C-Build` run in 
parallel and unblocks downstream regions as soon as `C-Build` finishes:

![0-worker B inside R1 creates a 60 s false barrier vs. Level 0 skip regions 
running R3 and R4 in 
parallel](https://gist.githubusercontent.com/Xiao-zhen-Liu/0ab5fc9832b2688c2fa9aa46ec9f8f4a/raw/a269f32831b1723ae2f536d1a45a85967baa5ab0/fig03-skip-regions.png)

### 4. Extending beyond Eager Reuse: Two-Layer Search vs. One-Layer Search

When a downstream saved result (`D`) sits on slow storage while an upstream 
saved result (`B`) sits on fast storage, recomputing `C -> D` and pipelining 
can beat Eager Reuse. Both **Two-Layer Search** (wrapping `main`'s 2-state 
scheduler) and **One-Layer Search** (replacing it with a 4-state link queue) 
build on **#8731 to #8734 unchanged**:

![Comparison of Option A (Two-Layer Search) and Option B (One-Layer Search) on 
a chain with fast and slow saved 
results](https://gist.githubusercontent.com/Xiao-zhen-Liu/0ab5fc9832b2688c2fa9aa46ec9f8f4a/raw/a269f32831b1723ae2f536d1a45a85967baa5ab0/fig04-two-layer-vs-one-layer.png)

### 5. Two framework choices to settle

Because open PRs **#8731 to #8734** (along with the execution, 
runtime-statistics filter, storage, and editor UI PRs) are shared unchanged 
across every option, aligning on two choices unblocks the merge plan:

1. **Rollout scope:** merge the **Two-Phase Eager Reuse MVP first** and add 
cost-based search next, **or** land **cost-based reuse search from the start** 
before enabling cache reuse.
2. **Search architecture:** **Option A (Two-Layer Search)** wrapping `main`'s 
unchanged 2-state `CostBasedScheduleGenerator`, **or** **Option B (One-Layer 
Search)** replacing the 2-state loop with the 4-state link queue.

![Shared #8731-#8734 foundation and the two framework decisions to 
settle](https://gist.githubusercontent.com/Xiao-zhen-Liu/0ab5fc9832b2688c2fa9aa46ec9f8f4a/raw/a269f32831b1723ae2f536d1a45a85967baa5ab0/fig05-decisions.png)

<details>
<summary>Additional October 1 details (intact PhysicalPlan, no SKIPPED state, 
runtime-statistics history filter, planned Peanut)</summary>

| Question from October 1 | How the design handles it |
| --- | --- |
| Keep `PhysicalPlan` intact for downstream lookups | Scheduler keeps full 
`PhysicalPlan`; only retained operators enter region search (#8731, #8733). |
| No new `SKIPPED` operator or region state | Reused operators finish as 
`COMPLETED` with `reused_from_cache = true` (#6729) and saved row counts from 
`operator_port_cache` (#5967). |
| Do not skew `CostEstimator` / Ramen runtime history | Persist 
`reused_from_cache` in runtime statistics and ignore `reused_from_cache = true` 
rows so 0-worker, 0.0 s runs never overwrite compute history. |
| Compatibility with planned Peanut (`U_train -> U_predict`) | Planner and 
search operate on the expanded `PhysicalPlan`, so mini-operators from planned 
Peanut are scheduled and cached like any other operators. |

</details>


GitHub link: 
https://github.com/apache/texera/discussions/5880#discussioncomment-19505621

----
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to: [email protected]

Reply via email to