GitHub user CaroFernando created a discussion: Review mode for the agent: accept, reject, or request changes before edits are kept
## Summary Today the agent applies every workflow edit as soon as it makes it. `addOperator`, `modifyOperator` and `deleteOperator` change the workflow mid-run, the change is saved, and it appears on the canvas. The user can only react after the whole run, when several edits may have stacked up. In #8682 the reporter asked for a way to review changes first, because "sometimes it edits to be worse". I'd like to add an optional **review mode**. The agent runs as it does today. When the run ends, the chat shows the before → after changes to the workflow, with three choices: **Accept all**, **Reject all** and **Request changes**. This is a design proposal, so there is no PR yet. I'd like agreement on the behavior before writing code. ## Today A real run on a local build (fork main, model `anthropic/claude-haiku-5.5`), prompt: *"Build a small workflow: a CSV File Scan, then a Filter keeping rows where age > 30, then a Bar Chart of count per city."* | Step | Tool | Effect | |---|---|---| | 1 | `addOperator` | CSV File Scan applied to the canvas and saved | | 2 | `addOperator` | Filter `age > 30` applied and saved | | 3 | `addOperator` | Aggregate (count per city) applied and saved | | 4 | `addOperator` | Bar Chart applied and saved | | 5–6 | `executeOperator` | Both runs failed (no warehouse / file selected) | | 7 | — | Final answer asking for the CSV path | All four edits landed before the user saw anything, and they stayed even though the run ended without a working result. There was no point at which the user could say "not like this". <img width="1693" height="939" alt="image" src="https://github.com/user-attachments/assets/50aec39a-0ae6-4e26-a299-dd821e717ba8"; /> ## Proposal <img width="1400" height="820" alt="review-pending" src="https://github.com/user-attachments/assets/491d7fb3-dac5-4df0-88e0-78e01d225350"; /> *Mockup. After the run, the chat shows one review card with every change since the last accepted workflow. Changed operators are highlighted on the canvas, which is locked until the user decides.* ### Behavior 1. **A per-agent setting with two values: `auto` and `review`.** `auto` is today's behavior and stays the default, so nothing changes for existing users. 2. **Review once per run, not per edit.** The agent loop is untouched. In the run above, approving each of the four `addOperator` calls separately, without seeing the next one, would be harder than judging the finished pipeline once. 3. **All or nothing.** Edits depend on each other (add A, link B→A, modify A), so rejecting one of them can leave a broken workflow. There is no per-edit accept. 4. **Request changes is a normal chat message** sent while a review is pending. The agent continues from its pending edits, and the card always shows the change from the **last accepted** workflow, not just the latest run. 5. **The canvas is locked while a review is pending**, so the user's edits and the agent's can't mix. It unlocks on accept or reject. 6. **Stop and errors also end in a review** if anything changed. The run above is a good example: it failed, but its four edits would still come up for review, and Reject all would undo them. A run that only read or executed shows no card. ### Request changes <img width="1400" height="820" alt="review-changes" src="https://github.com/user-attachments/assets/925665e2-8281-482b-972a-34e51a762ef6"; /> *Mockup. The user asked for a pie chart instead. The agent revised its pending edits, and the card still shows the full change from the original (empty) workflow: 4 added, with the chart now a Pie Chart.* ### Setting <img width="1400" height="820" alt="review-settings" src="https://github.com/user-attachments/assets/36ad0002-6272-4bc7-bf2c-c11032f4a2b8"; /> *Mockup. A switch in the existing agent settings modal. Off means today's behavior.* ### Lifecycle ```mermaid stateDiagram-v2 [*] --> Idle Idle --> Running: prompt (baseline = current workflow) Running --> Idle: run ends, no changes Running --> ReviewPending: run ends / stopped / error, with changes ReviewPending --> Running: Request changes (same baseline) ReviewPending --> Idle: Accept all (keep edits) ReviewPending --> Idle: Reject all (restore baseline) ``` ## Implementation sketch - **agent-service:** before the first run of a review cycle, the agent keeps a copy of the workflow (the "baseline"). After the run, if the workflow differs from the baseline, the review is pending. Accept drops the baseline. Reject restores and saves it. The tools and prompts do not change. - **WebSocket:** one new server event for "review pending / resolved", and two client commands, accept and reject. Request changes reuses the existing prompt command. - **Frontend:** the settings switch, the review card, the canvas lock and highlight, and a readable diff of operators and properties instead of raw JSON. Raw JSON in the chat is what caused the confusion in #8682. I have a 4-PR split planned (backend state → WS frames → UI card → diff view). Each PR keeps CI green, and the first two are invisible until the UI lands. ## Alternatives considered - **Approve each edit as it happens.** It lets the user stop a bad edit before others build on it. But the tool has to block on the user, which has to work with stop, timeouts and reconnects, and approving 3 to 6 prompts per run is tedious. - **Hold edits back until accept** (keep the canvas unchanged during the run). It's safer for saved data, but the user can't see what the agent is doing, and both autosave paths would need to be paused. ## Related work #6039 proposes **Revert / Redo per agent turn**: undo an agent turn *after* its edits were kept. This proposal is the other half: review the edits *before* they are kept. They fit together. "Reject all" here restores the same "workflow before the turn" snapshot that a revert would, so both can share one restore mechanism in agent-service. As far as I can tell, #6039 hasn't landed yet. @bobbai00 @Ma77Ball, since you weighed in there, I'd value your view on whether these should be designed together. ## Open questions 1. **After-run review or per-edit approval?** I lean toward after-run. 2. **Saving during review.** In my proposal, pending edits show on the canvas and are saved like today, and Reject restores the previous version. Would maintainers prefer pending edits not to be saved until accepted? A related point: the baseline lives in agent-service memory, so a restart during a review keeps the edits. #5267 (persisting agent state) would fix that later. 3. **Where the setting lives.** Per agent (my proposal), per user, or global? 4. **Conflicts with planned work.** Does this overlap the Revert/Redo work from #6039, or anything planned for agent-service (#5747 refactor) or the agent panel? Related: #8682 (origin), #6039 (revert/redo agent turns), #5267 (agent state persistence), #5747 (agent-service refactor) GitHub link: https://github.com/apache/texera/discussions/8931 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
