GitHub user eugenegujing created a discussion: Proposal: Making the workspace 
agent useful to a new user by seeing run errors, reading the docs, knowing the 
datasets

The agent can build and run workflows, but it cannot see the errors from a run 
the user started, cannot read Texera's own documentation, and does not know 
which datasets the user has. I want to propose three small, independent 
changes, each tracked as its own issue and PR. The first two are ready to 
start, and the third is a sketch I would like feedback on first. This follows 
up on #9246.

## 1. Let the agent see the errors from the user's own run

Today the agent only sees an error when it ran the operator itself through its 
`executeOperator` tool. When the user clicks Run and the workflow fails, the 
agent has no idea. So "why did this fail?" is the one question a new user most 
wants to ask it, and the one it cannot answer. The system prompt already tells 
the model how to fix a failing operator in place. What is missing is the input.

The change is on the frontend side. When the user sends a message, the frontend 
attaches the outcome of the latest run, which means the fatal errors and the 
last error-level console message of each operator, with stack traces truncated. 
The agent service then shows them under the matching operator in the dataflow 
section of the context, the same way compilation errors already appear there. A 
short prompt rule asks the model to explain the error in plain language, name 
the operator, suggest the fix, and offer to apply it before editing anything.

Why not read the errors from the backend? Fatal errors are not persisted 
anywhere. They live in the computing unit's in-memory state and reach the 
browser over the websocket. The frontend already holds exactly this data, and 
the AI Fix tab from #8637 reads it. Attaching it to the prompt needs no backend 
or database change. If fatal errors get persisted later, the source can switch 
to a REST call without changing the agent side.

This touches the frontend agent service, the agent-service types and context 
serialization, one prompt rule, and tests. It edits the same file as #9244, so 
it will land after that.

## 2. Give the agent the documentation

The system prompt tells the model to read documentation first to understand 
exact definitions, but there is no documentation tool. The model is told to do 
something it cannot do. Meanwhile, most of the questions @zyratlo's researcher 
asked were "how do I do X in the UI", and the answers exist in `docs/` and on 
texera.apache.org.

The change is to bundle `docs/` into the agent-service image at build time and 
build an in-memory keyword index over the pages on startup, split by section. 
Two read-only tools go on top of it. `searchDocs(query)` returns the best 
matching sections with a link to the published page, and `readDoc(path, 
section?)` returns the text. The prompt rule above becomes concrete. For 
questions about how Texera works, search the docs, read the page, answer with a 
link, and do not edit the workflow.

I want to keep this small. No embeddings, no model download, no new dependency. 
A plain keyword index over about 500 KB of markdown is enough. Docs are bundled 
rather than fetched at runtime so the answers match the deployed version and 
work offline. When #9031 regenerates the operator reference pages, the index 
picks them up with no change.

One side effect is useful on its own. Running a set of new-user questions 
against the index also shows which questions have no page at all, and that list 
can go to #8773.

This is one new tool file, one prompt change, one `COPY docs` line in the 
Dockerfile, a `TEXERA_DOCS_DIR` override for local runs, and tests. #4932 is 
related, since llms.txt could be generated from the same index later.

## 3. A sketch for letting the agent see the user's datasets

The agent does not know which datasets the user has or what a dataset file path 
looks like, so it guesses paths and gets them wrong (#4588).

The idea is two read-only tools backed by file-service endpoints that already 
exist, `listDatasets()` and `listDatasetFiles(dataset, version?)`. The second 
returns each file's full path in the form the file scan operators accept, so 
the agent can fill it in directly. The tools call file-service with the user's 
own delegated token, so the user's access rules are enforced where they already 
are. This is discovery only. Reading a dataset from inside a Python UDF is the 
mount feature (#6866, #6896, #6897) and the open questions in #9048.

There is prior work here. @aicam prototyped dataset tools (list, get, versions, 
and more) as part of a standalone MCP server on a fork branch in July, 
following the discussion in #5610, where the conclusion was that Texera's own 
actions should stay inline tools. The shape above is the inline version of that 
prototype's read-only subset. #5918 raises how the agent should call backend 
endpoints without duplicating the REST contract by hand, and I would rather 
follow whatever comes out of that than add another hand-written client.

I am not proposing to start this yet. The agent service does not bind an agent 
to its owner (#5268) and is getting request authentication in #9043. A tool 
that lists a user's datasets should land after those, so I would rather agree 
on the shape now and open the issue once #9043 is in.

## What I am asking

- Any suggestions on the three changes, or other ideas for what the agent 
should be able to see or do for a new user?
- For 2, bundling docs into the image versus fetching from the website at 
runtime. I prefer bundling for the reasons above.
- For 3, any objection to the shape, or to waiting on #9043?

I will open one issue per change and link the PRs here.


GitHub link: https://github.com/apache/texera/discussions/9377

----
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to: [email protected]

Reply via email to