GitHub user Yicong-Huang added a comment to the discussion: RFC: Workflow Performance Profiler - full design & implementation walkthrough
I personally very much like this feature, I view this feature as three folds: 1. displays the power of our engine: very interactive, can quickly gather metadata during execution. super powerful 2. utilizes our workflow UI, to display/embed information on this UI. 3. agent suggestions to apply/fix/help user based on the profile results. So it is a very good unification of interactive engine + workflow UI + agent. I think your draft architecture makes sense. very good start point! to move forward, we need smaller steps. we may not need all the features, though. so will do the cherry pick carefully. A few comments: 1. I would not call it profile, at least not towards user. I think we want it to be low-tech user facing, and they may not understand profile or profiling. I still prefer to keep it as "workflow status". to the end user, it is "what's going on with my workflow?". Currently I think we have "state" and "statistics" two concepts inside status (please correct me if I'm wrong, or we might need to do some refactor). Can you extend a concept of "profile" inside the "WorkflowStatusService"? The WorkflowStatusService should become layer that contains the ground truth information. 2. it might be good to have different overlay layers, for instance you can introduce "heat map" layer. see other current workflow overlay layers, user can choose to toggle on/off any of those: <img width="306" height="518" alt="CleanShot 2026-05-25 at 22 28 57@2x" src="https://github.com/user-attachments/assets/3175d5b1-7852-408f-b116-c421b0ca30a2"; /> The layer will just display the corresponding information from `WorkflowStatusService`. for your suggest plan, I would not considered it in term of PRs, but in terms of issues. each issue is a todo task. and each issue could correspond to multiple PRs (feature, revert, fix, reapply, doc, etc.) So high levelly I suggest we open an umbrella issue (don't name it profile/profiler though), and go with those three sub-issues first. 1. refactor existing service (e.g., WorkflowStatusService) to capture/store the information on the frontend which you need to display. this is purely information level, no UI change yet. 2. start to add one overlay layer. could be "heat map": which operator is the bottleneck, etc. this is UI visual part. 3. make sure any information displayed on the frontend should be able to be reconstructed from the backend. meaning that if user refreshes the page, he/she will see the same information. we can come back to this discussion after you finish the first three. WDYT? GitHub link: https://github.com/apache/texera/discussions/5216#discussioncomment-17057577 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
