KyleKDang opened a new issue, #5127:
URL: https://github.com/apache/texera/issues/5127

   ### Feature Summary
   
   Currently, users cannot view the Python code that corresponds to workflow 
operators. Many users would find it convenient to access the underlying Python 
code for their workflows.
   
   ### Proposed Solution or Design
   
   Add a WorkflowToPythonTranslator component within the existing Amber engine 
that converts a workflow's logical plan into a standalone Python script.
   How it works:
   
   The translator takes a workflow's logical plan (the DAG of operators) and 
iterates through the operators in topological order.
   Each operator descriptor (e.g., CSVScanSourceOpDesc) implements a 
StandaloneCodeGenerator trait, which provides a generateStandaloneCode() method 
that returns the equivalent Python code for that operator.
   The translator calls substituteVars() to replace placeholder variable names 
(like in1df/out1df) with the actual assigned variable names in the generated 
script.
   Leaf operators that produce DataFrames are printed at the end, so the script 
gives visible output when run locally.
   
   Architecture Diagram:
   <img width="2024" height="1252" alt="Image" 
src="https://github.com/user-attachments/assets/ac3188e7-d847-42fe-a6a4-b3c17c085f59";
 />
   
   BarChart Demo:
   Workflow with a CSV scan, filter, sort, and bar chart. Shows the workflow 
result in Texera, exports to Python, and runs the generated script to produce a 
matching bar chart.
   
https://github.com/user-attachments/assets/70cadd54-5d0e-4cad-b957-76aab2abee0e
   
   DataFrame Demo:
   
https://github.com/user-attachments/assets/0e546c82-b958-40af-920f-893a2edb6829
   
   ### Affected Area
   
   Workflow Engine (Amber)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to