GitHub user kz930 added a comment to the discussion: Scaling verification of
workflow to Python translation — looking for input
@chenlica Following our meeting, here is the concrete per-operator change,
using the two simple cases we discussed.
For each operator, making it verifiable should be a small and uniform change:
1. Implement `generateStandaloneCode()` and `producesDataFrame()` for the
operator’s Python translation.
2. Let auto-config fill the operator fields using the shared fixture.
Type-matching should cover most fields; when type alone is not enough, add a
one-line `@SampleColumn` hint.
3. Once the operator implements `generateStandaloneCode()`,
`OperatorBehaviorSpec` auto-discovers it and compares the native Texera output
with the generated Python output in the correct bucket: table, Plotly JSON, or
HTML. No per-operator test needs to be written.
Two simple examples:
`CSVScan` source/table operator:
* `generateStandaloneCode()` emits `pd.read_csv(...)`.
* No annotation is needed.
* It is verified with `assert_frame_equal` between the native output and the
generated Python output.
`ChoroplethMap` visualization operator:
* `generateStandaloneCode()` emits `px.choropleth(...)` and writes
`output.json`.
* `locations` needs `@SampleColumn("iso_country")`, because a generic text
column may render nothing.
* `color` can be filled by type.
* It is verified through Plotly JSON comparison.
Plan: first finish and verify the existing operators one by one with this plan.
Then, for future operators: shared fixture by default; when it can't represent
the operator's input well, the author provides a small dataset for a more
accurate test.
GitHub link:
https://github.com/apache/texera/discussions/6072#discussioncomment-17565113
----
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to: [email protected]
