jens-scheffler-bosch opened a new issue, #26215:
URL: https://github.com/apache/airflow/issues/26215
### Description
Proposal for Contribution for an extensible Trigger UI feature in Airflow.
## Design proposal (Feedback welcome)
### Part 1) Specifying Trigger UI on DAG Level
We propose to extend the DAG class with an additional attribute so that
UI(s) (one or multiple per DAG) can be specified in the DAG.
* Attribute name proposal: `trigger_ui`
* Type proposal: `Union[TriggerUIBase, List[TriggerUIBase]` (One or a list
of UI definition inherited from an abstract UI class which implements the
trigger UI)
* Default value proposal: `[TriggerNoUI(), TriggerJsonUI()]` (Means the
current/today's state, user can pick to trigger with or without parameters)
With this extension the current behavior is continued and users can specify
if a specific or multiple UIs are offered for the Trigger DAG option.
### Part 2) UI Changes for Trigger Button
The function of the trigger DAG button in DAG overview landing ("Home" /
`templates/airflow/dags.html`) as well as DAG detail pages (grid, graph, ...
view / `templates/airflow/dag.html`) is adjusted so that:
1) If there is a single Trigger UI specified for the DAG, the button
directly opens the form on click
2) If a list of Trigger UIs is defined for the DAG, then a list of UI's is
presented, similar like today's drop-down with the today's two options (with
and without parameters).
Menu names for (2) and URLs are determined by the UI class members linked to
the DAG.
### Part 3) Standard implementations for TriggerNoUI, TriggerJsonUI
Two implementations for triggering w/o UI and parameters and the current
JSON entry form will be migrated to the new UI structure, so that users can
define that one, the other or both can be used for DAGs.
Name proposals:
0) TriggerUIBase: Base class for any Trigger UI, defines the base parameters
and defaults which every Trigger UI is expected to provide:
* `url_template`: URL template (into which the DAG name is inserted to
direct the user to)
* `name`: Name of the trigger UI to display in the drop-down
* `description`: Optional descriptive test to supply as
hover-over/tool-tip)
1) TriggerNoUI (inherits TriggerUIBase): Skips a user confirmation and entry
form and upon call runs the DAG w/o parameters (`DagRun.conf = {}`)
2) TriggerJsonUI (inherits TriggerUIBase): Same like the current UI to enter
a JSON into a text box and trigger the DAG. Any valid JSON accepted.
### Part 4) Standard Implementation for Simple Forms (Actually core new
feature)
Implement/Contribute a user-definable key/value entry form named
`TriggerFormUI` (inherits TriggerUIBase) which allows the user to easily enter
parameters for triggering a DAG. Form could look like:
```
Parameter 1: <HTML input box for entering a value>
(Optional Description and hints)
Parameter 2: <HTML Select box of options>
(Optional Description and hints)
Parameter 3: <HTML Checkbox on/off>
(Optional Description and hints)
<Trigger DAG Button>
```
The resulting JSON would use the parameter keys and values and render the
following `DagRun.conf` and trigger the DAG:
```
{
"parameter_1": "user input",
"parameter_2": "user selection",
"parameter_3": true/false value
}
```
The number of form values, parameter names, parameter types, options, order
and descriptions should be freely configurable in the DAG definition.
The trigger form should provide the following general parameters (at least):
* `name`: The name of the form to be used in pick lists and in the headline
* `description`: Descriptive test which is printed in hover over of menus
and which will be rendered as description between headline and form start
* (Implicitly the DAG to which the form is linked to which will be triggered)
The trigger form elements (list of elements can be picked freely):
* General options of each form element (Base class `TriggerFormUIElement`:
* `name` (str): Name of the parameter, used as technical key in the JSON,
must be unique per form (e.g. "param1")
* `display` (str): Label which is displayed on left side of entry field
(e.g. "Parameter 1")
* `help` (Optional[str]=Null): Descriptive help text which is optionally
rendered below the form element, might contain HTML formatting code
* `required` (Optional[bool]=False): Flag if the user is required to
enter/pick a value before submission is possible
* `default` (Optional[str]=Null): Default value to present when the user
opens the form
* Element types provided in the base implementation
* `TriggerFormUIString` (inherits `TriggerFormUIElement`): Provides a
simple HTML string input box.
* `TriggerFormUISelect` (inherits `TriggerFormUIElement`): Provides a HTML
select box with a list of pre-defined string options. Options are provided
static as array of strings.
* `TriggerFormUIArray` (inherits `TriggerFormUIElement`): Provides a
simple HTML text area allowing to enter multiple lines of text. Each line
entered will be converted to a string and the strings will be used as value
array.
* `TriggerFormUICheckbox` (inherits `TriggerFormUIElement`): Provides a
HTML Checkbox to select on/off, will be converted to true/false as value
* Other element types (optionally, might be added later?) for making futher
cool features - depending on how much energy is left
* `TriggerFormUIHelp` (inherits `TriggerFormUIElement`): Provides no
actual parameter value but allows to add a HTML block of help
* `TriggerFormUIBreak` (inherits `TriggerFormUIElement`): Provides no
actual parameter value but adds a horizontal splitter
* Adding the options to validate string values e.g. with a RegEx
* Allowing to provide int values (besides just strings)
* Allowing to have an "advanced" section for more options which the user
might not need in all cases
* Allowing to view the generated `DagRun.conf` so that a user can
copy/paste as well
* Allowing to user extend the form elements...
### Part 5) (Optional) Extended for Templated Form based on the Simple form
but uses fields to run a template through Jinja
Implement (optionally, might be future extension as well?) a
`TriggerTemplateFormUI` (inherits TriggerFormUI) which adds a Jinja2 JSON
template which will be templated with the collected form fields so that more
complex `DagRun.conf` parameter structures can be created on top of just
key/value
### Part 6) Examples
Provide 1-2 example DAGs which show how the trigger forms can be used.
Adjust existing examples as needed.
### Part 7) Documentation
Provide needed documentation to describe the feature and options. This would
include an description how to add custom forms above the standards via Airflow
Plugins and custom Python code.
### Use case/motivation
As user of Airflow for our custom workflows we often use `DagRun.conf`
attributes to control content and flow. Current UI allows (only) to launch via
REST API with given parameters or using a JSON structure in the UI to trigger
with parameters. This is technically feasible but not user friendly. A user
needs to model, check and understand the JSON and enter parameters manually
without the option to validate before trigger.
Similar like Jenkins or Github/Azure pipelines we desire an UI option to
trigger with a UI and specifying parameters. We'd like to have a similar
capability in Airflow.
Current workarounds used in multiple places are:
1) Implementing a custom (additional) Web UI which implements the required
forms outside/on top of Airflow. This UI accepts user input and in the back-end
triggers Airflow via REST API. This is flexible but replicates the efforts for
operation, deployment, release as well and redundantly need to implement access
control, logging etc.
2) Implementing an custom Airflow Plugin which hosts additional
launch/trigger UIs inside Airflow. We are using this but it is actually a bit
redundant to other trigger options and is only 50% user friendly
I/we propose this as a feature and would like to contribute this with a
following PR - would this be supported if we contribute this feature to be
merged?
### Related issues
Note: This proposal is similar and/or related to #11054 but a bit more
detailed and concrete. Might be also related to #22408 and contribute to AIP-38
(https://github.com/apache/airflow/projects/9)?
### Are you willing to submit a PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
