narrowizard opened a new issue, #8408: URL: https://github.com/apache/incubator-devlake/issues/8408
### Search before asking - [x] I had searched in the [issues](https://github.com/apache/incubator-devlake/issues?q=is%3Aissue) and found no similar feature requirement. ### Use case DevLake currently provides a valuable notification mechanism that alerts users or systems when a pipeline has completed (either successfully or with an error). This is essential for understanding the final outcome of data collection and transformation jobs. However, DevLake users need visibility into the pipeline's overall status *before* it completes. Knowing when a pipeline is successfully *created* and, more importantly, when it actually starts *running* provides crucial real-time feedback. This is particularly important for: 1. Confirming that a manually triggered or scheduled pipeline has successfully started. 2. Integrating DevLake into workflows where subsequent steps depend on a pipeline entering the 'running' state. 3. Providing users with immediate status updates without requiring them to actively monitor the DevLake UI or API. Extending the notification system to cover pipeline creation and the transition to the 'running' state would significantly improve the user experience and integration capabilities by providing timely, pipeline-level status updates. ### Description DevLake currently supports sending notifications upon pipeline completion (success or failure), provided the `NOTIFICATION_ENDPOINT` and `NOTIFICATION_SECRET` environment variables are configured. We propose enhancing this existing notification system to send notifications for additional *pipeline-level* states. Specifically, we would like to add notifications for the following events: 1. **Pipeline Creation:** A notification should be sent immediately after a pipeline object is successfully created in the DevLake system (either manually via the API/UI or automatically triggered by a blueprint). 2. **Pipeline Running:** A notification should be sent when a pipeline transitions from a pending or queued state to an active 'running' state (i.e., when the pipeline execution process begins). These new notifications should leverage the existing `NOTIFICATION_ENDPOINT` and `NOTIFICATION_SECRET` configuration. The notification payload for these events should ideally include essential information such as the pipeline ID, the specific state change (e.g., `PIPELINE_CREATED`, `PIPELINE_RUNNING`), and potentially information about how the pipeline was triggered (e.g., manual, blueprint ID). This enhancement would provide users and integrated systems with earlier and clearer insights into the overall progress of a pipeline execution, complementing the existing completion notifications. ### Related issues _No response_ ### Are you willing to submit a PR? - [x] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@devlake.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
