narrowizard opened a new issue, #8531: URL: https://github.com/apache/incubator-devlake/issues/8531
### Search before asking - [x] I had searched in the [issues](https://github.com/apache/incubator-devlake/issues?q=is%3Aissue) and found no similar feature requirement. ### Use case In environments where multiple pipelines are triggered simultaneously—either manually or through scheduled blueprints—users often need to prioritize critical data collection tasks over less urgent ones. For example: - During a high-priority incident response, a pipeline collecting real-time metrics from a DevOps tool (e.g., Jira or GitHub) should run before routine daily reports. - In resource-constrained setups, ensuring high-priority pipelines (e.g., for compliance audits or key stakeholder insights) execute first can prevent delays in extracting time-sensitive engineering excellence or developer experience data. - For community growth analysis, urgent pipelines analyzing recent contributions could be prioritized over historical data refreshes. This feature would enable more advanced task control, allowing teams to manage queue overloads efficiently without manual intervention. ### Description Currently, pipelines in Apache DevLake—whether created manually or scheduled via blueprints—are executed based on their trigger order or timing, without an inherent mechanism for prioritization. This can lead to inefficiencies when multiple pipelines compete for resources. To address this, introduce a new "priority" attribute to the pipeline entity: - The priority could be an integer field (e.g., ranging from 1 to 10, where higher values indicate higher priority). - This attribute should be configurable during pipeline creation (manual or via blueprint definitions). - When scheduling or queuing pipelines for execution, the scheduler should sort and process them in descending order of priority (highest first). If priorities are equal, fall back to the existing ordering (e.g., FIFO or timestamp-based). - Ensure backward compatibility: Existing pipelines without a priority could default to a medium value (e.g., 5). - Update relevant APIs, UI, and documentation to support setting and viewing priorities. This enhancement would provide finer-grained control over task execution, improving overall platform usability for complex DevOps data workflows. ### Related issues _No response_ ### Are you willing to submit a PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@devlake.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
