narrowizard opened a new issue, #8763: URL: https://github.com/apache/incubator-devlake/issues/8763
### Search before asking - [x] I had searched in the [issues](https://github.com/apache/incubator-devlake/issues?q=is%3Aissue) and found no similar feature requirement. ### Use case QA teams often encounter test results that are invalid due to environmental issues, flaky tests, false positives, or false negatives. Currently, the domain layer entity qa_test_case_executions does not have a standard way to mark such executions as invalid. This makes it difficult to filter out unreliable results when calculating quality metrics (e.g., pass rate, flakiness) or generating reports. Adding an is_invalid flag will allow teams to exclude invalid test executions from their analysis, leading to more accurate insights into test stability and product quality. The field can be populated from tool-specific data (e.g., custom fields in Jira, TestRail, or other QA tools) or set manually through data enrichment. ### Description We propose adding a new boolean field is_invalid to the domain layer table qa_test_case_executions. This field should default to false to maintain backward compatibility with existing data and ETL pipelines. The domain layer schema definition must be updated accordingly, and the necessary ETL transformations should be extended to optionally map tool-layer data to this new field. For tools that natively support marking test executions as invalid (e.g., via a custom flag or status), the transformation logic can populate is_invalid based on that information. For other tools, the field will remain false unless manually updated. This addition will enable downstream metrics and dashboards to properly account for invalid test results, improving the reliability of quality reports. ### Related issues _No response_ ### Are you willing to submit a PR? - [x] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
