jbsmith7741 opened a new pull request, #8983: URL: https://github.com/apache/devlake/pull/8983
### Summary Closes [#8941](https://github.com/apache/devlake/issues/8941). DevLake's SonarQube plugin previously stored only point-in-time metrics. Each sync overwrote prior values, so there was no way to answer questions like "how has coverage trended over the past year?" This PR adds project-level historical collection from SonarQube's `measures/search_history` and `project_analyses/search` APIs, persists them through the standard Raw → Tool → Domain pipeline, and adds Grafana trend panels that honor the time picker. **Data pipeline** ```mermaid flowchart LR SQ[SonarQube API] --> C1[CollectProjectMetricsHistory] SQ --> C2[CollectProjectAnalyses] C1 --> R1["_raw_sonarqube_api_project_metrics_history"] C2 --> R2["_raw_sonarqube_api_project_analyses"] R1 --> T1["_tool_sonarqube_project_metrics_history"] R2 --> T2["_tool_sonarqube_project_analyses"] T1 --> D["cq_project_metrics_history"] T2 --> D ``` **SonarQube plugin** - Five new subtasks: collect/extract project metrics history, collect/extract project analyses, convert to domain - New tool tables: `_tool_sonarqube_project_metrics_history`, `_tool_sonarqube_project_analyses` - New domain table: `cq_project_metrics_history` (coverage, ncloc, bugs, vulnerabilities, code_smells, ratings, complexity, duplication, etc.) - Uses `NewStatefulApiCollector` with sync policy `TimeAfter` for initial backfill and incremental runs - Metrics collected: coverage, ncloc, bugs, reliability_rating, code_smells, sqale_rating, complexity, cognitive_complexity, vulnerabilities, security_rating, security_hotspots, duplicated_lines_density **Connection validation** - Reject Global (`sqa_`) and Project (`sqp_`) analysis tokens on SonarQube Server at connection test — they authenticate but cannot read metrics via the Web API - SonarCloud skips prefix validation (token formats differ) - Unit tests in `connection_test.go` **Grafana dashboards** - Collapsed **Historical Trends** row added to SonarQube Server and SonarQube Cloud dashboards (MySQL and PostgreSQL variants) - Four time-series panels: coverage, bugs & vulnerabilities, code smells, duplication - Panels query `cq_project_metrics_history` and use `$__timeFilter(pmh.analysis_date)` - Existing snapshot stat panels are unchanged ### Does this close any open issues? Closes #8941 ### Screenshots <img width="1098" height="653" alt="SonarQube_Historic_Metrics" src="https://github.com/user-attachments/assets/c6ec3ed7-18de-4cc3-84e4-bd0e3173cc8d"; /> ### Other Information **How to verify** 1. Configure a SonarQube connection with a User token (`squ_` prefix on Server). 2. Run a blueprint sync for a project with analysis history. 3. Confirm data in `_tool_sonarqube_project_metrics_history`, `_tool_sonarqube_project_analyses`, and `cq_project_metrics_history`. 4. Open the SonarQube Server or SonarQube Cloud dashboard in Grafana, select a project, set time range to Last 6 months, and expand **Historical Trends**. 5. Connection test: `sqa_` / `sqp_` tokens on Server should fail with a clear message; `squ_` should succeed. 6. Run unit tests: `cd backend && go test ./plugins/sonarqube/models/... -run TestValidateUserTokenPrefix` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
