PDGGK opened a new pull request, #9317:
URL: https://github.com/apache/devlake/pull/9317
### Summary
`convertApiTimelineRecords` is the azuredevops_go subtask that turns Azure
Pipelines timeline records (stages and jobs) into `cicd_tasks`. Its
`DomainTypes` was `CODEREVIEW`, while the two subtasks it depends on,
`collectApiTimelineRecords` and `extractApiTimelineRecords`, are `CICD`.
`makePipelinePlanV200` selects subtasks by domain type, so whenever a scope's
entities do not include `CODEREVIEW`, the timeline records are collected and
extracted but never converted:
- Repositories hosted outside Azure Repos (GitHub, Bitbucket) and built with
Azure Pipelines. The plan keeps only `CICD` (plus `CODE` for public
repositories) for these, so this happens on every run.
- Azure Repos scopes whose scope config selects CI/CD without Code Review.
For those scopes `cicd_tasks` stays empty. The `deploymentPattern` /
`productionPattern` matches on stage and job names are lost, and the dora
plugin, which also detects deployments through `cicd_tasks`, misses every
deployment that can only be recognized at the stage or job level.
This PR registers the converter under `CICD`, the same domain type as the
collector and extractor it depends on. Azure Repos scopes that include both
Code Review and CI/CD (the default) plan the converter exactly as before.
Before, the subtasks planned for a private GitHub repository with all
entities selected (the CI/CD part of the plan):
```
collectApiBuilds convertApiBuilds extractApiBuilds collectApiTimelineRecords
extractApiTimelineRecords
```
After:
```
collectApiBuilds convertApiBuilds extractApiBuilds collectApiTimelineRecords
convertApiTimelineRecords extractApiTimelineRecords
```
The result is the same for a public GitHub repository and for an Azure Repos
scope with only CI/CD selected.
To check the effect on the domain layer, I ran the plugin's e2e fixtures
(`raw_tables/_raw_azuredevops_go_api_builds.csv` and
`_raw_azuredevops_go_api_timeline_records.csv`) against MySQL 8.4, running only
the build and timeline subtasks that the plan selects for a CICD-only scope:
```
before: ran=[extractApiBuilds extractApiTimelineRecords]
_tool_azuredevops_go_timeline_records=5 cicd_tasks(azuredevops_go)=0
production deployment tasks=0
after: ran=[extractApiBuilds extractApiTimelineRecords
convertApiTimelineRecords]
_tool_azuredevops_go_timeline_records=5 cicd_tasks(azuredevops_go)=5
production deployment tasks=1
```
### Does this close any open issues?
Closes #8943
### Screenshots
Not applicable (backend only). The before/after output is above.
### Other Information
Testing:
- New test
`TestMakeDataSourcePipelinePlanV200ConvertsTimelineRecordsWithCicd` in
`api/blueprint_v200_test.go` checks the planned subtasks for a private GitHub
repository and for an Azure Repos scope with CICD only. When I reverted only
the `DomainTypes` change, both cases failed, and the diff shows
`convertApiTimelineRecords` missing after `extractApiTimelineRecords`.
- `go test` for `plugins/azuredevops_go/api` and
`plugins/azuredevops_go/tasks` passes.
- `plugins/azuredevops_go/e2e` passes against MySQL 8.4 (all 5 data-flow
tests, including `TestAzuredevopsTimelineDataFlow`).
- `gofmt`, `go vet`, and `golangci-lint` v2.15.0 on the two touched packages
are clean.
- PostgreSQL e2e and the rest of the suite are left to CI.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
