This is an automated email from the ASF dual-hosted git repository. lynwee pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/incubator-devlake-website.git
commit 0dce57a612098d58ec13f0c716e1b7feac8061dc Author: Startrekzky <[email protected]> AuthorDate: Tue Sep 19 18:41:10 2023 +0800 docs: update CICD models' description --- docs/DataModels/DevLakeDomainLayerSchema.md | 61 ++++++++++++---------- .../DataModels/DevLakeDomainLayerSchema.md | 61 ++++++++++++---------- .../DataModels/DevLakeDomainLayerSchema.md | 61 ++++++++++++---------- 3 files changed, 96 insertions(+), 87 deletions(-) diff --git a/docs/DataModels/DevLakeDomainLayerSchema.md b/docs/DataModels/DevLakeDomainLayerSchema.md index 1d66f2ca97..1c18362448 100644 --- a/docs/DataModels/DevLakeDomainLayerSchema.md +++ b/docs/DataModels/DevLakeDomainLayerSchema.md @@ -441,36 +441,37 @@ Normal comments, review bodies, reviews' inline comments of GitHub's pull reques #### cicd_scopes -The entity to filter or group 'cicd_pipelines' or 'cicd_tasks'. +The entity to filter or group 'cicd_pipelines'. - For GitHub: a GitHub repo is converted to a cicd_scope - For GitLab: a GitLab project is converted to a cicd_scope - For Jenkins: a Jenkins job is converted to a cicd_scope +- For Bamboo CI: a Bamboo plan is converted to a cicd_scope | **field** | **type** | **length** | **description** | **key** | | :------------- | :------- | :--------- | :---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :------ | -| `id` | varchar | 255 | A cicd_scope's `id` is composed of "< plugin >:< Entity >:< PK0 >[:PK1]..."<br/>For example, a Github cicd_scope's id is like "< github >:< GithubRepos >:< ConnectionId >:< GithubRepoId >". E.g. 'github:GithubRepos:1:384111310' | PK | +| `id` | varchar | 255 | A cicd_scope's `id` is composed of "< plugin >:< Entity >:< PluginConnectionId >[:PK1]..."<br/>For example, a GitHub cicd_scope's id is "github:GithubRepos:< GithubConnectionId >:< GithubRepoId >", a Bamboo cicd_scope's id is 'bamboo:BambooPlan:< BambooConnectionId >:< BambooPlanKey >' | PK | | `name` | varchar | 255 | The name of cicd_scope. | | | `description` | longtext | | The description of cicd_scope. | | | `url` | varchar | 255 | The url of cicd_scope. E.g. https://github.com/apache/incubator-devlake or https://jenkins.xxx.cn/view/PROD/job/OPS_releasev2/ | | -| `created_date` | datetime | 3 | cicd_scope creation date | | -| `updated_date` | datetime | 3 | Date of the last data collection for this cicd_scope | | +| `created_date` | datetime | 3 | Creation date of the cicd_scope, nullable | | +| `updated_date` | datetime | 3 | Updation date of the cicd_scope, nullable | | #### cicd_pipelines -A cicd_pipeline is a series of cicd_tasks, e.g. a GitHub workflow run, a GitLab pipeline, a BitBucket pipeline, a Jenkins build, etc. +A cicd_pipeline is the abstraction of a top-level CI/CD execution, e.g. a GitHub workflow run, a GitLab pipeline, a BitBucket pipeline, a Jenkins build, a Bamboo plan build, etc. A cicd_pipeline contains one or more of cicd_tasks. | **field** | **type** | **length** | **description** | **key** | | :-------------- | :-------------- | :--------- | :-------------------------------------------------------------------------------------------- | :---------------- | | `id` | varchar | 255 | This key is generated based on details from the original plugin | PK | | `name` | varchar | 255 | For gitlab, as there is no name for pipeline, so we use projectId, others have their own name | | -| `result` | varchar | 100 | The result of this task, e.g. SUCCESS, FAILURE | | -| `status` | varchar | 100 | The status of this task, e.g. IN_PROGRESS, DONE | | -| `type` | varchar | 100 | To indicate if this is a DEPLOYMENT | | -| `duration_sec` | bigint unsigned | | how long does this task take | | -| `created_date` | datetime | 3 | when did this task start | | -| `finished_date` | datetime | 3 | when did this task finish | | -| `environment` | varchar | 255 | To indicate the environment in which the task is running | | +| `result` | varchar | 100 | The result of the pipeline. It will be standardized to 'SUCCESS', 'FAILURE' in DevLake based on each plugin's possible results. | | +| `status` | varchar | 100 | The status of the pipeline. It will be standardized to 'DONE', 'IN_PROGRESS' in DevLake based on each plugin's possible statues. | +| `type` | varchar | 100 | The value will be set to 'DEPLOYMENT' if it matched the ReGex configured in the Scope Config, otherwise it is an empty string. | | +| `environment` | varchar | 255 | The value will be set to 'PRODUCTION' if it matched the ReGex configured in the Scope Config, otherwise it is an empty string. | | +| `duration_sec` | bigint unsigned | | how long does this pipeline take | | +| `created_date` | datetime | 3 | When this pipeline started | | +| `finished_date` | datetime | 3 | When this pipeline finished | | | `cicd_scope_id` | longtext | | The id of cicd_scope this pipeline belongs to | FK_cicd_scopes.id | #### cicd_pipeline_commits @@ -485,32 +486,34 @@ A cicd_pipeline is a series of cicd_tasks, e.g. a GitHub workflow run, a GitLab #### cicd_tasks -A cicd_task is the abstraction of the smallest unit of CICD tasks. +A cicd_task is the abstraction of the bottom-level CI/CD excecution. -- For GitHub: a cicd_task is a GitHub job -- For GitLab: a cicd_task is a GitLab job +- For GitHub: a cicd_task is a GitHub job run in a GitHub workflow run. +- For GitLab: a cicd_task is a GitLab job run of a GitLab pipeline run. - For Jenkins: a cicd_task is a subtask of a Jenkins build. If a build does not have subtask(s), then the build will also be saved as a cicd_task in this table. +- For Bamboo CI: a cicd_task is a Bamboo job build in a Bamboo plan build. + | **field** | **type** | **length** | **description** | **key** | | :-------------- | :-------------- | :--------- | :---------------------------------------------------------------------------------------- | :---------------- | | `id` | varchar | 255 | This key is generated based on details from the original plugin | PK | | `name` | varchar | 255 | | | -| `pipeline_id` | varchar | 255 | The id of pipeline | | -| `result` | varchar | 100 | The result of this task, e.g. SUCCESS, FAILURE | | -| `status` | varchar | 100 | The status of this task, e.g. IN_PROGRESS, DONE | | -| `type` | varchar | 100 | To indicate if this is a deployment-type task | | -| `duration_sec` | bigint unsigned | | how long does this task take | | -| `started_date` | datetime | 3 | when did this task start | | -| `finished_date` | datetime | 3 | when did this task finish | | -| `environment` | varchar | 255 | To indicate the environment in which the task is running, e.g. production, staging, test. | | -| `cicd_scope_id` | longtext | | The id of cicd_scope this pipeline belongs to | FK_cicd_scopes.id | +| `pipeline_id` | varchar | 255 | The id of the cicd_pipeline it belongs to | | +| `result` | varchar | 100 | The result of the task. It will be standardized to 'SUCCESS', 'FAILURE' in DevLake based on each plugin's possible. | | +| `status` | varchar | 100 | The status of the task. It will be standardized to 'DONE', 'IN_PROGRESS' in DevLake based on each plugin's possible statues. | | +| `type` | varchar | 100 | The value will be set to 'DEPLOYMENT' if it matched the ReGex configured in the Scope Config, otherwise it is an empty string. | | +| `environment` | varchar | 255 | The value will be set to 'PRODUCTION' if it matched the ReGex configured in the Scope Config, otherwise it is an empty string. | +| `duration_sec` | bigint unsigned | | How long does this task take | | +| `started_date` | datetime | 3 | When this task started | | +| `finished_date` | datetime | 3 | When this task finished | | | +| `cicd_scope_id` | longtext | | The id of cicd_scope this task belongs to | FK_cicd_scopes.id | #### cicd_deployment_commits A cicd_deployment_commit is a deployment in a specific repo. A deployment may come from several sources: -- Domain layer [cicd_pipelines](#cicd_pipelines), such as GitHub workflow run, GitLab pipelines, Jenkins builds and BitBucket pipelines, etc. Deployments from cicd_pipelines will be transformed according to the regex configuration set in the Blueprint transformation before adding to this table. -- Tool layer deployments: in v0.17, only the BitBucket plugin collects the independent deployment entity which you can find in table.\_tool_bitbucket_deployments, but there will be more in the future. +- Domain layer [cicd_pipelines](#cicd_pipelines), such as GitHub workflow runs, GitLab pipelines, Jenkins builds and BitBucket pipelines, etc. Deployments from cicd_pipelines will be transformed according to the regex configuration set in the Blueprint transformation before adding to this table. +- Tool layer deployments: in v0.18, only the BitBucket and Bamboo plugins collect the independent deployment entity which you can find in table.\_tool_bitbucket_deployments and \_tool_bamboo_deploy_builds, but there will be more in the future. - Deployments pushed directly from webhooks You can query deployments from this table by `SELECT DISTINCT cicd_deployment_id FROM cicd_deployments_commits`. @@ -526,15 +529,15 @@ Normally, one deployment only deploy to one repo. But in some cases, one deploym | `result` | varchar | 100 | The result of the deployment, e.g. SUCCESS, FAILURE | | | `status` | varchar | 100 | The status of this deployment, e.g. IN_PROGRESS, DONE | | | `environment` | varchar | 255 | The environment to deploy, only 'PRODUCTION' deployment will appear in v0.17 | | -| `created_date` | datetime | 3 | The created time of the deployment | | -| `started_date` | datetime | 3 | The started time of the deployment | | +| `created_date` | datetime | 3 | The created time of the deployment. Deprecated. | | +| `started_date` | datetime | 3 | The started time of the deployment. | | | `finished_date` | datetime | 3 | The finished time of the deployment | | | `duration_sec` | bigint | | The time this deployment takes | | | `commit_sha` | char | 40 | The commit sha that triggers the deployment | | | `ref_name` | varchar | 255 | The ref (branch/tag) name of the commit | | | `repo_id` | varchar | 255 | - | | | `repo_url` | varchar | 191 | The url of the repo | | -| `prev_success_deployment_commit_id` | varchar | 255 | The last successful deployment_commit_id before this one, which is used to calculate how many new commits have been deployed by this deployment_commit | | +| `prev_success_deployment_commit_id` | varchar | 255 | The last successful deployment_commit_id before this one in the same `cicd_scope`, `repo` and `environment`, which is used to calculate the new commits deployed by this deployment, thereby measuring [DORA - change lead time](../Metrics/LeadTimeForChanges.md). [...] ### Domain 5 - Code Quality diff --git a/versioned_docs/version-v0.18/DataModels/DevLakeDomainLayerSchema.md b/versioned_docs/version-v0.18/DataModels/DevLakeDomainLayerSchema.md index 3be4949460..ec149c3c09 100644 --- a/versioned_docs/version-v0.18/DataModels/DevLakeDomainLayerSchema.md +++ b/versioned_docs/version-v0.18/DataModels/DevLakeDomainLayerSchema.md @@ -431,36 +431,37 @@ Normal comments, review bodies, reviews' inline comments of GitHub's pull reques #### cicd_scopes -The entity to filter or group 'cicd_pipelines' or 'cicd_tasks'. +The entity to filter or group 'cicd_pipelines'. - For GitHub: a GitHub repo is converted to a cicd_scope - For GitLab: a GitLab project is converted to a cicd_scope - For Jenkins: a Jenkins job is converted to a cicd_scope +- For Bamboo CI: a Bamboo plan is converted to a cicd_scope | **field** | **type** | **length** | **description** | **key** | | :------------- | :------- | :--------- | :---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :------ | -| `id` | varchar | 255 | A cicd_scope's `id` is composed of "< plugin >:< Entity >:< PK0 >[:PK1]..."<br/>For example, a Github cicd_scope's id is like "< github >:< GithubRepos >:< ConnectionId >:< GithubRepoId >". E.g. 'github:GithubRepos:1:384111310' | PK | +| `id` | varchar | 255 | A cicd_scope's `id` is composed of "< plugin >:< Entity >:< PluginConnectionId >[:PK1]..."<br/>For example, a GitHub cicd_scope's id is "github:GithubRepos:< GithubConnectionId >:< GithubRepoId >", a Bamboo cicd_scope's id is 'bamboo:BambooPlan:< BambooConnectionId >:< BambooPlanKey >' | PK | | `name` | varchar | 255 | The name of cicd_scope. | | | `description` | longtext | | The description of cicd_scope. | | | `url` | varchar | 255 | The url of cicd_scope. E.g. https://github.com/apache/incubator-devlake or https://jenkins.xxx.cn/view/PROD/job/OPS_releasev2/ | | -| `created_date` | datetime | 3 | cicd_scope creation date | | -| `updated_date` | datetime | 3 | Date of the last data collection for this cicd_scope | | +| `created_date` | datetime | 3 | Creation date of the cicd_scope, nullable | | +| `updated_date` | datetime | 3 | Updation date of the cicd_scope, nullable | | #### cicd_pipelines -A cicd_pipeline is a series of cicd_tasks, e.g. a GitHub workflow run, a GitLab pipeline, a BitBucket pipeline, a Jenkins build, etc. +A cicd_pipeline is the abstraction of a top-level CI/CD execution, e.g. a GitHub workflow run, a GitLab pipeline, a BitBucket pipeline, a Jenkins build, a Bamboo plan build, etc. A cicd_pipeline contains one or more of cicd_tasks. | **field** | **type** | **length** | **description** | **key** | | :-------------- | :-------------- | :--------- | :-------------------------------------------------------------------------------------------- | :---------------- | | `id` | varchar | 255 | This key is generated based on details from the original plugin | PK | | `name` | varchar | 255 | For gitlab, as there is no name for pipeline, so we use projectId, others have their own name | | -| `result` | varchar | 100 | The result of this task, e.g. SUCCESS, FAILURE | | -| `status` | varchar | 100 | The status of this task, e.g. IN_PROGRESS, DONE | | -| `type` | varchar | 100 | To indicate if this is a DEPLOYMENT | | -| `duration_sec` | bigint unsigned | | how long does this task take | | -| `created_date` | datetime | 3 | when did this task start | | -| `finished_date` | datetime | 3 | when did this task finish | | -| `environment` | varchar | 255 | To indicate the environment in which the task is running | | +| `result` | varchar | 100 | The result of the pipeline. It will be standardized to 'SUCCESS', 'FAILURE' in DevLake based on each plugin's possible results. | | +| `status` | varchar | 100 | The status of the pipeline. It will be standardized to 'DONE', 'IN_PROGRESS' in DevLake based on each plugin's possible statues. | +| `type` | varchar | 100 | The value will be set to 'DEPLOYMENT' if it matched the ReGex configured in the Scope Config, otherwise it is an empty string. | | +| `environment` | varchar | 255 | The value will be set to 'PRODUCTION' if it matched the ReGex configured in the Scope Config, otherwise it is an empty string. | | +| `duration_sec` | bigint unsigned | | how long does this pipeline take | | +| `created_date` | datetime | 3 | When this pipeline started | | +| `finished_date` | datetime | 3 | When this pipeline finished | | | `cicd_scope_id` | longtext | | The id of cicd_scope this pipeline belongs to | FK_cicd_scopes.id | #### cicd_pipeline_commits @@ -475,32 +476,34 @@ A cicd_pipeline is a series of cicd_tasks, e.g. a GitHub workflow run, a GitLab #### cicd_tasks -A cicd_task is the abstraction of the smallest unit of CICD tasks. +A cicd_task is the abstraction of the bottom-level CI/CD excecution. -- For GitHub: a cicd_task is a GitHub job -- For GitLab: a cicd_task is a GitLab job +- For GitHub: a cicd_task is a GitHub job run in a GitHub workflow run. +- For GitLab: a cicd_task is a GitLab job run of a GitLab pipeline run. - For Jenkins: a cicd_task is a subtask of a Jenkins build. If a build does not have subtask(s), then the build will also be saved as a cicd_task in this table. +- For Bamboo CI: a cicd_task is a Bamboo job build in a Bamboo plan build. + | **field** | **type** | **length** | **description** | **key** | | :-------------- | :-------------- | :--------- | :---------------------------------------------------------------------------------------- | :---------------- | | `id` | varchar | 255 | This key is generated based on details from the original plugin | PK | | `name` | varchar | 255 | | | -| `pipeline_id` | varchar | 255 | The id of pipeline | | -| `result` | varchar | 100 | The result of this task, e.g. SUCCESS, FAILURE | | -| `status` | varchar | 100 | The status of this task, e.g. IN_PROGRESS, DONE | | -| `type` | varchar | 100 | To indicate if this is a deployment-type task | | -| `duration_sec` | bigint unsigned | | how long does this task take | | -| `started_date` | datetime | 3 | when did this task start | | -| `finished_date` | datetime | 3 | when did this task finish | | -| `environment` | varchar | 255 | To indicate the environment in which the task is running, e.g. production, staging, test. | | -| `cicd_scope_id` | longtext | | The id of cicd_scope this pipeline belongs to | FK_cicd_scopes.id | +| `pipeline_id` | varchar | 255 | The id of the cicd_pipeline it belongs to | | +| `result` | varchar | 100 | The result of the task. It will be standardized to 'SUCCESS', 'FAILURE' in DevLake based on each plugin's possible. | | +| `status` | varchar | 100 | The status of the task. It will be standardized to 'DONE', 'IN_PROGRESS' in DevLake based on each plugin's possible statues. | | +| `type` | varchar | 100 | The value will be set to 'DEPLOYMENT' if it matched the ReGex configured in the Scope Config, otherwise it is an empty string. | | +| `environment` | varchar | 255 | The value will be set to 'PRODUCTION' if it matched the ReGex configured in the Scope Config, otherwise it is an empty string. | +| `duration_sec` | bigint unsigned | | How long does this task take | | +| `started_date` | datetime | 3 | When this task started | | +| `finished_date` | datetime | 3 | When this task finished | | | +| `cicd_scope_id` | longtext | | The id of cicd_scope this task belongs to | FK_cicd_scopes.id | #### cicd_deployment_commits A cicd_deployment_commit is a deployment in a specific repo. A deployment may come from several sources: -- Domain layer [cicd_pipelines](#cicd_pipelines), such as GitHub workflow run, GitLab pipelines, Jenkins builds and BitBucket pipelines, etc. Deployments from cicd_pipelines will be transformed according to the regex configuration set in the Blueprint transformation before adding to this table. -- Tool layer deployments: in v0.17, only the BitBucket plugin collects the independent deployment entity which you can find in table.\_tool_bitbucket_deployments, but there will be more in the future. +- Domain layer [cicd_pipelines](#cicd_pipelines), such as GitHub workflow runs, GitLab pipelines, Jenkins builds and BitBucket pipelines, etc. Deployments from cicd_pipelines will be transformed according to the regex configuration set in the Blueprint transformation before adding to this table. +- Tool layer deployments: in v0.18, only the BitBucket and Bamboo plugins collect the independent deployment entity which you can find in table.\_tool_bitbucket_deployments and \_tool_bamboo_deploy_builds, but there will be more in the future. - Deployments pushed directly from webhooks You can query deployments from this table by `SELECT DISTINCT cicd_deployment_id FROM cicd_deployments_commits`. @@ -516,15 +519,15 @@ Normally, one deployment only deploy to one repo. But in some cases, one deploym | `result` | varchar | 100 | The result of the deployment, e.g. SUCCESS, FAILURE | | | `status` | varchar | 100 | The status of this deployment, e.g. IN_PROGRESS, DONE | | | `environment` | varchar | 255 | The environment to deploy, only 'PRODUCTION' deployment will appear in v0.17 | | -| `created_date` | datetime | 3 | The created time of the deployment | | -| `started_date` | datetime | 3 | The started time of the deployment | | +| `created_date` | datetime | 3 | The created time of the deployment. Deprecated. | | +| `started_date` | datetime | 3 | The started time of the deployment. | | | `finished_date` | datetime | 3 | The finished time of the deployment | | | `duration_sec` | bigint | | The time this deployment takes | | | `commit_sha` | char | 40 | The commit sha that triggers the deployment | | | `ref_name` | varchar | 255 | The ref (branch/tag) name of the commit | | | `repo_id` | varchar | 255 | - | | | `repo_url` | varchar | 191 | The url of the repo | | -| `prev_success_deployment_commit_id` | varchar | 255 | The last successful deployment_commit_id before this one, which is used to calculate how many new commits have been deployed by this deployment_commit | | +| `prev_success_deployment_commit_id` | varchar | 255 | The last successful deployment_commit_id before this one in the same `cicd_scope`, `repo` and `environment`, which is used to calculate the new commits deployed by this deployment, thereby measuring [DORA - change lead time](../Metrics/LeadTimeForChanges.md). [...] ### Domain 5 - Code Quality diff --git a/versioned_docs/version-v0.19/DataModels/DevLakeDomainLayerSchema.md b/versioned_docs/version-v0.19/DataModels/DevLakeDomainLayerSchema.md index 1d66f2ca97..1c18362448 100644 --- a/versioned_docs/version-v0.19/DataModels/DevLakeDomainLayerSchema.md +++ b/versioned_docs/version-v0.19/DataModels/DevLakeDomainLayerSchema.md @@ -441,36 +441,37 @@ Normal comments, review bodies, reviews' inline comments of GitHub's pull reques #### cicd_scopes -The entity to filter or group 'cicd_pipelines' or 'cicd_tasks'. +The entity to filter or group 'cicd_pipelines'. - For GitHub: a GitHub repo is converted to a cicd_scope - For GitLab: a GitLab project is converted to a cicd_scope - For Jenkins: a Jenkins job is converted to a cicd_scope +- For Bamboo CI: a Bamboo plan is converted to a cicd_scope | **field** | **type** | **length** | **description** | **key** | | :------------- | :------- | :--------- | :---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :------ | -| `id` | varchar | 255 | A cicd_scope's `id` is composed of "< plugin >:< Entity >:< PK0 >[:PK1]..."<br/>For example, a Github cicd_scope's id is like "< github >:< GithubRepos >:< ConnectionId >:< GithubRepoId >". E.g. 'github:GithubRepos:1:384111310' | PK | +| `id` | varchar | 255 | A cicd_scope's `id` is composed of "< plugin >:< Entity >:< PluginConnectionId >[:PK1]..."<br/>For example, a GitHub cicd_scope's id is "github:GithubRepos:< GithubConnectionId >:< GithubRepoId >", a Bamboo cicd_scope's id is 'bamboo:BambooPlan:< BambooConnectionId >:< BambooPlanKey >' | PK | | `name` | varchar | 255 | The name of cicd_scope. | | | `description` | longtext | | The description of cicd_scope. | | | `url` | varchar | 255 | The url of cicd_scope. E.g. https://github.com/apache/incubator-devlake or https://jenkins.xxx.cn/view/PROD/job/OPS_releasev2/ | | -| `created_date` | datetime | 3 | cicd_scope creation date | | -| `updated_date` | datetime | 3 | Date of the last data collection for this cicd_scope | | +| `created_date` | datetime | 3 | Creation date of the cicd_scope, nullable | | +| `updated_date` | datetime | 3 | Updation date of the cicd_scope, nullable | | #### cicd_pipelines -A cicd_pipeline is a series of cicd_tasks, e.g. a GitHub workflow run, a GitLab pipeline, a BitBucket pipeline, a Jenkins build, etc. +A cicd_pipeline is the abstraction of a top-level CI/CD execution, e.g. a GitHub workflow run, a GitLab pipeline, a BitBucket pipeline, a Jenkins build, a Bamboo plan build, etc. A cicd_pipeline contains one or more of cicd_tasks. | **field** | **type** | **length** | **description** | **key** | | :-------------- | :-------------- | :--------- | :-------------------------------------------------------------------------------------------- | :---------------- | | `id` | varchar | 255 | This key is generated based on details from the original plugin | PK | | `name` | varchar | 255 | For gitlab, as there is no name for pipeline, so we use projectId, others have their own name | | -| `result` | varchar | 100 | The result of this task, e.g. SUCCESS, FAILURE | | -| `status` | varchar | 100 | The status of this task, e.g. IN_PROGRESS, DONE | | -| `type` | varchar | 100 | To indicate if this is a DEPLOYMENT | | -| `duration_sec` | bigint unsigned | | how long does this task take | | -| `created_date` | datetime | 3 | when did this task start | | -| `finished_date` | datetime | 3 | when did this task finish | | -| `environment` | varchar | 255 | To indicate the environment in which the task is running | | +| `result` | varchar | 100 | The result of the pipeline. It will be standardized to 'SUCCESS', 'FAILURE' in DevLake based on each plugin's possible results. | | +| `status` | varchar | 100 | The status of the pipeline. It will be standardized to 'DONE', 'IN_PROGRESS' in DevLake based on each plugin's possible statues. | +| `type` | varchar | 100 | The value will be set to 'DEPLOYMENT' if it matched the ReGex configured in the Scope Config, otherwise it is an empty string. | | +| `environment` | varchar | 255 | The value will be set to 'PRODUCTION' if it matched the ReGex configured in the Scope Config, otherwise it is an empty string. | | +| `duration_sec` | bigint unsigned | | how long does this pipeline take | | +| `created_date` | datetime | 3 | When this pipeline started | | +| `finished_date` | datetime | 3 | When this pipeline finished | | | `cicd_scope_id` | longtext | | The id of cicd_scope this pipeline belongs to | FK_cicd_scopes.id | #### cicd_pipeline_commits @@ -485,32 +486,34 @@ A cicd_pipeline is a series of cicd_tasks, e.g. a GitHub workflow run, a GitLab #### cicd_tasks -A cicd_task is the abstraction of the smallest unit of CICD tasks. +A cicd_task is the abstraction of the bottom-level CI/CD excecution. -- For GitHub: a cicd_task is a GitHub job -- For GitLab: a cicd_task is a GitLab job +- For GitHub: a cicd_task is a GitHub job run in a GitHub workflow run. +- For GitLab: a cicd_task is a GitLab job run of a GitLab pipeline run. - For Jenkins: a cicd_task is a subtask of a Jenkins build. If a build does not have subtask(s), then the build will also be saved as a cicd_task in this table. +- For Bamboo CI: a cicd_task is a Bamboo job build in a Bamboo plan build. + | **field** | **type** | **length** | **description** | **key** | | :-------------- | :-------------- | :--------- | :---------------------------------------------------------------------------------------- | :---------------- | | `id` | varchar | 255 | This key is generated based on details from the original plugin | PK | | `name` | varchar | 255 | | | -| `pipeline_id` | varchar | 255 | The id of pipeline | | -| `result` | varchar | 100 | The result of this task, e.g. SUCCESS, FAILURE | | -| `status` | varchar | 100 | The status of this task, e.g. IN_PROGRESS, DONE | | -| `type` | varchar | 100 | To indicate if this is a deployment-type task | | -| `duration_sec` | bigint unsigned | | how long does this task take | | -| `started_date` | datetime | 3 | when did this task start | | -| `finished_date` | datetime | 3 | when did this task finish | | -| `environment` | varchar | 255 | To indicate the environment in which the task is running, e.g. production, staging, test. | | -| `cicd_scope_id` | longtext | | The id of cicd_scope this pipeline belongs to | FK_cicd_scopes.id | +| `pipeline_id` | varchar | 255 | The id of the cicd_pipeline it belongs to | | +| `result` | varchar | 100 | The result of the task. It will be standardized to 'SUCCESS', 'FAILURE' in DevLake based on each plugin's possible. | | +| `status` | varchar | 100 | The status of the task. It will be standardized to 'DONE', 'IN_PROGRESS' in DevLake based on each plugin's possible statues. | | +| `type` | varchar | 100 | The value will be set to 'DEPLOYMENT' if it matched the ReGex configured in the Scope Config, otherwise it is an empty string. | | +| `environment` | varchar | 255 | The value will be set to 'PRODUCTION' if it matched the ReGex configured in the Scope Config, otherwise it is an empty string. | +| `duration_sec` | bigint unsigned | | How long does this task take | | +| `started_date` | datetime | 3 | When this task started | | +| `finished_date` | datetime | 3 | When this task finished | | | +| `cicd_scope_id` | longtext | | The id of cicd_scope this task belongs to | FK_cicd_scopes.id | #### cicd_deployment_commits A cicd_deployment_commit is a deployment in a specific repo. A deployment may come from several sources: -- Domain layer [cicd_pipelines](#cicd_pipelines), such as GitHub workflow run, GitLab pipelines, Jenkins builds and BitBucket pipelines, etc. Deployments from cicd_pipelines will be transformed according to the regex configuration set in the Blueprint transformation before adding to this table. -- Tool layer deployments: in v0.17, only the BitBucket plugin collects the independent deployment entity which you can find in table.\_tool_bitbucket_deployments, but there will be more in the future. +- Domain layer [cicd_pipelines](#cicd_pipelines), such as GitHub workflow runs, GitLab pipelines, Jenkins builds and BitBucket pipelines, etc. Deployments from cicd_pipelines will be transformed according to the regex configuration set in the Blueprint transformation before adding to this table. +- Tool layer deployments: in v0.18, only the BitBucket and Bamboo plugins collect the independent deployment entity which you can find in table.\_tool_bitbucket_deployments and \_tool_bamboo_deploy_builds, but there will be more in the future. - Deployments pushed directly from webhooks You can query deployments from this table by `SELECT DISTINCT cicd_deployment_id FROM cicd_deployments_commits`. @@ -526,15 +529,15 @@ Normally, one deployment only deploy to one repo. But in some cases, one deploym | `result` | varchar | 100 | The result of the deployment, e.g. SUCCESS, FAILURE | | | `status` | varchar | 100 | The status of this deployment, e.g. IN_PROGRESS, DONE | | | `environment` | varchar | 255 | The environment to deploy, only 'PRODUCTION' deployment will appear in v0.17 | | -| `created_date` | datetime | 3 | The created time of the deployment | | -| `started_date` | datetime | 3 | The started time of the deployment | | +| `created_date` | datetime | 3 | The created time of the deployment. Deprecated. | | +| `started_date` | datetime | 3 | The started time of the deployment. | | | `finished_date` | datetime | 3 | The finished time of the deployment | | | `duration_sec` | bigint | | The time this deployment takes | | | `commit_sha` | char | 40 | The commit sha that triggers the deployment | | | `ref_name` | varchar | 255 | The ref (branch/tag) name of the commit | | | `repo_id` | varchar | 255 | - | | | `repo_url` | varchar | 191 | The url of the repo | | -| `prev_success_deployment_commit_id` | varchar | 255 | The last successful deployment_commit_id before this one, which is used to calculate how many new commits have been deployed by this deployment_commit | | +| `prev_success_deployment_commit_id` | varchar | 255 | The last successful deployment_commit_id before this one in the same `cicd_scope`, `repo` and `environment`, which is used to calculate the new commits deployed by this deployment, thereby measuring [DORA - change lead time](../Metrics/LeadTimeForChanges.md). [...] ### Domain 5 - Code Quality
