klesh commented on code in PR #308:
URL:
https://github.com/apache/incubator-devlake-website/pull/308#discussion_r1034494828
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,320 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ `Project` is **a set of [Scope](../Glossary.md#data-scope) from different
domains**, a way to group different resources, and it is crucial for some
metric calculations like `Dora`.
+---
+
+# Summary
+
+In related calculations such as `Dora` indicators, we often encounter
situations requiring comprehensive calculations based on data from multiple
data sources.
Review Comment:
For some metric calculations such as the DORA metric,
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,320 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ `Project` is **a set of [Scope](../Glossary.md#data-scope) from different
domains**, a way to group different resources, and it is crucial for some
metric calculations like `Dora`.
+---
+
+# Summary
+
+In related calculations such as `Dora` indicators, we often encounter
situations requiring comprehensive calculations based on data from multiple
data sources.
+
+For example, we do code version management on `Gitlab`, and then do
corresponding compilation and deployment operations in `Jenkins`. At this time,
our indicator calculation will depend on the data of `Gitlab` and `Jenkins` at
the same time.
+
+However, the reality is that sometimes, we have plural `Gitlab` connections
and plural `Jenkins` connections.
+
+At this time, to support calculations that rely on multiple data sources like
`Dora`, we need to correctly associate different
[Scope](../Glossary.md#data-scope) under different data sources through a
method.
+
+So, we created `Project`
Review Comment:
delete
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,320 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ `Project` is **a set of [Scope](../Glossary.md#data-scope) from different
domains**, a way to group different resources, and it is crucial for some
metric calculations like `Dora`.
+---
+
+# Summary
+
+In related calculations such as `Dora` indicators, we often encounter
situations requiring comprehensive calculations based on data from multiple
data sources.
+
+For example, we do code version management on `Gitlab`, and then do
corresponding compilation and deployment operations in `Jenkins`. At this time,
our indicator calculation will depend on the data of `Gitlab` and `Jenkins` at
the same time.
Review Comment:
For example, we may use Gitlab for code version control, Jenkins for CI/CD,
to calculate PR deployment cycle time, we need to know which Gitlab Projects
and Jenkins Jobs are related for correctness and performance reasons.
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,320 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ `Project` is **a set of [Scope](../Glossary.md#data-scope) from different
domains**, a way to group different resources, and it is crucial for some
metric calculations like `Dora`.
+---
+
+# Summary
+
+In related calculations such as `Dora` indicators, we often encounter
situations requiring comprehensive calculations based on data from multiple
data sources.
+
+For example, we do code version management on `Gitlab`, and then do
corresponding compilation and deployment operations in `Jenkins`. At this time,
our indicator calculation will depend on the data of `Gitlab` and `Jenkins` at
the same time.
+
+However, the reality is that sometimes, we have plural `Gitlab` connections
and plural `Jenkins` connections.
+
+At this time, to support calculations that rely on multiple data sources like
`Dora`, we need to correctly associate different
[Scope](../Glossary.md#data-scope) under different data sources through a
method.
+
+So, we created `Project`
+
+`Project` is **a set of [Scope](../Glossary.md#data-scope) from different
domains**, a way to group different resources, and it is crucial for some
metric calculation like `Dora`.
+
+By adding the `project_name` field to the `blueprint` table, we associate
`project` with `blueprint` one-to-one.
+
+As a developer, when we are doing a new plugin development, or when we do some
other operations that need to be associated with `Project` to add, delete, and
modify operations, we need to have a detailed understanding of `Project`.
Review Comment:
I think it would be better to rephrase Line24-31 into a section named
"Interfaces" and it should be placed after the "Model" section.
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,320 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ `Project` is **a set of [Scope](../Glossary.md#data-scope) from different
domains**, a way to group different resources, and it is crucial for some
metric calculations like `Dora`.
+---
+
+# Summary
+
+In related calculations such as `Dora` indicators, we often encounter
situations requiring comprehensive calculations based on data from multiple
data sources.
+
+For example, we do code version management on `Gitlab`, and then do
corresponding compilation and deployment operations in `Jenkins`. At this time,
our indicator calculation will depend on the data of `Gitlab` and `Jenkins` at
the same time.
+
+However, the reality is that sometimes, we have plural `Gitlab` connections
and plural `Jenkins` connections.
+
+At this time, to support calculations that rely on multiple data sources like
`Dora`, we need to correctly associate different
[Scope](../Glossary.md#data-scope) under different data sources through a
method.
+
+So, we created `Project`
+
+`Project` is **a set of [Scope](../Glossary.md#data-scope) from different
domains**, a way to group different resources, and it is crucial for some
metric calculation like `Dora`.
+
+By adding the `project_name` field to the `blueprint` table, we associate
`project` with `blueprint` one-to-one.
+
+As a developer, when we are doing a new plugin development, or when we do some
other operations that need to be associated with `Project` to add, delete, and
modify operations, we need to have a detailed understanding of `Project`.
+
+Next, let us introduce `Project` in the following order:
+- `Project` related models
+- Related APIs that can be used to manipulate `Project` models
+- The interface that needs to be implemented when developing various plugins
to support the `Project`.
+ - The interface that needs to be implemented to develop the `Data
Source Plugin`
+ - The interface that needs to be implemented to develop the `Metrics
Plugin`
+
+# Models
+
+To support project we contains the following three models:
+ - `projects` describes a project object, including its name, creation and
update time and other basic information
+ - `project_metrics` describes what metrics plugin a project had enabled.
+ - `project_mapping` describes the mapping relationship of project and scope,
including the name of the project、table and row_id.
+
+## projects
+
+| **field** | **type** | **length** | **description** |
**key** |
+| ------------- | -------- | ---------- | ----------------------------- |
------- |
+| `name` | varchar | 255 | name for project | PK
|
+| `description` | longtext | | description of the project |
|
+| `created_at` | datetime | 3 | created time of project |
|
+| `updated_at` | datetime | 3 | last updated time of project |
|
+
+
+| **name** | **describe** | **created_at** |
**updated_at** |
+| --------- | ------------------------------------ | ----------------------- |
------------------------|
+| project_1 | this is one of the test projects | 2022-11-01 01:22:13.000 |
2022-11-01 02:24:15.000 |
+| project_2 | this is another project test project | 2022-11-01 01:23:29.000 |
2022-11-01 02:27:24.000 |
+
+## project_metrics
+
+| **field** | **type** | **length** | **description**
| **key** |
+| --------------- | -------- | ---------- |
---------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project
| PK |
+| `plugin_name` | varchar | 255 | name for plugin
| PK |
+| `plugin_option` | longtext | | check if metric plugins have been
enabled by the project | |
+| `enable` | tinyint | 1 | if the metric plugins is enabled
| |
+
+
+| **project_name** | **plugin_name** | **plugin_option** | **enable** |
+| ---------------- | --------------- | ----------------- | ---------- |
+| project_1 | dora | {} | true |
+| project_2 | dora | {} | false |
+
+## project_mapping
+
+| **field** | **type** | **length** | **description**
| **key** |
+| -------------- | -------- | ---------- |
------------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project
| PK |
+| `table` | varchar | 255 | the table name of
[Scope](../Glossary.md#data-scope) | PK |
+| `row_id` | varchar | 255 | the row_id in the
[Scope](../Glossary.md#data-scope) table | PK |
+
+
+| **project_name** | **table** | **row_id** |
+| ---------------- | --------- | ------------------------ |
+| project_1 | Repo | gitlab:GithubRepo:1:lake |
+| project_1 | Board | jira:JiraBoard:1:lake |
+| project_2 | Repo | github:GithubRepo:1:lake |
+
+# How to operate the table through the API
+
+By visiting `[Your Config-UI Host]/api/swagger/index.html`, you can go to the
swagger document of `Devlake` deployed. In this document, `framework/projects`
`framework/ProjectMetrics`, and `framework/plugins ` is the related API of
`project`.
+
+To protect the integrity of the data in the database as much as possible, we
recommend using the API to perform corresponding operations instead of directly
using SQL statements to operate the tables in the database.
+
+When we need to manipulate the `projects` table, we can use the
`framework/projects`:
+- We can create a new `project` object through the `POST` `/projects`
interface, which requires a `projectName` field as its name, which is unique.
+
+After we created the `project` object or we already had a `project`:
+- We can use the `GET /projects/:projectName` interface to obtain specific
`project` information. In addition to the name, this information also includes
the update time and creation time of the project, and will be associated with
all The relevant `project_metrics` and `project_mapping` information of the
current `project` will be returned after integration.
+- We can use the `PATCH /projects/:projectName` interface to update the
information of a specific `project`, such as updating its related `description`
information.
+- We can get all `project` information through the `GET /projects` interface.
The returned information can be filtered by setting the parameter `search`, and
the returned information can be paged by setting the parameters `page`, and
`pageSize`.
+
+After we created the `project` object or we already had a `project`:
+- we can use the `POST /project_metrics/:projectName/:pluginName` to create a
set of relationships between `project` and `Metrics Plugin`, Use the
`projectName` and `pluginName` we obtained earlier to build this relationship.
At the same time configure the corresponding `option` for it.
Review Comment:
it should be `/projects/:projectName/metrics`, we should update the code as
well.
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,320 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ `Project` is **a set of [Scope](../Glossary.md#data-scope) from different
domains**, a way to group different resources, and it is crucial for some
metric calculations like `Dora`.
+---
+
+# Summary
+
+In related calculations such as `Dora` indicators, we often encounter
situations requiring comprehensive calculations based on data from multiple
data sources.
+
+For example, we do code version management on `Gitlab`, and then do
corresponding compilation and deployment operations in `Jenkins`. At this time,
our indicator calculation will depend on the data of `Gitlab` and `Jenkins` at
the same time.
+
+However, the reality is that sometimes, we have plural `Gitlab` connections
and plural `Jenkins` connections.
+
+At this time, to support calculations that rely on multiple data sources like
`Dora`, we need to correctly associate different
[Scope](../Glossary.md#data-scope) under different data sources through a
method.
+
+So, we created `Project`
+
+`Project` is **a set of [Scope](../Glossary.md#data-scope) from different
domains**, a way to group different resources, and it is crucial for some
metric calculation like `Dora`.
+
+By adding the `project_name` field to the `blueprint` table, we associate
`project` with `blueprint` one-to-one.
+
+As a developer, when we are doing a new plugin development, or when we do some
other operations that need to be associated with `Project` to add, delete, and
modify operations, we need to have a detailed understanding of `Project`.
+
+Next, let us introduce `Project` in the following order:
+- `Project` related models
+- Related APIs that can be used to manipulate `Project` models
+- The interface that needs to be implemented when developing various plugins
to support the `Project`.
+ - The interface that needs to be implemented to develop the `Data
Source Plugin`
+ - The interface that needs to be implemented to develop the `Metrics
Plugin`
+
+# Models
+
+To support project we contains the following three models:
+ - `projects` describes a project object, including its name, creation and
update time and other basic information
+ - `project_metrics` describes what metrics plugin a project had enabled.
+ - `project_mapping` describes the mapping relationship of project and scope,
including the name of the project、table and row_id.
+
+## projects
+
+| **field** | **type** | **length** | **description** |
**key** |
+| ------------- | -------- | ---------- | ----------------------------- |
------- |
+| `name` | varchar | 255 | name for project | PK
|
+| `description` | longtext | | description of the project |
|
+| `created_at` | datetime | 3 | created time of project |
|
+| `updated_at` | datetime | 3 | last updated time of project |
|
+
+
+| **name** | **describe** | **created_at** |
**updated_at** |
+| --------- | ------------------------------------ | ----------------------- |
------------------------|
+| project_1 | this is one of the test projects | 2022-11-01 01:22:13.000 |
2022-11-01 02:24:15.000 |
+| project_2 | this is another project test project | 2022-11-01 01:23:29.000 |
2022-11-01 02:27:24.000 |
+
+## project_metrics
+
+| **field** | **type** | **length** | **description**
| **key** |
+| --------------- | -------- | ---------- |
---------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project
| PK |
+| `plugin_name` | varchar | 255 | name for plugin
| PK |
+| `plugin_option` | longtext | | check if metric plugins have been
enabled by the project | |
+| `enable` | tinyint | 1 | if the metric plugins is enabled
| |
+
+
+| **project_name** | **plugin_name** | **plugin_option** | **enable** |
+| ---------------- | --------------- | ----------------- | ---------- |
+| project_1 | dora | {} | true |
+| project_2 | dora | {} | false |
+
+## project_mapping
+
+| **field** | **type** | **length** | **description**
| **key** |
+| -------------- | -------- | ---------- |
------------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project
| PK |
+| `table` | varchar | 255 | the table name of
[Scope](../Glossary.md#data-scope) | PK |
+| `row_id` | varchar | 255 | the row_id in the
[Scope](../Glossary.md#data-scope) table | PK |
+
+
+| **project_name** | **table** | **row_id** |
+| ---------------- | --------- | ------------------------ |
+| project_1 | Repo | gitlab:GithubRepo:1:lake |
+| project_1 | Board | jira:JiraBoard:1:lake |
+| project_2 | Repo | github:GithubRepo:1:lake |
+
+# How to operate the table through the API
+
+By visiting `[Your Config-UI Host]/api/swagger/index.html`, you can go to the
swagger document of `Devlake` deployed. In this document, `framework/projects`
`framework/ProjectMetrics`, and `framework/plugins ` is the related API of
`project`.
+
+To protect the integrity of the data in the database as much as possible, we
recommend using the API to perform corresponding operations instead of directly
using SQL statements to operate the tables in the database.
+
+When we need to manipulate the `projects` table, we can use the
`framework/projects`:
+- We can create a new `project` object through the `POST` `/projects`
interface, which requires a `projectName` field as its name, which is unique.
+
+After we created the `project` object or we already had a `project`:
+- We can use the `GET /projects/:projectName` interface to obtain specific
`project` information. In addition to the name, this information also includes
the update time and creation time of the project, and will be associated with
all The relevant `project_metrics` and `project_mapping` information of the
current `project` will be returned after integration.
+- We can use the `PATCH /projects/:projectName` interface to update the
information of a specific `project`, such as updating its related `description`
information.
+- We can get all `project` information through the `GET /projects` interface.
The returned information can be filtered by setting the parameter `search`, and
the returned information can be paged by setting the parameters `page`, and
`pageSize`.
+
+After we created the `project` object or we already had a `project`:
+- we can use the `POST /project_metrics/:projectName/:pluginName` to create a
set of relationships between `project` and `Metrics Plugin`, Use the
`projectName` and `pluginName` we obtained earlier to build this relationship.
At the same time configure the corresponding `option` for it.
+
+After we create a `project_metrics` relation or we already had one:
+- We can use the `GET /project_metrics/:projectName/:pluginName` to get a set
of `project_metrics` information.
+- We can use the `PATCH /project_metrics/:projectName/:pluginName` to update
the `project_metrics` information.
+
+If we need to know which plugins are `Metrics Plugin`:
+- we can use the `GET` `/plugins`, it will return a list of all plugin names
and information about the plugins and `Metrics Plugin`.
+
+# The interface that needs to be implemented
+
+We divide plugins into three categories
+- The first category is `Base Plugin`, such as `core`, `helper`, etc. These
plug-ins provide basic support services for other plug-ins.
Review Comment:
no such thing as `Base Plugin`. `core` and `helper` are not plugins.
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,320 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ `Project` is **a set of [Scope](../Glossary.md#data-scope) from different
domains**, a way to group different resources, and it is crucial for some
metric calculations like `Dora`.
+---
+
+# Summary
+
+In related calculations such as `Dora` indicators, we often encounter
situations requiring comprehensive calculations based on data from multiple
data sources.
+
+For example, we do code version management on `Gitlab`, and then do
corresponding compilation and deployment operations in `Jenkins`. At this time,
our indicator calculation will depend on the data of `Gitlab` and `Jenkins` at
the same time.
+
+However, the reality is that sometimes, we have plural `Gitlab` connections
and plural `Jenkins` connections.
Review Comment:
However, in most cases, we have multiple Gitlab Projects / Jenkins Jobs that
belong to different teams/agendas in our Apache DevLake database.
To distinguish them into different groups. The `Project` is introduced in
v0.15. Essentially, a project consists of a set of
[Scopes](../Glossary.md#data-scope), i.e., a couple of Gitlab Projects, Jira
Boards or Jenkins Jobs, etc.
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,320 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ `Project` is **a set of [Scope](../Glossary.md#data-scope) from different
domains**, a way to group different resources, and it is crucial for some
metric calculations like `Dora`.
+---
+
+# Summary
+
+In related calculations such as `Dora` indicators, we often encounter
situations requiring comprehensive calculations based on data from multiple
data sources.
+
+For example, we do code version management on `Gitlab`, and then do
corresponding compilation and deployment operations in `Jenkins`. At this time,
our indicator calculation will depend on the data of `Gitlab` and `Jenkins` at
the same time.
+
+However, the reality is that sometimes, we have plural `Gitlab` connections
and plural `Jenkins` connections.
+
+At this time, to support calculations that rely on multiple data sources like
`Dora`, we need to correctly associate different
[Scope](../Glossary.md#data-scope) under different data sources through a
method.
+
+So, we created `Project`
+
+`Project` is **a set of [Scope](../Glossary.md#data-scope) from different
domains**, a way to group different resources, and it is crucial for some
metric calculation like `Dora`.
+
+By adding the `project_name` field to the `blueprint` table, we associate
`project` with `blueprint` one-to-one.
Review Comment:
change "one-to-one" to "in one-to-one manner"
this sentence is describe the relationship between `project` and
`blueprint`, which should be moved to the Models section.
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,320 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ `Project` is **a set of [Scope](../Glossary.md#data-scope) from different
domains**, a way to group different resources, and it is crucial for some
metric calculations like `Dora`.
+---
+
+# Summary
+
+In related calculations such as `Dora` indicators, we often encounter
situations requiring comprehensive calculations based on data from multiple
data sources.
+
+For example, we do code version management on `Gitlab`, and then do
corresponding compilation and deployment operations in `Jenkins`. At this time,
our indicator calculation will depend on the data of `Gitlab` and `Jenkins` at
the same time.
+
+However, the reality is that sometimes, we have plural `Gitlab` connections
and plural `Jenkins` connections.
+
+At this time, to support calculations that rely on multiple data sources like
`Dora`, we need to correctly associate different
[Scope](../Glossary.md#data-scope) under different data sources through a
method.
Review Comment:
delete
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,320 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ `Project` is **a set of [Scope](../Glossary.md#data-scope) from different
domains**, a way to group different resources, and it is crucial for some
metric calculations like `Dora`.
+---
+
+# Summary
+
+In related calculations such as `Dora` indicators, we often encounter
situations requiring comprehensive calculations based on data from multiple
data sources.
+
+For example, we do code version management on `Gitlab`, and then do
corresponding compilation and deployment operations in `Jenkins`. At this time,
our indicator calculation will depend on the data of `Gitlab` and `Jenkins` at
the same time.
+
+However, the reality is that sometimes, we have plural `Gitlab` connections
and plural `Jenkins` connections.
+
+At this time, to support calculations that rely on multiple data sources like
`Dora`, we need to correctly associate different
[Scope](../Glossary.md#data-scope) under different data sources through a
method.
+
+So, we created `Project`
+
+`Project` is **a set of [Scope](../Glossary.md#data-scope) from different
domains**, a way to group different resources, and it is crucial for some
metric calculation like `Dora`.
+
+By adding the `project_name` field to the `blueprint` table, we associate
`project` with `blueprint` one-to-one.
+
+As a developer, when we are doing a new plugin development, or when we do some
other operations that need to be associated with `Project` to add, delete, and
modify operations, we need to have a detailed understanding of `Project`.
+
+Next, let us introduce `Project` in the following order:
+- `Project` related models
+- Related APIs that can be used to manipulate `Project` models
+- The interface that needs to be implemented when developing various plugins
to support the `Project`.
+ - The interface that needs to be implemented to develop the `Data
Source Plugin`
+ - The interface that needs to be implemented to develop the `Metrics
Plugin`
+
+# Models
+
+To support project we contains the following three models:
+ - `projects` describes a project object, including its name, creation and
update time and other basic information
+ - `project_metrics` describes what metrics plugin a project had enabled.
+ - `project_mapping` describes the mapping relationship of project and scope,
including the name of the project、table and row_id.
+
+## projects
+
+| **field** | **type** | **length** | **description** |
**key** |
+| ------------- | -------- | ---------- | ----------------------------- |
------- |
+| `name` | varchar | 255 | name for project | PK
|
+| `description` | longtext | | description of the project |
|
+| `created_at` | datetime | 3 | created time of project |
|
+| `updated_at` | datetime | 3 | last updated time of project |
|
+
+
+| **name** | **describe** | **created_at** |
**updated_at** |
+| --------- | ------------------------------------ | ----------------------- |
------------------------|
+| project_1 | this is one of the test projects | 2022-11-01 01:22:13.000 |
2022-11-01 02:24:15.000 |
+| project_2 | this is another project test project | 2022-11-01 01:23:29.000 |
2022-11-01 02:27:24.000 |
+
+## project_metrics
+
+| **field** | **type** | **length** | **description**
| **key** |
+| --------------- | -------- | ---------- |
---------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project
| PK |
+| `plugin_name` | varchar | 255 | name for plugin
| PK |
+| `plugin_option` | longtext | | check if metric plugins have been
enabled by the project | |
+| `enable` | tinyint | 1 | if the metric plugins is enabled
| |
+
+
+| **project_name** | **plugin_name** | **plugin_option** | **enable** |
+| ---------------- | --------------- | ----------------- | ---------- |
+| project_1 | dora | {} | true |
+| project_2 | dora | {} | false |
+
+## project_mapping
+
+| **field** | **type** | **length** | **description**
| **key** |
+| -------------- | -------- | ---------- |
------------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project
| PK |
+| `table` | varchar | 255 | the table name of
[Scope](../Glossary.md#data-scope) | PK |
+| `row_id` | varchar | 255 | the row_id in the
[Scope](../Glossary.md#data-scope) table | PK |
+
+
+| **project_name** | **table** | **row_id** |
+| ---------------- | --------- | ------------------------ |
+| project_1 | Repo | gitlab:GithubRepo:1:lake |
+| project_1 | Board | jira:JiraBoard:1:lake |
+| project_2 | Repo | github:GithubRepo:1:lake |
+
+# How to operate the table through the API
Review Comment:
rename suggestion: "How to manage project via API"
and I don't think we need to be so specific (line 87~107)
Just tell developers to check the swagger doc(with a link) and lay out the
major endpoints would be good enough:
For API specification, please check the [swagger
doc](how-to-view-the-swagger-doc-link)
Related endpoints:
1. `/projects`
2. `/projects/:projectName/metrics`
3. `/plugins`
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,320 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ `Project` is **a set of [Scope](../Glossary.md#data-scope) from different
domains**, a way to group different resources, and it is crucial for some
metric calculations like `Dora`.
+---
+
+# Summary
+
+In related calculations such as `Dora` indicators, we often encounter
situations requiring comprehensive calculations based on data from multiple
data sources.
+
+For example, we do code version management on `Gitlab`, and then do
corresponding compilation and deployment operations in `Jenkins`. At this time,
our indicator calculation will depend on the data of `Gitlab` and `Jenkins` at
the same time.
+
+However, the reality is that sometimes, we have plural `Gitlab` connections
and plural `Jenkins` connections.
+
+At this time, to support calculations that rely on multiple data sources like
`Dora`, we need to correctly associate different
[Scope](../Glossary.md#data-scope) under different data sources through a
method.
+
+So, we created `Project`
+
+`Project` is **a set of [Scope](../Glossary.md#data-scope) from different
domains**, a way to group different resources, and it is crucial for some
metric calculation like `Dora`.
+
+By adding the `project_name` field to the `blueprint` table, we associate
`project` with `blueprint` one-to-one.
+
+As a developer, when we are doing a new plugin development, or when we do some
other operations that need to be associated with `Project` to add, delete, and
modify operations, we need to have a detailed understanding of `Project`.
+
+Next, let us introduce `Project` in the following order:
+- `Project` related models
+- Related APIs that can be used to manipulate `Project` models
+- The interface that needs to be implemented when developing various plugins
to support the `Project`.
+ - The interface that needs to be implemented to develop the `Data
Source Plugin`
+ - The interface that needs to be implemented to develop the `Metrics
Plugin`
+
+# Models
+
+To support project we contains the following three models:
+ - `projects` describes a project object, including its name, creation and
update time and other basic information
+ - `project_metrics` describes what metrics plugin a project had enabled.
+ - `project_mapping` describes the mapping relationship of project and scope,
including the name of the project、table and row_id.
+
+## projects
+
+| **field** | **type** | **length** | **description** |
**key** |
+| ------------- | -------- | ---------- | ----------------------------- |
------- |
+| `name` | varchar | 255 | name for project | PK
|
+| `description` | longtext | | description of the project |
|
+| `created_at` | datetime | 3 | created time of project |
|
+| `updated_at` | datetime | 3 | last updated time of project |
|
+
+
+| **name** | **describe** | **created_at** |
**updated_at** |
+| --------- | ------------------------------------ | ----------------------- |
------------------------|
+| project_1 | this is one of the test projects | 2022-11-01 01:22:13.000 |
2022-11-01 02:24:15.000 |
+| project_2 | this is another project test project | 2022-11-01 01:23:29.000 |
2022-11-01 02:27:24.000 |
+
+## project_metrics
+
+| **field** | **type** | **length** | **description**
| **key** |
+| --------------- | -------- | ---------- |
---------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project
| PK |
+| `plugin_name` | varchar | 255 | name for plugin
| PK |
+| `plugin_option` | longtext | | check if metric plugins have been
enabled by the project | |
+| `enable` | tinyint | 1 | if the metric plugins is enabled
| |
+
+
+| **project_name** | **plugin_name** | **plugin_option** | **enable** |
+| ---------------- | --------------- | ----------------- | ---------- |
+| project_1 | dora | {} | true |
+| project_2 | dora | {} | false |
+
+## project_mapping
+
+| **field** | **type** | **length** | **description**
| **key** |
+| -------------- | -------- | ---------- |
------------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project
| PK |
+| `table` | varchar | 255 | the table name of
[Scope](../Glossary.md#data-scope) | PK |
+| `row_id` | varchar | 255 | the row_id in the
[Scope](../Glossary.md#data-scope) table | PK |
+
+
+| **project_name** | **table** | **row_id** |
+| ---------------- | --------- | ------------------------ |
+| project_1 | Repo | gitlab:GithubRepo:1:lake |
+| project_1 | Board | jira:JiraBoard:1:lake |
+| project_2 | Repo | github:GithubRepo:1:lake |
+
+# How to operate the table through the API
+
+By visiting `[Your Config-UI Host]/api/swagger/index.html`, you can go to the
swagger document of `Devlake` deployed. In this document, `framework/projects`
`framework/ProjectMetrics`, and `framework/plugins ` is the related API of
`project`.
+
+To protect the integrity of the data in the database as much as possible, we
recommend using the API to perform corresponding operations instead of directly
using SQL statements to operate the tables in the database.
+
+When we need to manipulate the `projects` table, we can use the
`framework/projects`:
+- We can create a new `project` object through the `POST` `/projects`
interface, which requires a `projectName` field as its name, which is unique.
+
+After we created the `project` object or we already had a `project`:
+- We can use the `GET /projects/:projectName` interface to obtain specific
`project` information. In addition to the name, this information also includes
the update time and creation time of the project, and will be associated with
all The relevant `project_metrics` and `project_mapping` information of the
current `project` will be returned after integration.
+- We can use the `PATCH /projects/:projectName` interface to update the
information of a specific `project`, such as updating its related `description`
information.
+- We can get all `project` information through the `GET /projects` interface.
The returned information can be filtered by setting the parameter `search`, and
the returned information can be paged by setting the parameters `page`, and
`pageSize`.
+
+After we created the `project` object or we already had a `project`:
+- we can use the `POST /project_metrics/:projectName/:pluginName` to create a
set of relationships between `project` and `Metrics Plugin`, Use the
`projectName` and `pluginName` we obtained earlier to build this relationship.
At the same time configure the corresponding `option` for it.
+
+After we create a `project_metrics` relation or we already had one:
+- We can use the `GET /project_metrics/:projectName/:pluginName` to get a set
of `project_metrics` information.
+- We can use the `PATCH /project_metrics/:projectName/:pluginName` to update
the `project_metrics` information.
+
+If we need to know which plugins are `Metrics Plugin`:
+- we can use the `GET` `/plugins`, it will return a list of all plugin names
and information about the plugins and `Metrics Plugin`.
+
+# The interface that needs to be implemented
+
+We divide plugins into three categories
+- The first category is `Base Plugin`, such as `core`, `helper`, etc. These
plug-ins provide basic support services for other plug-ins.
+- The second category is `Data Source Plugin`, such as `GitLab` `GitHub`
`Jira` `Jenkins`, etc. These plugins collect data from various data sources
+- The third category is `Metrics Plugin`, such as `Dora`, etc. These plug-ins
do not directly contact the data source but do secondary calculations based on
+
+the collected data after the `Data Source Plugin` works
+
+The second and third types of plug-ins here need to implement the
corresponding interface to support the project.
+
+## Data Source Plugin
+
+`Data Source Plugin` is a plugin used to extract data from a data source.
+
+For example `GitLab` `GitHub` `Jira` `Jenkins` etc.
+
+These plugins, from various data sources, extract data into the database and
store them, they deal directly with the data source, so we classify them as
`Data Source Plugin`.
+
+## the DataSourcePluginBlueprintV200 interface
Review Comment:
Move to the "Interfaces" section as well.
No need to be so detailed, laying out so much code is confusing.
We describe the big picture here and ask the developer to check the source
code for detail, there are comments laying around inside the source file.
##########
docs/DeveloperManuals/Project.md:
##########
@@ -0,0 +1,320 @@
+---
+title: "Project"
+sidebar_position: 5
+description: >
+ `Project` is **a set of [Scope](../Glossary.md#data-scope) from different
domains**, a way to group different resources, and it is crucial for some
metric calculations like `Dora`.
+---
+
+# Summary
+
+In related calculations such as `Dora` indicators, we often encounter
situations requiring comprehensive calculations based on data from multiple
data sources.
+
+For example, we do code version management on `Gitlab`, and then do
corresponding compilation and deployment operations in `Jenkins`. At this time,
our indicator calculation will depend on the data of `Gitlab` and `Jenkins` at
the same time.
+
+However, the reality is that sometimes, we have plural `Gitlab` connections
and plural `Jenkins` connections.
+
+At this time, to support calculations that rely on multiple data sources like
`Dora`, we need to correctly associate different
[Scope](../Glossary.md#data-scope) under different data sources through a
method.
+
+So, we created `Project`
+
+`Project` is **a set of [Scope](../Glossary.md#data-scope) from different
domains**, a way to group different resources, and it is crucial for some
metric calculation like `Dora`.
+
+By adding the `project_name` field to the `blueprint` table, we associate
`project` with `blueprint` one-to-one.
+
+As a developer, when we are doing a new plugin development, or when we do some
other operations that need to be associated with `Project` to add, delete, and
modify operations, we need to have a detailed understanding of `Project`.
+
+Next, let us introduce `Project` in the following order:
+- `Project` related models
+- Related APIs that can be used to manipulate `Project` models
+- The interface that needs to be implemented when developing various plugins
to support the `Project`.
+ - The interface that needs to be implemented to develop the `Data
Source Plugin`
+ - The interface that needs to be implemented to develop the `Metrics
Plugin`
+
+# Models
+
+To support project we contains the following three models:
+ - `projects` describes a project object, including its name, creation and
update time and other basic information
+ - `project_metrics` describes what metrics plugin a project had enabled.
+ - `project_mapping` describes the mapping relationship of project and scope,
including the name of the project、table and row_id.
+
+## projects
+
+| **field** | **type** | **length** | **description** |
**key** |
+| ------------- | -------- | ---------- | ----------------------------- |
------- |
+| `name` | varchar | 255 | name for project | PK
|
+| `description` | longtext | | description of the project |
|
+| `created_at` | datetime | 3 | created time of project |
|
+| `updated_at` | datetime | 3 | last updated time of project |
|
+
+
+| **name** | **describe** | **created_at** |
**updated_at** |
+| --------- | ------------------------------------ | ----------------------- |
------------------------|
+| project_1 | this is one of the test projects | 2022-11-01 01:22:13.000 |
2022-11-01 02:24:15.000 |
+| project_2 | this is another project test project | 2022-11-01 01:23:29.000 |
2022-11-01 02:27:24.000 |
+
+## project_metrics
+
+| **field** | **type** | **length** | **description**
| **key** |
+| --------------- | -------- | ---------- |
---------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project
| PK |
+| `plugin_name` | varchar | 255 | name for plugin
| PK |
+| `plugin_option` | longtext | | check if metric plugins have been
enabled by the project | |
+| `enable` | tinyint | 1 | if the metric plugins is enabled
| |
+
+
+| **project_name** | **plugin_name** | **plugin_option** | **enable** |
+| ---------------- | --------------- | ----------------- | ---------- |
+| project_1 | dora | {} | true |
+| project_2 | dora | {} | false |
+
+## project_mapping
+
+| **field** | **type** | **length** | **description**
| **key** |
+| -------------- | -------- | ---------- |
------------------------------------------------------------- | ------- |
+| `project_name` | varchar | 255 | name for project
| PK |
+| `table` | varchar | 255 | the table name of
[Scope](../Glossary.md#data-scope) | PK |
+| `row_id` | varchar | 255 | the row_id in the
[Scope](../Glossary.md#data-scope) table | PK |
+
+
+| **project_name** | **table** | **row_id** |
+| ---------------- | --------- | ------------------------ |
+| project_1 | Repo | gitlab:GithubRepo:1:lake |
+| project_1 | Board | jira:JiraBoard:1:lake |
+| project_2 | Repo | github:GithubRepo:1:lake |
+
+# How to operate the table through the API
+
+By visiting `[Your Config-UI Host]/api/swagger/index.html`, you can go to the
swagger document of `Devlake` deployed. In this document, `framework/projects`
`framework/ProjectMetrics`, and `framework/plugins ` is the related API of
`project`.
+
+To protect the integrity of the data in the database as much as possible, we
recommend using the API to perform corresponding operations instead of directly
using SQL statements to operate the tables in the database.
+
+When we need to manipulate the `projects` table, we can use the
`framework/projects`:
+- We can create a new `project` object through the `POST` `/projects`
interface, which requires a `projectName` field as its name, which is unique.
+
+After we created the `project` object or we already had a `project`:
+- We can use the `GET /projects/:projectName` interface to obtain specific
`project` information. In addition to the name, this information also includes
the update time and creation time of the project, and will be associated with
all The relevant `project_metrics` and `project_mapping` information of the
current `project` will be returned after integration.
+- We can use the `PATCH /projects/:projectName` interface to update the
information of a specific `project`, such as updating its related `description`
information.
+- We can get all `project` information through the `GET /projects` interface.
The returned information can be filtered by setting the parameter `search`, and
the returned information can be paged by setting the parameters `page`, and
`pageSize`.
+
+After we created the `project` object or we already had a `project`:
+- we can use the `POST /project_metrics/:projectName/:pluginName` to create a
set of relationships between `project` and `Metrics Plugin`, Use the
`projectName` and `pluginName` we obtained earlier to build this relationship.
At the same time configure the corresponding `option` for it.
+
+After we create a `project_metrics` relation or we already had one:
+- We can use the `GET /project_metrics/:projectName/:pluginName` to get a set
of `project_metrics` information.
+- We can use the `PATCH /project_metrics/:projectName/:pluginName` to update
the `project_metrics` information.
+
+If we need to know which plugins are `Metrics Plugin`:
+- we can use the `GET` `/plugins`, it will return a list of all plugin names
and information about the plugins and `Metrics Plugin`.
+
+# The interface that needs to be implemented
+
+We divide plugins into three categories
+- The first category is `Base Plugin`, such as `core`, `helper`, etc. These
plug-ins provide basic support services for other plug-ins.
+- The second category is `Data Source Plugin`, such as `GitLab` `GitHub`
`Jira` `Jenkins`, etc. These plugins collect data from various data sources
Review Comment:
Combine this information(Line133-134) to the "Interface" section
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
