This is an automated email from the ASF dual-hosted git repository.
zky pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-devlake-website.git
The following commit(s) were added to refs/heads/main by this push:
new 6c0695a8d2 Update KeyConcepts.md (#445)
6c0695a8d2 is described below
commit 6c0695a8d2965c6457f8c19f4fbe2bdc8b5c4d55
Author: Xuanlin Bao <[email protected]>
AuthorDate: Mon Feb 27 21:32:31 2023 +0800
Update KeyConcepts.md (#445)
* Update KeyConcepts.md
According to offical architecture document, the data flow should be raw
data->tool layer tables->domain layer tables.
As a result, roles in pipelines shoue be
collectors->extractors->converters->enrichers.
Among them, the function of extractors should be extract data from raw data
table to **tool layer tables** rather than extract data from raw data table to
**domain layer tables**.
* Update ToolLayerSchema.md
Naming convension of tool layer schema tables should be
`_tool_{plugin}_{entity}` rather than `_raw_{plugin}_{entity}`.
* Update KeyConcepts.md
The function of extractors is extracting data from `raw data tables` to
`tool layer tables`.
* Update ToolLayerSchema.md
Naming convension of tool layer schema tables should be
`_tool_{plugin}_{entity}` rather than `_raw_{plugin}_{entity}`.
* Update KeyConcepts.md
The function of `Extractors` is extracting data from `raw data tables` to
`tool layer tables`.
---
docs/DataModels/ToolLayerSchema.md | 2 +-
docs/Overview/KeyConcepts.md | 2 +-
versioned_docs/version-v0.15/DataModels/ToolLayerSchema.md | 2 +-
versioned_docs/version-v0.15/Overview/KeyConcepts.md | 2 +-
4 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/docs/DataModels/ToolLayerSchema.md
b/docs/DataModels/ToolLayerSchema.md
index 889e1a23c8..9d846e5d3b 100644
--- a/docs/DataModels/ToolLayerSchema.md
+++ b/docs/DataModels/ToolLayerSchema.md
@@ -19,7 +19,7 @@ As a user, you can check tool data tables to verify data
quality if you have con
## Data Models
-Tool layer tables start with a prefix `_tool_`. Each plugin contains multiple
tool data tables, the naming convension of these tables is
`_raw_{plugin}_{entity}`. For instance,
+Tool layer tables start with a prefix `_tool_`. Each plugin contains multiple
tool data tables, the naming convension of these tables is
`_tool_{plugin}_{entity}`. For instance,
- _tool_jira_issues
- _tool_jira_boards
- _tool_jira_board_issues`
diff --git a/docs/Overview/KeyConcepts.md b/docs/Overview/KeyConcepts.md
index 55f04a64fe..aef52e781a 100644
--- a/docs/Overview/KeyConcepts.md
+++ b/docs/Overview/KeyConcepts.md
@@ -105,6 +105,6 @@ Notice: **You can manually orchestrate the pipeline in
Configuration UI Advanced
### Subtasks
**A subtask is the minimal work unit in a pipeline that performs in any of the
four roles: `Collectors`, `Extractors`, `Converters` and `Enrichers`.**
Subtasks are executed in sequential orders.
- `Collectors`: Collect raw data from data sources, normally via DevLake API
and stored into `raw data table`
-- `Extractors`: Extract data from `raw data table` to `domain layer tables`
+- `Extractors`: Extract data from `raw data tables` to `tool layer tables`
- `Converters`: Convert data from `tool layer tables` into `domain layer
tables`
- `Enrichers`: Enrich data from one domain to other domains. For instance, the
Fourier Transformation can examine `issue_changelog` to show time distribution
of an issue on every assignee.
diff --git a/versioned_docs/version-v0.15/DataModels/ToolLayerSchema.md
b/versioned_docs/version-v0.15/DataModels/ToolLayerSchema.md
index 889e1a23c8..9d846e5d3b 100644
--- a/versioned_docs/version-v0.15/DataModels/ToolLayerSchema.md
+++ b/versioned_docs/version-v0.15/DataModels/ToolLayerSchema.md
@@ -19,7 +19,7 @@ As a user, you can check tool data tables to verify data
quality if you have con
## Data Models
-Tool layer tables start with a prefix `_tool_`. Each plugin contains multiple
tool data tables, the naming convension of these tables is
`_raw_{plugin}_{entity}`. For instance,
+Tool layer tables start with a prefix `_tool_`. Each plugin contains multiple
tool data tables, the naming convension of these tables is
`_tool_{plugin}_{entity}`. For instance,
- _tool_jira_issues
- _tool_jira_boards
- _tool_jira_board_issues`
diff --git a/versioned_docs/version-v0.15/Overview/KeyConcepts.md
b/versioned_docs/version-v0.15/Overview/KeyConcepts.md
index aa011c1ae6..5ad1c4696d 100644
--- a/versioned_docs/version-v0.15/Overview/KeyConcepts.md
+++ b/versioned_docs/version-v0.15/Overview/KeyConcepts.md
@@ -105,6 +105,6 @@ Notice: **You can manually orchestrate the pipeline in
Configuration UI Advanced
### Subtasks
**A subtask is the minimal work unit in a pipeline that performs in any of the
four roles: `Collectors`, `Extractors`, `Converters` and `Enrichers`.**
Subtasks are executed in sequential orders.
- `Collectors`: Collect raw data from data sources, normally via DevLake API
and stored into `raw data table`
-- `Extractors`: Extract data from `raw data table` to `domain layer tables`
+- `Extractors`: Extract data from `raw data tables` to `tool layer tables`
- `Converters`: Convert data from `tool layer tables` into `domain layer
tables`
- `Enrichers`: Enrich data from one domain to other domains. For instance, the
Fourier Transformation can examine `issue_changelog` to show time distribution
of an issue on every assignee.
