This is an automated email from the ASF dual-hosted git repository.
hez pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-devlake.git
The following commit(s) were added to refs/heads/main by this push:
new 3e390fc15 [doc-4391]: Rework developer's manual docs (#5674)
3e390fc15 is described below
commit 3e390fc15faeeda02c49eb0f0c2cf2a851f45fbf
Author: Keon Amini <[email protected]>
AuthorDate: Tue Jul 25 03:42:41 2023 +0330
[doc-4391]: Rework developer's manual docs (#5674)
* refactor: remove plugin code generator
* docs: developer's manual
* refactor: remove old mermaid diagram
* refactor: rename the manual file
---
README.md | 4 +-
backend/DevelopmentManual.md | 191 +++++++++++++++++++++
backend/generator/README.md | 45 -----
backend/generator/cmd/create_collector.go | 153 -----------------
backend/generator/cmd/create_extractor.go | 131 --------------
backend/generator/cmd/create_migration.go | 135 ---------------
backend/generator/cmd/create_plugin.go | 155 -----------------
backend/generator/cmd/e2e_raw_create.go | 104 -----------
backend/generator/cmd/generator_doc.go | 36 ----
backend/generator/cmd/init_migration.go | 89 ----------
backend/generator/cmd/root.go | 77 ---------
backend/generator/docs/generator.md | 24 ---
backend/generator/docs/generator_completion.md | 32 ----
.../generator/docs/generator_completion_bash.md | 50 ------
.../generator/docs/generator_completion_fish.md | 41 -----
.../docs/generator_completion_powershell.md | 38 ----
backend/generator/docs/generator_completion_zsh.md | 52 ------
.../generator/docs/generator_create-collector.md | 32 ----
backend/generator/docs/generator_create-e2e-raw.md | 32 ----
.../generator/docs/generator_create-extractor.md | 32 ----
.../generator/docs/generator_create-migration.md | 33 ----
backend/generator/docs/generator_create-plugin.md | 31 ----
backend/generator/docs/generator_generator-doc.md | 26 ---
backend/generator/docs/generator_init-migration.md | 32 ----
backend/generator/main.go | 29 ----
.../migrationscripts/add_init_tables.go-template | 41 -----
.../archived/connection.go-template | 16 --
.../migrationscripts/migration.go-template | 41 -----
.../migration_with_config.go-template | 47 -----
.../template/migrationscripts/register.go-template | 27 ---
.../template/plugin/api/blueprint.go-template | 69 --------
.../template/plugin/api/connection.go-template | 167 ------------------
.../generator/template/plugin/api/init.go-template | 37 ----
.../plugin/impl/impl_complete_plugin.go-template | 134 ---------------
.../template/plugin/models/connection.go-template | 51 ------
.../template/plugin/plugin_main.go-template | 90 ----------
.../plugin/plugin_main_complete_plugin.go-template | 43 -----
.../template/plugin/tasks/api_client.go-template | 74 --------
.../plugin/tasks/api_collector.go-template | 75 --------
.../template/plugin/tasks/extractor.go-template | 56 ------
.../template/plugin/tasks/shared.go-template | 41 -----
.../template/plugin/tasks/task_data.go-template | 32 ----
.../tasks/task_data_complete_plugin.go-template | 70 --------
backend/generator/util/template.go | 103 -----------
backend/python/README.md | 2 +-
45 files changed, 194 insertions(+), 2626 deletions(-)
diff --git a/README.md b/README.md
index 0f09a456b..be959622c 100644
--- a/README.md
+++ b/README.md
@@ -74,7 +74,7 @@ Please read the [contribution
guidelines](https://devlake.apache.org/community)
- [Submit a
PR](https://devlake.apache.org/community/make-contribution/development-workflow):
Start with [good first
issues](https://github.com/apache/incubator-devlake/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22)
or [issues with no
assignees](https://github.com/apache/incubator-devlake/issues?q=is%3Aissue+is%3Aopen+no%3Aassignee)
- [Join Mailing list](https://devlake.apache.org/community/subscribe):
Initiate or participate in project discussions on the mailing list
- [Write a
Blog](https://devlake.apache.org/community/make-contribution/BlogSubmission):
Write a blog to share your use cases about Apache DevLake
-- [Contribute a
Plugin](https://devlake.apache.org/docs/DeveloperManuals/PluginImplementation):
[Add a
plugin](https://github.com/apache/incubator-devlake/issues?q=is%3Aissue+is%3Aopen+label%3Aadd-a-plugin+)
to integrate Apache DevLake with more data sources for the community
+- [Develop a Plugin](./backend/DevelopmentSetup.md): [Add a
plugin](https://github.com/apache/incubator-devlake/issues?q=is%3Aissue+is%3Aopen+label%3Aadd-a-plugin+)
to integrate Apache DevLake with more data sources for the community
## 📄 Contributing to Documentation:
@@ -87,7 +87,7 @@ Please read the [contribution
guidelines](https://devlake.apache.org/community)
## 💙 Community
-- Slack: Message us on <a
href="https://join.slack.com/t/devlake-io/shared_invite/zt-18uayb6ut-cHOjiYcBwERQ8VVPZ9cQQw";
target="_blank">Slack</a>
+- Slack: Message us on <a
href="https://join.slack.com/t/devlake-io/shared_invite/zt-18uayb6ut-cHOjiYcBwERQ8VVPZ9cQQw";
target="_blank">Slack</a>
- Wechat Community: [Check the
barcode](resources/img/wechat_community_barcode.png)
## 📄 License<a id="license"></a>
diff --git a/backend/DevelopmentManual.md b/backend/DevelopmentManual.md
new file mode 100644
index 000000000..d4a695737
--- /dev/null
+++ b/backend/DevelopmentManual.md
@@ -0,0 +1,191 @@
+# Plugin Implementation
+
+If your favorite DevOps tool is not yet supported by DevLake, don't worry.
It's not difficult to implement a DevLake plugin. In this post, we'll go
through the basics of DevLake plugins and build an example plugin from scratch
together.
+
+## What is a plugin?
+
+A DevLake plugin is a library that hooks into DevLake core at run-time.
+
+A plugin may extend DevLake's capability in three ways:
+
+1. Integrating with new data sources
+2. Transforming/enriching existing data
+3. Exporting DevLake data to other data systems
+
+## Types of plugins
+
+There are, as of now, support for two types of plugins:
+
+1. __*Go plugins*__: These are the primary type of plugins used by DevLake,
and require the developer to write the Go code starting from fetching
(collecting) data from data sources to converting them into our normalized data
models and storing them.
+These are shared libraries built with Go's `plugin` package and are hooked
into DevLake at runtime.
+2. __*Python Plugins*__: These plugins serve the same purpose but are written
fully in Python. They are conceptually the same but obviously different in
terms of implementation. See [this manual](./python/README.md) for details on
them.
+They are **all** hooked into DevLake at runtime and are communicated with
using RPC calls (based on shell calls as of now) into Python abstracted away by
the framework.
+
+
+## How do plugins work?
+
+A plugin mainly consists of a collection of subtasks that can be executed by
the DevLake framework. For data source plugins, a subtask may be collecting a
single entity from the data source (e.g., issues from Jira).
+Besides the subtasks, there are hooks that a plugin can implement to customize
its initialization, migration, and more. See
[here](#Plugin-package-and-code-structure) for details.
+
+## Overview of developing a plugin in Go
+
+In this section, we will walk you through one of our existing Go plugins with
the goal that by the end you will have
+an idea of the code you need to write for your own plugin. Most plugins follow
the same code structure and layout, so,
+as long as you understand one, you should have sufficient knowledge to tackle
and figure out the others.
+
+First, lets talk about the package structure. Each Go plugin is placed in the
directory `plugins/<plugin-name>`. The plugins
+are meant to be independent of each other, so no two plugin packages should
reference each other. The `core` and `helpers` packages
+contain the compile-time dependencies (within DevLake) that plugins may use.
The `server` package contains the logic to bootstrap the
+DevLake server and load the plugins at runtime. It should *not* be called by
any of the plugins. The `impls` package contains implementations
+of core interfaces and should be avoided to be used at compile-time as well.
+
+Go plugins are compiled as *.so files and loaded in at runtime by the server,
however they may also be compiled with the server code for **testing purposes**,
+and we will show how that can be done [later](#Testing); doing so has the
advantage of speeding up development for more rapid testing.
+
+If you look at the plugins listed under `plugins/`, you may notice some
familiar names, such as `github`, `jira`, `gitlab`, `jenkins`, etc.
+These are "datasource" plugins; they do what you expect: they pull data from
these datasources and after some processing and transformations (a.k.a.
extraction and conversion), we store
+them in the DevLake database. There are other types of plugins as well, such
`gitextractor` and `refdiff`; these are "helper" plugins: they are meant to run
+in conjunction with these datasource plugins and often perform additional
post-processing. An important "helper" plugin is the `dora` plugin which is
responsible
+for DORA metric calculations on the collected data.
+
+For this tutorial, we will use the
[gitlab](https://github.com/apache/incubator-DevLake/tree/main/backend/plugins/gitlab)
plugin as reference.
+
+### Plugin package and code structure
+
+The plugins generally break down into the following packages:
+* `api`: This package contains the API endpoints (REST) that the plugin
exposes. These endpoints expose CRUD operations on
[connections](https://github.com/apache/incubator-DevLake/blob/main/backend/plugins/gitlab/api/connection.go),
+[scopes](https://github.com/apache/incubator-DevLake/blob/main/backend/plugins/gitlab/api/scope.go),
[scope-configs](https://github.com/apache/incubator-DevLake/blob/main/backend/plugins/gitlab/api/scope_config.go),
and possibly more (such as
[remote-scopes](https://github.com/apache/incubator-DevLake/blob/main/backend/plugins/gitlab/api/remote.go)
to fetch raw scopes from the datasource if applicable).
+Note the documentation structure of the functions in these example: that's the
syntax for Go-Swagger which is responsible for their Swagger documentation
generation.
+
+ This is where we also define the logic used by the server to run the plugin
based on the constraints set by a blueprint (the user's configuration); that is,
+a "pipeline plan", which defines what input data to feed into the plugin,
which subtasks (see `tasks` package) to execute, and which other plugins to run
after this plugin's execution and the input to each of them (typically "helper"
plugins).
+See
[this](https://github.com/apache/incubator-DevLake/blob/main/backend/plugins/gitlab/api/blueprint_v200.go)
for reference.
+* `e2e`: This package contains "pseudo-integration-tests" for plugins; tests
that use mocked data and test the plugin's extraction and conversion logic of
that data against an actual database. The test data is formatted in CSV files
structured based on the expected database table structures.
+* `impl`: This package is where the plugin is actually defined, and is the
"entrypoint": it is the implementation of multiple interfaces. See
[this](https://github.com/apache/incubator-DevLake/tree/main/backend/plugins/gitlab/impl)
for example.
+ * At a minimum, a plugin must implement
[PluginMeta](https://github.com/apache/incubator-DevLake/blob/main/backend/core/plugin/plugin_meta.go).
+ * Bootstrap logic for the plugin (on server startup) is defined by
implementing
[PluginInit](https://github.com/apache/incubator-DevLake/blob/main/backend/core/plugin/plugin_init.go).Init().
This will typically initialize global variables accessible by the plugin.
+ * APIs exposed by the plugin in the `api` package are also registered here
by implementing
[PluginApi](https://github.com/apache/incubator-DevLake/blob/main/backend/core/plugin/plugin_api.go).ApiResources().
You will nearly almost need this.
+ * The plugin's core execution logic is defined by implementing the methods
on
[PluginTask](https://github.com/apache/incubator-DevLake/blob/main/backend/core/plugin/plugin_task.go).
You will nearly almost need this.
+ * The plugin's database migration logic (table definitions and their
alterations) is registered via implementing
[PluginMigration](https://github.com/apache/incubator-DevLake/blob/main/backend/core/plugin/plugin_migration.go).MigrationScripts().
You will nearly almost need this.
+ * The plugin's Pipeline-Plan definition logic is registered by implementing
[DataSourcePluginBlueprintV200](https://github.com/apache/incubator-DevLake/blob/main/backend/core/plugin/plugin_blueprint.go).
You will need this if you're writing a datasource-based plugin.
+ * You will need to implement
[PluginSource](https://github.com/apache/incubator-DevLake/blob/main/backend/core/plugin/plugin_meta.go)
if your plugin has APIs for one or more of Connections, Scopes and
ScopeConfigs. If one or more is not applicable, the respective method should
return `nil`.
+ * You will need to implement
[PluginModel](https://github.com/apache/incubator-DevLake/blob/main/backend/core/plugin/plugin_model.go)
if your plugin has any database-based models (practically always the case).
You will have to explicitly list out such models, which will be defined in the
`models` package. We have an automated test `plugins/table_info_test.go` to
+ detect if any models are missing from this list (and will fail the CICD
execution).
+* `models`: This is where all the database models for the plugin are used.
These are called "tool" models, and their respective table-names are to be
prefixed with `_tool_`. A subpackage, `migrationscripts` defines the
database DDL logic for setting and altering the tables for these models. These
are called migrations, and they are applied sequentially based on their
`Version()` on server startup, since the last successful migration.
+ See [gitlab's
migrations](https://github.com/apache/incubator-DevLake/tree/main/backend/plugins/gitlab/models/migrationscripts).
+* `tasks`: This package contains all the functions needed by
[PluginTask](https://github.com/apache/incubator-DevLake/blob/main/backend/core/plugin/plugin_task.go).SubTaskMetas():
+These are the collectors, extractors and convertors of each datasource entity
that you want to support. More will be said about these below.
+
+### Collectors
+
+Collectors fetch data from a data-source and store that raw data in the
appropriate \_raw_ table in the DevLake database. The process of fetching is
+more involved than it sounds, and is greatly influenced by the quality of the
API you're interacting with. Things to think about are:
+- Does the API support returning paginated results? You should always prefer
that, if supported, for optimal performance.
+- Does the API support incremental data collection? DevLake supports
"bookmarking" fetched data based on a time-parameter on the data (stateful
collectors), which can be
+defined. This allows us to refresh only newer data and not have to fetch
everything from scratch. But again, the API needs to honor fetching data on
+such a basis.
+
+See
[this](https://github.com/apache/incubator-DevLake/blob/main/backend/plugins/gitlab/tasks/issue_collector.go)
for an example of a stateful collector.
+Here's
[one](https://github.com/apache/incubator-DevLake/blob/main/backend/plugins/gitlab/tasks/account_collector.go)
for a stateless collector. Be sure to
+read the in-code documentation of the api-clients used in these examples to
familiarize yourself with their technicalities, capabilities and/or limitations.
+
+
+### Extractors
+
+Extractors are called after the collectors are done. They convert the raw data
from collectors to "tool" models. Tool Models are DevLake's representation of
+a data-source's data. They need not have 1:1 correspondence with them as we
are usually interested in a subset of the fields. This is also where the
settings of the
+configured `ScopeConfig` (if applicable) get applied: that is, this is where
user-defined transformations get applied to the certain fields on the raw data
to set
+certain fields on the target tool model.
[Here's](https://github.com/apache/incubator-DevLake/blob/main/backend/plugins/gitlab/tasks/issue_extractor.go)
an
+example of an extractor.
+
+
+### Convertors
+
+Convertors convert the tool models emitted by extractors to "domain models".
Domain models are models agnostic in form to any data source. They are generic,
+standardized representations of various concepts. For example, a generic
"commit", or a generic "issue". With respect to DevLake's dashboard queries, we
+write queries against the database tables of these domain models.
+You will need to know to what domain model(s) a given tool model should map
(Read about them
[here](https://DevLake.apache.org/docs/next/DataModels/DevLakeDomainLayerSchema/)).
+This also depends on the requirements of your plugin, of course. See [this
package](https://github.com/apache/incubator-DevLake/tree/main/backend/core/models/domainlayer)
for a list of all the domain models;
+they are logically distributed into packages that represent different entities
(e.g. Code, Devops, Ticket, etc).
+See
[here](https://github.com/apache/incubator-DevLake/blob/main/backend/plugins/gitlab/tasks/issue_convertor.go)
for an example of a convertor.
+
+<br>
+<br>
+We have covered all the components of a DevLake plugin. Happy coding!
+
+### Testing
+
+There are three types of test strategies:
+1. Unit-tests: these are component level tests that mock dependencies. They
typically exist next to the source file they're testing.
+2. E2E-tests: these test plugin extractor and convertor subtasks by using
faked data to simulate the result of data collection.
+3. Integration-tests: these test the entire DevLake server as a whole. A [Go
Client](https://github.com/apache/incubator-devlake/blob/main/backend/test/helper/api.go)
([example
initialization](https://github.com/apache/incubator-devlake/blob/main/backend/test/helper/client_factory.go))
+has been written to either interact with an existing DevLake server (via its
APIs) or spin up an in-memory instance of DevLake for the purpose of testing
(note you need to have
+a separate database instance running). We have some integration tests written
for some of our plugins (see
[this](https://github.com/apache/incubator-devlake/tree/main/backend/test/e2e/manual)),
which is the best
+source to learn how to write these tests and interact with the client. Note
that the in-memory DevLake instance directly compiles the
+plugins you specify and in doing so saves a significant amount of time and
overhead
([example](https://github.com/apache/incubator-devlake/blob/d717d2aa897742ab4789b9a44e9e5a4c1e28adcf/backend/test/e2e/manual/gitlab/gitlab_test.go#L62)).
+
+We highly encourage you to leverage the Go client to perform quick tests as
you write either framework-level or plugin code.
+
+#### Full-scale testing using the client
+
+You may also leverage the Go client to set up a full DevLake test environment.
Start off by writing a test function with a body similar to this:
+```go
+func TestDevlakeServer(t *testing.T) {
+ // set up any environment variable you need for the server here
+ setupEnvironmentVars()
+ _ = helper.ConnectLocalServer(t, &helper.LocalClientConfig{
+ ServerPort: 8080,
+ DbURL:
"mysql://merico:[email protected]:3306/lake?charset=utf8mb4&parseTime=True",
//DO NOT USE localhost - it breaks Python
+ CreateServer: true, // start an in-memory server
+ TruncateDb: false, // get rid of all existing data in the
DevLake database before startup
+ DropDb: false, // drop all tables in the DevLake database
before startup
+ Plugins: []plugin.PluginMeta{ // list of plugins to make
available to the server. Just create an empty instance of your intended plugin
structs (conventionally declared in their respective `impl.go`)
+ github.Github{},
+ githubGraphql.GithubGraphql{},
+ gitextractor.GitExtractor{}, // this requires that your
host machine has libgit2 installed, otherwise you'll get compilation errors
+ gitlab.Gitlab(""),
+ jenkins.Jenkins{},
+ webhook.Webhook{},
+ pagerduty.PagerDuty{},
+ tapd.Tapd{},
+ //refdiff.RefDiff{}, // not needed to be specified - already
included by default
+ //dora.Dora{}, // not needed to be specified - already included by
default
+ //org.Org{}, // not needed to be specified - already included by
default
+ },
+ })
+ time.Sleep(math.MaxInt)
+}
+
+// example env variables to enable Python-debugging
+func setupEnvironmentVars() {
+ os.Setenv("USE_PYTHON_DEBUGGER", "pycharm")
+ // The Debug host is your host-IP as seen by the IDE. This is usually just
"localhost", but will be different if you're launching your IDE from WSL,
+ // which you'd need if developing on Windows. In that case, it is the
IP of the WSL vEthernet network interface (e.g. 192.168.0.1).
+ // Make sure, additionally, that the firewall is not blocking network
access to your IDE - inbound/outbound.
+ os.Setenv("PYTHON_DEBUG_HOST", "localhost")
+ os.Setenv("PYTHON_DEBUG_PORT", "32000")
+}
+```
+Adjust the declared list of plugins and other parameters of the constructor
based on your needs. This spins up a server on your host machine
+on port 8080, and runs indefinitely.
+Again, make sure your database instance is already running (likely on Docker).
In the above, it is accessible via 127.0.0.1:3306.
+
+The next step is to set up the config-ui container. You will want to spin up
the container with the env variable `DEVLAKE_ENDPOINT: ${HOST_IP}:8080`,
+where `{HOST_IP}` is the IP of the host machine as seen by the docker
containers. Typically, on newer versions of Docker, adding this to the
config-ui's
+docker-compose config should suffice:
+```yaml
+ extra_hosts:
+ - "host.docker.internal:host-gateway"
+```
+In which case the `{HOST_IP}` will be `host.docker.internal`. This however may
not work on all operating systems, and you'll have to do
+some research to find out how to access the host from within docker.
+
+Once you get this done, the config-ui container will be able to talk to the
in-memory DevLake server that you spun up on your machine; you may start using
the UI as normal.
+It is highly recommended that you use an IDE to spin up the server. It
provides much more convenience, and allows for simple debugging using
breakpoints.
+
+Since some of this configuration requires writing custom code/config, it's
recommended you write them to files that will be ignored by git (using
.gitignore), so you
+don't accidentally end up pushing them upstream.
+
+#### Submit the code as open source code
+We encourage ideas and contributions ~ Let's use migration scripts, domain
layers and other discussed concepts to write normative and
+platform-neutral code. More info at
[here](https://DevLake.apache.org/docs/DataModels/DevLakeDomainLayerSchema) or
contact us for ebullient help.
diff --git a/backend/generator/README.md b/backend/generator/README.md
deleted file mode 100644
index f029c3171..000000000
--- a/backend/generator/README.md
+++ /dev/null
@@ -1,45 +0,0 @@
-# Apache DevLake Cli Tool -- Code Generator
-
-## How to use?
-
-Just run by `go run`:
-
-```bash
-go run generator/main.go [command]
-```
-
-Help also integrate in it:
-
-```bash
-go run generator/main.go help
-```
-
-## Plugin Related
-
-* [create-collector](docs/generator_create-collector.md) - Create a new
collector
-* [create-extractor](docs/generator_create-extractor.md) - Create a new
extractor
-* [create-plugin](docs/generator_create-plugin.md)
-
-Usage Gif:
-
-
-## Migration Related
-
-* [init-migration](docs/generator_init-migration.md) - Init migration
for plugin
-* [create-migration](docs/generator_create-migration.md) - Create a new
migration
-
-Usage Gif:
-
-
-## E2E Test Related
-
-* [create-e2e-raw](docs/generator_create-e2e-raw.md) - Export raw
table to csv in E2E test
-
-Usage Gif:
-
-
-## Others
-
-* [completion](docs/generator_completion.md) - Generate the
autocompletion script for the specified shell
-* [generator-doc](docs/generator_generator-doc.md) - generate document
for generator
-* [global options](docs/generator.md)
diff --git a/backend/generator/cmd/create_collector.go
b/backend/generator/cmd/create_collector.go
deleted file mode 100644
index cec2ade3f..000000000
--- a/backend/generator/cmd/create_collector.go
+++ /dev/null
@@ -1,153 +0,0 @@
-/*
-Licensed to the Apache Software Foundation (ASF) under one or more
-contributor license agreements. See the NOTICE file distributed with
-this work for additional information regarding copyright ownership.
-The ASF licenses this file to You under the Apache License, Version 2.0
-(the "License"); you may not use this file except in compliance with
-the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-package cmd
-
-import (
- "fmt"
- "github.com/apache/incubator-devlake/core/errors"
- "github.com/apache/incubator-devlake/generator/util"
- "github.com/iancoleman/strcase"
- "os"
- "path/filepath"
- "regexp"
- "strings"
-
- "github.com/manifoldco/promptui"
- "github.com/spf13/cobra"
-)
-
-func init() {
- rootCmd.AddCommand(createCollectorCmd)
-}
-
-func collectorNameNotExistValidateHoc(pluginName string) promptui.ValidateFunc
{
- collectorNameValidate := func(input string) error {
- if input == `` {
- return errors.Default.New("please input which data
would you will collect (snake_format)")
- }
- snakeNameReg := regexp.MustCompile(`^[A-Za-z][A-Za-z0-9_]*$`)
- if !snakeNameReg.MatchString(input) {
- return errors.Default.New("collector name invalid
(start with a-z and consist with a-z0-9_)")
- }
- _, err := os.Stat(filepath.Join(`plugins`, pluginName, `tasks`,
input+`_collector.go`))
- if os.IsNotExist(err) {
- return nil
- }
- if err != nil {
- return errors.Default.Wrap(err, "error getting
collector src file")
- }
- return errors.Default.New("collector exists")
- }
- return collectorNameValidate
-}
-
-func collectorNameExistValidateHoc(pluginName string) promptui.ValidateFunc {
- collectorNameValidate := func(input string) error {
- if input == `` {
- return errors.Default.New("please input which data
would you will collect (snake_format)")
- }
- _, err := os.Stat(filepath.Join(`plugins`, pluginName, `tasks`,
input+`_collector.go`))
- if err != nil {
- return errors.Default.Wrap(err, "error getting
collector src file")
- }
- return nil
- }
- return collectorNameValidate
-}
-
-var createCollectorCmd = &cobra.Command{
- Use: "create-collector [plugin_name] [collector_name]",
- Short: "Create a new collector",
- Long: `Create a new collector
-Type in what the name of collector is, then generator will create a new
collector in plugins/$plugin_name/tasks/$collector_name for you`,
- Run: func(cmd *cobra.Command, args []string) {
- var pluginName string
- var collectorName string
- var err error
-
- // try to get plugin name and collector name
- if len(args) > 0 {
- pluginName = args[0]
- }
- prompt := promptui.Prompt{
- Label: "plugin_name",
- Validate: pluginNameExistValidate(),
- Default: pluginName,
- }
- pluginName, err = prompt.Run()
- cobra.CheckErr(err)
- pluginName = strings.ToLower(pluginName)
-
- if len(args) > 1 {
- collectorName = args[1]
- }
- prompt = promptui.Prompt{
- Label: "collector_data_name",
- Validate: collectorNameNotExistValidateHoc(pluginName),
- Default: collectorName,
- }
- collectorName, err = prompt.Run()
- cobra.CheckErr(err)
- collectorName = strings.ToLower(collectorName)
-
- prompt = promptui.Prompt{
- Label: "http_path",
- Validate: func(input string) error {
- if input == `` {
- return errors.BadInput.New("http_path
required")
- }
- if strings.HasPrefix(input, `/`) {
- return errors.BadInput.New("http_path
shouldn't start with '/'")
- }
- return nil
- },
- }
- httpPath, err := prompt.Run()
- cobra.CheckErr(err)
-
- // read template
- templates := map[string]string{
- collectorName + `_collector.go`:
util.ReadTemplate("generator/template/plugin/tasks/api_collector.go-template"),
- }
-
- // create vars
- values := map[string]string{}
- util.GenerateAllFormatVar(values, `plugin_name`, pluginName)
- util.GenerateAllFormatVar(values, `collector_data_name`,
collectorName)
- values[`HttpPath`] = httpPath
- collectorDataNameUpperCamel := strcase.ToCamel(collectorName)
- values = util.DetectExistVars(templates, values)
- println(`vars in template:`, fmt.Sprint(values))
-
- // write template
- util.ReplaceVarInTemplates(templates, values)
- util.WriteTemplates(filepath.Join(`plugins`, pluginName,
`tasks`), templates)
- if modifyExistCode {
- util.ReplaceVarInFile(
- filepath.Join(`plugins`, pluginName,
`impl/impl.go`),
- regexp.MustCompile(`(return
+\[]core\.SubTaskMeta ?\{ ?\n?)((\s*[\w.]+,\n)*)(\s*})`),
- fmt.Sprintf("$1$2\t\ttasks.Collect%sMeta,\n$4",
collectorDataNameUpperCamel),
- )
- util.ReplaceVarInFile(
- filepath.Join(`plugins`, pluginName,
fmt.Sprintf(`%s.go`, pluginName)),
- regexp.MustCompile(`(return
+\[]core\.SubTaskMeta ?\{ ?\n?)((\s*[\w.]+,\n)*)(\s*})`),
- fmt.Sprintf("$1$2\t\ttasks.Collect%sMeta,\n$4",
collectorDataNameUpperCamel),
- )
- }
- },
-}
diff --git a/backend/generator/cmd/create_extractor.go
b/backend/generator/cmd/create_extractor.go
deleted file mode 100644
index 52aaefcd9..000000000
--- a/backend/generator/cmd/create_extractor.go
+++ /dev/null
@@ -1,131 +0,0 @@
-/*
-Licensed to the Apache Software Foundation (ASF) under one or more
-contributor license agreements. See the NOTICE file distributed with
-this work for additional information regarding copyright ownership.
-The ASF licenses this file to You under the Apache License, Version 2.0
-(the "License"); you may not use this file except in compliance with
-the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-package cmd
-
-import (
- "fmt"
- "github.com/apache/incubator-devlake/core/errors"
- "github.com/apache/incubator-devlake/generator/util"
- "github.com/iancoleman/strcase"
- "github.com/manifoldco/promptui"
- "github.com/spf13/cobra"
- "os"
- "path/filepath"
- "regexp"
- "strings"
-)
-
-func init() {
- rootCmd.AddCommand(createExtractorCmd)
-}
-
-func extractorNameNotExistValidateHoc(pluginName string) promptui.ValidateFunc
{
- extractorNameValidate := func(input string) error {
- if input == `` {
- return errors.Default.New("please input which data
would you will extract (snake_format)")
- }
- snakeNameReg := regexp.MustCompile(`^[A-Za-z][A-Za-z0-9_]*$`)
- if !snakeNameReg.MatchString(input) {
- return errors.Default.New("extractor name invalid
(start with a-z and consist with a-z0-9_)")
- }
- _, err := os.Stat(filepath.Join(`plugins`, pluginName, `tasks`,
input+`_extractor.go`))
- if os.IsNotExist(err) {
- return nil
- }
- if err != nil {
- return err
- }
- return errors.Default.New("extractor exists")
- }
- return extractorNameValidate
-}
-
-var createExtractorCmd = &cobra.Command{
- Use: "create-extractor [plugin_name] [extractor_name]",
- Short: "Create a new extractor",
- Long: `Create a new extractor
-Type in what the name of extractor is, then generator will create a new
extractor in plugins/$plugin_name/tasks/$extractor_name for you`,
- Run: func(cmd *cobra.Command, args []string) {
- var pluginName string
- var extractorName string
- var err error
-
- // try to get plugin name and extractor name
- if len(args) > 0 {
- pluginName = args[0]
- }
- prompt := promptui.Prompt{
- Label: "plugin_name",
- Validate: pluginNameExistValidate(),
- Default: pluginName,
- }
- pluginName, err = prompt.Run()
- cobra.CheckErr(err)
- pluginName = strings.ToLower(pluginName)
-
- prompt = promptui.Prompt{
- Label: "collector_name",
- Validate: collectorNameExistValidateHoc(pluginName),
- }
- collectorName, err := prompt.Run()
- cobra.CheckErr(err)
- collectorName = strings.ToLower(collectorName)
-
- if len(args) > 1 {
- extractorName = args[1]
- }
- prompt = promptui.Prompt{
- Label: "extractor_name",
- Validate: extractorNameNotExistValidateHoc(pluginName),
- Default: extractorName,
- }
- extractorName, err = prompt.Run()
- cobra.CheckErr(err)
- extractorName = strings.ToLower(extractorName)
-
- // read template
- templates := map[string]string{
- extractorName + `_extractor.go`:
util.ReadTemplate("generator/template/plugin/tasks/extractor.go-template"),
- }
-
- // create vars
- values := map[string]string{}
- util.GenerateAllFormatVar(values, `plugin_name`, pluginName)
- util.GenerateAllFormatVar(values, `collector_data_name`,
collectorName)
- util.GenerateAllFormatVar(values, `extractor_data_name`,
extractorName)
- extractorDataNameUpperCamel := strcase.ToCamel(extractorName)
- values = util.DetectExistVars(templates, values)
- println(`vars in template:`, fmt.Sprint(values))
-
- // write template
- util.ReplaceVarInTemplates(templates, values)
- util.WriteTemplates(filepath.Join(`plugins`, pluginName,
`tasks`), templates)
- if modifyExistCode {
- util.ReplaceVarInFile(
- filepath.Join(`plugins`, pluginName,
fmt.Sprintf(`%s.go`, pluginName)),
- regexp.MustCompile(`(return
+\[]core\.SubTaskMeta ?\{ ?\n?)((\s*[\w.]+,\n)*)(\s*})`),
- fmt.Sprintf("$1$2\t\ttasks.Extract%sMeta,\n$4",
extractorDataNameUpperCamel),
- )
- util.ReplaceVarInFile(
- filepath.Join(`plugins`, pluginName,
`impl/impl.go`),
- regexp.MustCompile(`(return
+\[]core\.SubTaskMeta ?\{ ?\n?)((\s*[\w.]+,\n)*)(\s*})`),
- fmt.Sprintf("$1$2\t\ttasks.Extract%sMeta,\n$4",
extractorDataNameUpperCamel),
- )
- }
- },
-}
diff --git a/backend/generator/cmd/create_migration.go
b/backend/generator/cmd/create_migration.go
deleted file mode 100644
index adfcffc1c..000000000
--- a/backend/generator/cmd/create_migration.go
+++ /dev/null
@@ -1,135 +0,0 @@
-/*
-Licensed to the Apache Software Foundation (ASF) under one or more
-contributor license agreements. See the NOTICE file distributed with
-this work for additional information regarding copyright ownership.
-The ASF licenses this file to You under the Apache License, Version 2.0
-(the "License"); you may not use this file except in compliance with
-the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-package cmd
-
-import (
- "fmt"
- "github.com/apache/incubator-devlake/core/errors"
- "github.com/apache/incubator-devlake/generator/util"
- "github.com/iancoleman/strcase"
- "os"
- "path/filepath"
- "regexp"
- "time"
-
- "github.com/manifoldco/promptui"
- "github.com/spf13/cobra"
-)
-
-func init() {
- rootCmd.AddCommand(createMigrationCmd)
-}
-
-var createMigrationCmd = &cobra.Command{
- Use: "create-migration [plugin_name/framework]",
- Short: "Create a new migration",
- Long: `Create a new migration
-Type in what the purpose of migration is, then generator will create a new
migration in plugins/$plugin_name/models/migrationscripts/$date_$purpose.go for
you.
-If framework passed, generator will create a new migration in
models/migrationscripts/$date_$purpose.go`,
- Run: func(cmd *cobra.Command, args []string) {
- var pluginName string
- var purpose string
- var err error
-
- // try to get plugin name
- if len(args) > 0 {
- pluginName = args[0]
- }
- if pluginName == `` {
- pluginItems, err := pluginNames(true)
- cobra.CheckErr(err)
- prompt := promptui.Select{
- Label: "plugin_name",
- Items: pluginItems,
- }
- var rawErr error
- _, pluginName, rawErr = prompt.Run()
- cobra.CheckErr(rawErr)
- }
-
- // check if migrationscripts inited
- var migrationPath string
- if pluginName == `framework` {
- migrationPath = filepath.Join(`models`,
`migrationscripts`)
- } else {
- migrationPath = filepath.Join(`plugins`, pluginName,
`models`, `migrationscripts`)
- }
- _, err = os.Stat(migrationPath)
- if os.IsNotExist(err) {
- cobra.CheckErr(errors.Default.New(`migrationscripts not
init. please run init-migration first`))
- }
- cobra.CheckErr(err)
-
- prompt := promptui.Prompt{
- Label: "purpose",
- Validate: purposeNotExistValidate(),
- }
- purpose, err = prompt.Run()
- cobra.CheckErr(err)
-
- selector := promptui.Select{
- Label: "with_config (is this migrations will use
config?)",
- Items: []string{"No", "Yes"},
- }
- _, withConfig, err := selector.Run()
- cobra.CheckErr(err)
-
- // create vars
- values := map[string]string{}
- values[`Date`] = time.Now().Format(`20060102`)
- values[`Purpose`] = strcase.ToLowerCamel(purpose)
- existMigrations, err := os.ReadDir(migrationPath)
- cobra.CheckErr(err)
- values[`Count`] = fmt.Sprintf(`%06d`, len(existMigrations))
-
- // read template
- templates := map[string]string{}
- if withConfig == `Yes` {
-
templates[values[`Date`]+`_`+strcase.ToSnake(values[`Purpose`])+`.go`] =
util.ReadTemplate("generator/template/migrationscripts/migration_with_config.go-template")
- } else {
-
templates[values[`Date`]+`_`+strcase.ToSnake(values[`Purpose`])+`.go`] =
util.ReadTemplate("generator/template/migrationscripts/migration.go-template")
- }
- values = util.DetectExistVars(templates, values)
- println(`vars in template:`, fmt.Sprint(values))
-
- // write template
- util.ReplaceVarInTemplates(templates, values)
- util.WriteTemplates(migrationPath, templates)
- if modifyExistCode {
- util.ReplaceVarInFile(
- filepath.Join(migrationPath, `register.go`),
- regexp.MustCompile(`(return
+\[]migration\.Script ?\{ ?\n?)((\s*[\w.()]+,\n?)*)(\s*})`),
- fmt.Sprintf("$1$2\t\tnew(%s),\n$4",
strcase.ToLowerCamel(values[`Purpose`])),
- )
- }
- },
-}
-
-func purposeNotExistValidate() promptui.ValidateFunc {
- return func(input string) error {
- if input == `` {
- return errors.Default.New("purpose require")
- }
- camelNameReg := regexp.MustCompile(`^[a-z][A-Za-z0-9]*$`)
- if !camelNameReg.MatchString(input) {
- return errors.Default.New("purpose invalid (please use
camelCase format, start with a-z and consist with a-zA-Z0-9)")
- }
-
- return nil
- }
-}
diff --git a/backend/generator/cmd/create_plugin.go
b/backend/generator/cmd/create_plugin.go
deleted file mode 100644
index 22f8adedf..000000000
--- a/backend/generator/cmd/create_plugin.go
+++ /dev/null
@@ -1,155 +0,0 @@
-/*
-Licensed to the Apache Software Foundation (ASF) under one or more
-contributor license agreements. See the NOTICE file distributed with
-this work for additional information regarding copyright ownership.
-The ASF licenses this file to You under the Apache License, Version 2.0
-(the "License"); you may not use this file except in compliance with
-the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-package cmd
-
-import (
- "fmt"
- "github.com/apache/incubator-devlake/core/errors"
- "github.com/apache/incubator-devlake/generator/util"
- "github.com/manifoldco/promptui"
- "github.com/spf13/cobra"
- "os"
- "regexp"
- "strings"
- "time"
-)
-
-func init() {
- rootCmd.AddCommand(createPluginCmd)
-}
-
-func pluginNameNotExistValidate() promptui.ValidateFunc {
- return func(input string) error {
- if input == `` {
- return errors.Default.New("plugin name require")
- }
- snakeNameReg := regexp.MustCompile(`^[A-Za-z][A-Za-z0-9_]*$`)
- if !snakeNameReg.MatchString(input) {
- return errors.Default.New("plugin name invalid (start
with a-z and consist with a-z0-9_)")
- }
- if strings.ToLower(input) == `framework` ||
strings.ToLower(input) == `core` || strings.ToLower(input) == `helper` {
- return errors.Default.New("plugin name cannot be
`framework` or `core` or `helper`")
- }
- _, err := os.Stat(`plugins/` + input)
- if os.IsNotExist(err) {
- return nil
- }
- if err != nil {
- return errors.Default.Wrap(err, "err getting plugin
path")
- }
- return errors.Default.New("plugin exists")
- }
-}
-
-func pluginNameExistValidate() promptui.ValidateFunc {
- return func(input string) error {
- if input == `` {
- return errors.Default.New("plugin name require")
- }
- _, err := os.Stat(`plugins/` + input)
- if err != nil {
- return errors.Default.Wrap(err, "error getting plugin
path")
- }
- return nil
- }
-}
-
-func pluginNames(withFramework bool) (pluginItems []string, _ errors.Error) {
- files, err := os.ReadDir(`plugins`)
- if err != nil {
- return nil, errors.Default.Wrap(err, "error reading plugins
directory")
- }
- if withFramework {
- pluginItems = append(pluginItems, `framework`)
- }
- for _, file := range files {
- if file.IsDir() {
- pluginItems = append(pluginItems, file.Name())
- }
- }
- return pluginItems, nil
-}
-
-var createPluginCmd = &cobra.Command{
- Use: "create-plugin [plugin_name]",
- Short: "Create a new plugin",
- Long: `Create a new plugin
-Type in what the name of plugin is, then generator will create a new plugin in
plugins/$plugin_name for you`,
- Run: func(cmd *cobra.Command, args []string) {
- var pluginName string
-
- // try to get plugin name
- if len(args) > 0 {
- pluginName = args[0]
- }
- err := pluginNameNotExistValidate()(pluginName)
- if err != nil {
- prompt := promptui.Prompt{
- Label: "plugin_name",
- Validate: pluginNameNotExistValidate(),
- Default: pluginName,
- }
- pluginName, err = prompt.Run()
- cobra.CheckErr(err)
- pluginName = strings.ToLower(pluginName)
- }
-
- prompt := promptui.Select{
- Label: "complete_plugin (Will this plugin request HTTP
APIs?)",
- Items: []string{"Yes", "No"},
- }
- _, withApiClient, err := prompt.Run()
- cobra.CheckErr(err)
-
- values := map[string]string{}
- templates := map[string]string{}
- if withApiClient == `Yes` {
- versionTimestamp := time.Now().Format(`20060102`)
- values[`Date`] = versionTimestamp
- // read template
- templates = map[string]string{
- fmt.Sprintf(`%s.go`, pluginName):
util.ReadTemplate("generator/template/plugin/plugin_main_complete_plugin.go-template"),
- `impl/impl.go`:
util.ReadTemplate("generator/template/plugin/impl/impl_complete_plugin.go-template"),
- `tasks/api_client.go`:
util.ReadTemplate("generator/template/plugin/tasks/api_client.go-template"),
- `tasks/task_data.go`:
util.ReadTemplate("generator/template/plugin/tasks/task_data_complete_plugin.go-template"),
- `tasks/shared.go`:
util.ReadTemplate("generator/template/plugin/tasks/shared.go-template"),
- `api/connection.go`:
util.ReadTemplate("generator/template/plugin/api/connection.go-template"),
- `models/connection.go`:
util.ReadTemplate("generator/template/plugin/models/connection.go-template"),
-
fmt.Sprintf("models/migrationscripts/%s_add_init_tables.go", versionTimestamp):
util.ReadTemplate("generator/template/migrationscripts/add_init_tables.go-template"),
- `models/migrationscripts/register.go`:
util.ReadTemplate("generator/template/migrationscripts/register.go-template"),
- `api/init.go`:
util.ReadTemplate("generator/template/plugin/api/init.go-template"),
- `api/blueprint.go`:
util.ReadTemplate("generator/template/plugin/api/blueprint.go-template"),
- }
- util.GenerateAllFormatVar(values, `plugin_name`,
pluginName)
- } else if withApiClient == `No` {
- // read template
- templates = map[string]string{
- `plugin_main.go`:
util.ReadTemplate("generator/template/plugin/plugin_main.go-template"),
- `tasks/task_data.go`:
util.ReadTemplate("generator/template/plugin/tasks/task_data.go-template"),
- }
- util.GenerateAllFormatVar(values, `plugin_name`,
pluginName)
- }
-
- values = util.DetectExistVars(templates, values)
- println(`vars in template:`, fmt.Sprint(values))
-
- // write template
- util.ReplaceVarInTemplates(templates, values)
- util.WriteTemplates(`plugins/`+pluginName, templates)
- },
-}
diff --git a/backend/generator/cmd/e2e_raw_create.go
b/backend/generator/cmd/e2e_raw_create.go
deleted file mode 100644
index e96ecb8dd..000000000
--- a/backend/generator/cmd/e2e_raw_create.go
+++ /dev/null
@@ -1,104 +0,0 @@
-/*
-Licensed to the Apache Software Foundation (ASF) under one or more
-contributor license agreements. See the NOTICE file distributed with
-this work for additional information regarding copyright ownership.
-The ASF licenses this file to You under the Apache License, Version 2.0
-(the "License"); you may not use this file except in compliance with
-the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-package cmd
-
-import (
- "github.com/apache/incubator-devlake/core/errors"
- "github.com/apache/incubator-devlake/helpers/e2ehelper"
- "github.com/manifoldco/promptui"
- "github.com/spf13/cobra"
- "path/filepath"
- "strings"
-)
-
-func init() {
- rootCmd.AddCommand(createE2eRawCmd)
-}
-
-var createE2eRawCmd = &cobra.Command{
- Use: "create-e2e-raw [plugin_name] [raw_table_name]",
- Short: "Create _raw_table.csv for e2e test",
- Long: `Create _raw_table.csv for e2e test
-Type in what the raw_table is, then generator will export and save in
plugins/$plugin_name/e2e/_raw_$raw_name.csv for you.`,
- Run: func(cmd *cobra.Command, args []string) {
- var pluginName, rawTableName, csvFileName string
- var err error
-
- // try to get plugin name
- if len(args) > 0 {
- pluginName = args[0]
- }
- if pluginName == `` {
- pluginItems, err := pluginNames(false)
- cobra.CheckErr(err)
- prompt := promptui.Select{
- Label: "plugin_name",
- Items: pluginItems,
- }
- _, pluginName, err = errors.Convert001(prompt.Run())
- cobra.CheckErr(err)
- }
-
- // try to get rawTableName
- if len(args) > 1 {
- rawTableName = args[1]
- }
- if rawTableName == `` {
- prompt := promptui.Prompt{
- Label: "raw_table_name",
- Default: `_raw_`,
- Validate: func(input string) error {
- if input == `` {
- return
errors.Default.New("raw_table_name require")
- }
- if !strings.HasPrefix(input, `_raw_`) {
- return
errors.Default.New("raw_table_name should start with `_raw_`")
- }
- return nil
- },
- }
- rawTableName, err = prompt.Run()
- cobra.CheckErr(err)
- }
-
- // try to get rawTableName
- prompt := promptui.Prompt{
- Label: "csv_file_name",
- Default: rawTableName + `.csv`,
- Validate: func(input string) error {
- if input == `` {
- return
errors.Default.New("csv_file_name require")
- }
- if !strings.HasSuffix(input, `.csv`) {
- return
errors.Default.New("csv_file_name should end with `.csv`")
- }
- return nil
- },
- }
- csvFileName, err = prompt.Run()
- cobra.CheckErr(err)
-
- rawTablesPath := filepath.Join(`plugins`, pluginName, `e2e`,
`raw_tables`)
- dataflowTester := e2ehelper.NewDataFlowTester(nil, "gitlab",
nil)
- dataflowTester.ExportRawTable(
- rawTableName,
- filepath.Join(rawTablesPath, csvFileName),
- )
- println(csvFileName, ` generated in `, rawTablesPath)
- },
-}
diff --git a/backend/generator/cmd/generator_doc.go
b/backend/generator/cmd/generator_doc.go
deleted file mode 100644
index 1bf14705a..000000000
--- a/backend/generator/cmd/generator_doc.go
+++ /dev/null
@@ -1,36 +0,0 @@
-/*
-Licensed to the Apache Software Foundation (ASF) under one or more
-contributor license agreements. See the NOTICE file distributed with
-this work for additional information regarding copyright ownership.
-The ASF licenses this file to You under the Apache License, Version 2.0
-(the "License"); you may not use this file except in compliance with
-the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-package cmd
-
-import (
- "github.com/spf13/cobra"
- "github.com/spf13/cobra/doc"
-)
-
-func init() {
- rootCmd.AddCommand(GeneratorDocCmd)
-}
-
-// GeneratorDocCmd FIXME ...
-var GeneratorDocCmd = &cobra.Command{
- Use: "generator-doc",
- Short: "generate document for generator",
- RunE: func(cmd *cobra.Command, args []string) error {
- return doc.GenMarkdownTree(rootCmd, "generator/docs")
- },
-}
diff --git a/backend/generator/cmd/init_migration.go
b/backend/generator/cmd/init_migration.go
deleted file mode 100644
index bfd089e0a..000000000
--- a/backend/generator/cmd/init_migration.go
+++ /dev/null
@@ -1,89 +0,0 @@
-/*
-Licensed to the Apache Software Foundation (ASF) under one or more
-contributor license agreements. See the NOTICE file distributed with
-this work for additional information regarding copyright ownership.
-The ASF licenses this file to You under the Apache License, Version 2.0
-(the "License"); you may not use this file except in compliance with
-the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-package cmd
-
-import (
- "fmt"
- "github.com/apache/incubator-devlake/core/errors"
- "github.com/apache/incubator-devlake/generator/util"
- "github.com/iancoleman/strcase"
- "os"
- "path/filepath"
- "time"
-
- "github.com/manifoldco/promptui"
- "github.com/spf13/cobra"
-)
-
-func init() {
- rootCmd.AddCommand(initMigrationCmd)
-}
-
-var initMigrationCmd = &cobra.Command{
- Use: "init-migration [plugin_name]",
- Short: "Init migration for plugin",
- Long: `Init migration for plugin
-Type in which plugin do you want init migrations in, then generator will
create a init migration in plugins/$plugin_name/models/migrationscripts/ for
you.`,
- Run: func(cmd *cobra.Command, args []string) {
- var pluginName string
-
- // try to get plugin name
- if len(args) > 0 {
- pluginName = args[0]
- }
- if pluginName == `` {
- pluginItems, err := pluginNames(false)
- cobra.CheckErr(err)
- prompt := promptui.Select{
- Label: "plugin_name",
- Items: pluginItems,
- }
- _, pluginName, err = errors.Convert001(prompt.Run())
- cobra.CheckErr(err)
- }
- migrationPath := filepath.Join(`plugins`, pluginName, `models`,
`migrationscripts`)
- _, err := os.Stat(migrationPath)
- if !os.IsNotExist(err) {
- cobra.CheckErr(errors.Default.New(`migrationscripts
inited or path read file`))
- }
-
- // create vars
- values := map[string]string{}
- util.GenerateAllFormatVar(values, `plugin_name`, pluginName)
- versionTimestamp := time.Now().Format(`20060102`)
-
- // read template
- templates := map[string]string{
- fmt.Sprintf("%s_add_init_tables.go", versionTimestamp):
util.ReadTemplate("generator/template/migrationscripts/add_init_tables.go-template"),
- `register.go`:
util.ReadTemplate("generator/template/migrationscripts/register.go-template"),
- `archived/.gitkeep`: ``,
- }
-
- values[`Date`] = versionTimestamp
- values = util.DetectExistVars(templates, values)
- println(`vars in template:`, fmt.Sprint(values))
-
- // write template
- util.ReplaceVarInTemplates(templates, values)
- util.WriteTemplates(migrationPath, templates)
- if modifyExistCode {
- println("Last Step: add some code in plugin to
implement Migratable like this:\n" +
- "func (p " + strcase.ToCamel(pluginName) + ")
MigrationScripts() []plugin.MigrationScript {\n\treturn
migrationscripts.All()\n}")
- }
- },
-}
diff --git a/backend/generator/cmd/root.go b/backend/generator/cmd/root.go
deleted file mode 100644
index 5d91bd0a9..000000000
--- a/backend/generator/cmd/root.go
+++ /dev/null
@@ -1,77 +0,0 @@
-/*
-Licensed to the Apache Software Foundation (ASF) under one or more
-contributor license agreements. See the NOTICE file distributed with
-this work for additional information regarding copyright ownership.
-The ASF licenses this file to You under the Apache License, Version 2.0
-(the "License"); you may not use this file except in compliance with
-the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-package cmd
-
-import (
- "fmt"
- "github.com/apache/incubator-devlake/core/errors"
- goerrors "github.com/go-errors/errors"
- "os"
-
- "github.com/spf13/cobra"
- "github.com/spf13/viper"
-)
-
-var (
- // Used for flags.
- cfgFile string
- modifyExistCode bool
-
- rootCmd = &cobra.Command{
- Use: `generator [command]`,
- Short: "Apache DevLake Cli Tool -- Code Generator",
- }
-)
-
-// Execute executes the root command.
-func Execute() errors.Error {
- return errors.Default.WrapRaw(rootCmd.Execute())
-}
-
-func init() {
- cobra.OnInitialize(initConfig)
-
- rootCmd.PersistentFlags().StringVar(&cfgFile, "config", "", "config
file (default is PROJECT/.env)")
- rootCmd.PersistentFlags().BoolVar(&modifyExistCode, "modifyExistCode",
true, "allow generator modify exist code")
-}
-
-func initConfig() {
- if cfgFile != "" {
- // Use config file from the flag.
- viper.SetConfigFile(cfgFile)
- } else {
- viper.AddConfigPath("..")
- viper.AddConfigPath(".")
- viper.SetConfigType("env")
- viper.SetConfigName(".env")
- }
-
- viper.AutomaticEnv()
-
- err := viper.ReadInConfig()
-
- notFound := &viper.ConfigFileNotFoundError{}
- switch {
- case err != nil && !goerrors.As(err, notFound):
- cobra.CheckErr(err)
- case err != nil && goerrors.As(err, notFound):
- // The config file is optional, we shouldn't exit when the
config is not found
- default:
- fmt.Fprintln(os.Stderr, "Using config file:",
viper.ConfigFileUsed())
- }
-}
diff --git a/backend/generator/docs/generator.md
b/backend/generator/docs/generator.md
deleted file mode 100644
index 196bc5737..000000000
--- a/backend/generator/docs/generator.md
+++ /dev/null
@@ -1,24 +0,0 @@
-## generator
-
-Apache DevLake Cli Tool -- Code Generator
-
-### Options
-
-```
- --config string config file (default is PROJECT/.env)
- -h, --help help for generator
- --modifyExistCode allow generator modify exist code (default true)
-```
-
-### SEE ALSO
-
-* [generator completion](generator_completion.md) - Generate the
autocompletion script for the specified shell
-* [generator create-collector](generator_create-collector.md) - Create a
new collector
-* [generator create-e2e-raw](generator_create-e2e-raw.md) - Create
_raw_table.csv for e2e test
-* [generator create-extractor](generator_create-extractor.md) - Create a
new extractor
-* [generator create-migration](generator_create-migration.md) - Create a
new migration
-* [generator create-plugin](generator_create-plugin.md) - Create a new
plugin
-* [generator generator-doc](generator_generator-doc.md) - generate
document for generator
-* [generator init-migration](generator_init-migration.md) - Init migration
for plugin
-
-###### Auto generated by spf13/cobra on 27-Jun-2022
diff --git a/backend/generator/docs/generator_completion.md
b/backend/generator/docs/generator_completion.md
deleted file mode 100644
index 7cdb7fb3e..000000000
--- a/backend/generator/docs/generator_completion.md
+++ /dev/null
@@ -1,32 +0,0 @@
-## generator completion
-
-Generate the autocompletion script for the specified shell
-
-### Synopsis
-
-Generate the autocompletion script for generator for the specified shell.
-See each sub-command's help for details on how to use the generated script.
-
-### Options
-
-```
- -h, --help help for completion
-```
-
-### Options inherited from parent commands
-
-```
- --config string config file (default is PROJECT/.env)
- --modifyExistCode allow generator modify exist code (default true)
-```
-
-### SEE ALSO
-
-* [generator](generator.md) - Apache DevLake Cli Tool -- Code Generator
-* [generator completion bash](generator_completion_bash.md) - Generate the
autocompletion script for bash
-* [generator completion fish](generator_completion_fish.md) - Generate the
autocompletion script for fish
-* [generator completion powershell](generator_completion_powershell.md) -
Generate the autocompletion script for
- powershell
-* [generator completion zsh](generator_completion_zsh.md) - Generate the
autocompletion script for zsh
-
-###### Auto generated by spf13/cobra on 24-Jun-2022
diff --git a/backend/generator/docs/generator_completion_bash.md
b/backend/generator/docs/generator_completion_bash.md
deleted file mode 100644
index b00476c30..000000000
--- a/backend/generator/docs/generator_completion_bash.md
+++ /dev/null
@@ -1,50 +0,0 @@
-## generator completion bash
-
-Generate the autocompletion script for bash
-
-### Synopsis
-
-Generate the autocompletion script for the bash shell.
-
-This script depends on the 'bash-completion' package.
-If it is not installed already, you can install it via your OS's package
manager.
-
-To load completions in your current shell session:
-
- source <(generator completion bash)
-
-To load completions for every new session, execute once:
-
-#### Linux:
-
- generator completion bash > /etc/bash_completion.d/generator
-
-#### macOS:
-
- generator completion bash > $(brew
--prefix)/etc/bash_completion.d/generator
-
-You will need to start a new shell for this setup to take effect.
-
-```
-generator completion bash
-```
-
-### Options
-
-```
- -h, --help help for bash
- --no-descriptions disable completion descriptions
-```
-
-### Options inherited from parent commands
-
-```
- --config string config file (default is PROJECT/.env)
- --modifyExistCode allow generator modify exist code (default true)
-```
-
-### SEE ALSO
-
-* [generator completion](generator_completion.md) - Generate the
autocompletion script for the specified shell
-
-###### Auto generated by spf13/cobra on 24-Jun-2022
diff --git a/backend/generator/docs/generator_completion_fish.md
b/backend/generator/docs/generator_completion_fish.md
deleted file mode 100644
index 7ac46dd1a..000000000
--- a/backend/generator/docs/generator_completion_fish.md
+++ /dev/null
@@ -1,41 +0,0 @@
-## generator completion fish
-
-Generate the autocompletion script for fish
-
-### Synopsis
-
-Generate the autocompletion script for the fish shell.
-
-To load completions in your current shell session:
-
- generator completion fish | source
-
-To load completions for every new session, execute once:
-
- generator completion fish > ~/.config/fish/completions/generator.fish
-
-You will need to start a new shell for this setup to take effect.
-
-```
-generator completion fish [flags]
-```
-
-### Options
-
-```
- -h, --help help for fish
- --no-descriptions disable completion descriptions
-```
-
-### Options inherited from parent commands
-
-```
- --config string config file (default is PROJECT/.env)
- --modifyExistCode allow generator modify exist code (default true)
-```
-
-### SEE ALSO
-
-* [generator completion](generator_completion.md) - Generate the
autocompletion script for the specified shell
-
-###### Auto generated by spf13/cobra on 24-Jun-2022
diff --git a/backend/generator/docs/generator_completion_powershell.md
b/backend/generator/docs/generator_completion_powershell.md
deleted file mode 100644
index dba88371a..000000000
--- a/backend/generator/docs/generator_completion_powershell.md
+++ /dev/null
@@ -1,38 +0,0 @@
-## generator completion powershell
-
-Generate the autocompletion script for powershell
-
-### Synopsis
-
-Generate the autocompletion script for powershell.
-
-To load completions in your current shell session:
-
- generator completion powershell | Out-String | Invoke-Expression
-
-To load completions for every new session, add the output of the above command
-to your powershell profile.
-
-```
-generator completion powershell [flags]
-```
-
-### Options
-
-```
- -h, --help help for powershell
- --no-descriptions disable completion descriptions
-```
-
-### Options inherited from parent commands
-
-```
- --config string config file (default is PROJECT/.env)
- --modifyExistCode allow generator modify exist code (default true)
-```
-
-### SEE ALSO
-
-* [generator completion](generator_completion.md) - Generate the
autocompletion script for the specified shell
-
-###### Auto generated by spf13/cobra on 24-Jun-2022
diff --git a/backend/generator/docs/generator_completion_zsh.md
b/backend/generator/docs/generator_completion_zsh.md
deleted file mode 100644
index bdcca7ff0..000000000
--- a/backend/generator/docs/generator_completion_zsh.md
+++ /dev/null
@@ -1,52 +0,0 @@
-## generator completion zsh
-
-Generate the autocompletion script for zsh
-
-### Synopsis
-
-Generate the autocompletion script for the zsh shell.
-
-If shell completion is not already enabled in your environment you will need
-to enable it. You can execute the following once:
-
- echo "autoload -U compinit; compinit" >> ~/.zshrc
-
-To load completions in your current shell session:
-
- source <(generator completion zsh); compdef _generator generator
-
-To load completions for every new session, execute once:
-
-#### Linux:
-
- generator completion zsh > "${fpath[1]}/_generator"
-
-#### macOS:
-
- generator completion zsh > $(brew
--prefix)/share/zsh/site-functions/_generator
-
-You will need to start a new shell for this setup to take effect.
-
-```
-generator completion zsh [flags]
-```
-
-### Options
-
-```
- -h, --help help for zsh
- --no-descriptions disable completion descriptions
-```
-
-### Options inherited from parent commands
-
-```
- --config string config file (default is PROJECT/.env)
- --modifyExistCode allow generator modify exist code (default true)
-```
-
-### SEE ALSO
-
-* [generator completion](generator_completion.md) - Generate the
autocompletion script for the specified shell
-
-###### Auto generated by spf13/cobra on 24-Jun-2022
diff --git a/backend/generator/docs/generator_create-collector.md
b/backend/generator/docs/generator_create-collector.md
deleted file mode 100644
index 5db3778b4..000000000
--- a/backend/generator/docs/generator_create-collector.md
+++ /dev/null
@@ -1,32 +0,0 @@
-## generator create-collector
-
-Create a new collector
-
-### Synopsis
-
-Create a new collector
-Type in what the name of collector is, then generator will create a new
collector in
-plugins/$plugin_name/tasks/$collector_name for you
-
-```
-generator create-collector [plugin_name] [collector_name] [flags]
-```
-
-### Options
-
-```
- -h, --help help for create-collector
-```
-
-### Options inherited from parent commands
-
-```
- --config string config file (default is PROJECT/.env)
- --modifyExistCode allow generator modify exist code (default true)
-```
-
-### SEE ALSO
-
-* [generator](generator.md) - Apache DevLake Cli Tool -- Code Generator
-
-###### Auto generated by spf13/cobra on 24-Jun-2022
diff --git a/backend/generator/docs/generator_create-e2e-raw.md
b/backend/generator/docs/generator_create-e2e-raw.md
deleted file mode 100644
index 58a2bf82f..000000000
--- a/backend/generator/docs/generator_create-e2e-raw.md
+++ /dev/null
@@ -1,32 +0,0 @@
-## generator create-e2e-raw
-
-Create _raw_table.csv for e2e test
-
-### Synopsis
-
-Create _raw_table.csv for e2e test
-Type in what the raw_table is, then generator will export and save in
plugins/$plugin_name/e2e/_raw_$raw_name.csv for
-you.
-
-```
-generator create-e2e-raw [plugin_name] [raw_table_name] [csv_file_name] [flags]
-```
-
-### Options
-
-```
- -h, --help help for create-e2e-raw
-```
-
-### Options inherited from parent commands
-
-```
- --config string config file (default is PROJECT/.env)
- --modifyExistCode allow generator modify exist code (default true)
-```
-
-### SEE ALSO
-
-* [generator](generator.md) - Apache DevLake Cli Tool -- Code Generator
-
-###### Auto generated by spf13/cobra on 27-Jun-2022
diff --git a/backend/generator/docs/generator_create-extractor.md
b/backend/generator/docs/generator_create-extractor.md
deleted file mode 100644
index 8437abbba..000000000
--- a/backend/generator/docs/generator_create-extractor.md
+++ /dev/null
@@ -1,32 +0,0 @@
-## generator create-extractor
-
-Create a new extractor
-
-### Synopsis
-
-Create a new extractor
-Type in what the name of extractor is, then generator will create a new
extractor in
-plugins/$plugin_name/tasks/$extractor_name for you
-
-```
-generator create-extractor [plugin_name] [extractor_name] [flags]
-```
-
-### Options
-
-```
- -h, --help help for create-extractor
-```
-
-### Options inherited from parent commands
-
-```
- --config string config file (default is PROJECT/.env)
- --modifyExistCode allow generator modify exist code (default true)
-```
-
-### SEE ALSO
-
-* [generator](generator.md) - Apache DevLake Cli Tool -- Code Generator
-
-###### Auto generated by spf13/cobra on 24-Jun-2022
diff --git a/backend/generator/docs/generator_create-migration.md
b/backend/generator/docs/generator_create-migration.md
deleted file mode 100644
index c11e293d6..000000000
--- a/backend/generator/docs/generator_create-migration.md
+++ /dev/null
@@ -1,33 +0,0 @@
-## generator create-migration
-
-Create a new migration
-
-### Synopsis
-
-Create a new migration
-Type in what the purpose of migration is, then generator will create a new
migration in
-plugins/$plugin_name/models/migrationscripts/updateSchemasXXXXXXXX.go for you.
-If framework passed, generator will create a new migration in
models/migrationscripts/updateSchemasXXXXXXXX.go
-
-```
-generator create-migration [plugin_name/framework] [flags]
-```
-
-### Options
-
-```
- -h, --help help for create-migration
-```
-
-### Options inherited from parent commands
-
-```
- --config string config file (default is PROJECT/.env)
- --modifyExistCode allow generator modify exist code (default true)
-```
-
-### SEE ALSO
-
-* [generator](generator.md) - Apache DevLake Cli Tool -- Code Generator
-
-###### Auto generated by spf13/cobra on 24-Jun-2022
diff --git a/backend/generator/docs/generator_create-plugin.md
b/backend/generator/docs/generator_create-plugin.md
deleted file mode 100644
index 4ce385f48..000000000
--- a/backend/generator/docs/generator_create-plugin.md
+++ /dev/null
@@ -1,31 +0,0 @@
-## generator create-plugin
-
-Create a new plugin
-
-### Synopsis
-
-Create a new plugin
-Type in what the name of plugin is, then generator will create a new plugin in
plugins/$plugin_name for you
-
-```
-generator create-plugin [plugin_name] [flags]
-```
-
-### Options
-
-```
- -h, --help help for create-plugin
-```
-
-### Options inherited from parent commands
-
-```
- --config string config file (default is PROJECT/.env)
- --modifyExistCode allow generator modify exist code (default true)
-```
-
-### SEE ALSO
-
-* [generator](generator.md) - Apache DevLake Cli Tool -- Code Generator
-
-###### Auto generated by spf13/cobra on 24-Jun-2022
diff --git a/backend/generator/docs/generator_generator-doc.md
b/backend/generator/docs/generator_generator-doc.md
deleted file mode 100644
index a585851f4..000000000
--- a/backend/generator/docs/generator_generator-doc.md
+++ /dev/null
@@ -1,26 +0,0 @@
-## generator generator-doc
-
-generate document for generator
-
-```
-generator generator-doc [flags]
-```
-
-### Options
-
-```
- -h, --help help for generator-doc
-```
-
-### Options inherited from parent commands
-
-```
- --config string config file (default is PROJECT/.env)
- --modifyExistCode allow generator modify exist code (default true)
-```
-
-### SEE ALSO
-
-* [generator](generator.md) - Apache DevLake Cli Tool -- Code Generator
-
-###### Auto generated by spf13/cobra on 24-Jun-2022
diff --git a/backend/generator/docs/generator_init-migration.md
b/backend/generator/docs/generator_init-migration.md
deleted file mode 100644
index 853666ebf..000000000
--- a/backend/generator/docs/generator_init-migration.md
+++ /dev/null
@@ -1,32 +0,0 @@
-## generator init-migration
-
-Init migration for plugin
-
-### Synopsis
-
-Init migration for plugin
-Type in which plugin do you want init migrations in, then generator will
create a init migration in
-plugins/$plugin_name/models/migrationscripts/ for you.
-
-```
-generator init-migration [plugin_name] [flags]
-```
-
-### Options
-
-```
- -h, --help help for init-migration
-```
-
-### Options inherited from parent commands
-
-```
- --config string config file (default is PROJECT/.env)
- --modifyExistCode allow generator modify exist code (default true)
-```
-
-### SEE ALSO
-
-* [generator](generator.md) - Apache DevLake Cli Tool -- Code Generator
-
-###### Auto generated by spf13/cobra on 27-Jun-2022
diff --git a/backend/generator/main.go b/backend/generator/main.go
deleted file mode 100644
index 0515a4bee..000000000
--- a/backend/generator/main.go
+++ /dev/null
@@ -1,29 +0,0 @@
-/*
-Licensed to the Apache Software Foundation (ASF) under one or more
-contributor license agreements. See the NOTICE file distributed with
-this work for additional information regarding copyright ownership.
-The ASF licenses this file to You under the Apache License, Version 2.0
-(the "License"); you may not use this file except in compliance with
-the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-package main
-
-import (
- "github.com/apache/incubator-devlake/generator/cmd"
- "os"
-)
-
-func main() {
- if err := cmd.Execute(); err != nil {
- os.Exit(1)
- }
-}
diff --git
a/backend/generator/template/migrationscripts/add_init_tables.go-template
b/backend/generator/template/migrationscripts/add_init_tables.go-template
deleted file mode 100644
index a88f05346..000000000
--- a/backend/generator/template/migrationscripts/add_init_tables.go-template
+++ /dev/null
@@ -1,41 +0,0 @@
-/*
-Licensed to the Apache Software Foundation (ASF) under one or more
-contributor license agreements. See the NOTICE file distributed with
-this work for additional information regarding copyright ownership.
-The ASF licenses this file to You under the Apache License, Version 2.0
-(the "License"); you may not use this file except in compliance with
-the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-package migrationscripts
-
-import (
- "github.com/apache/incubator-devlake/errors"
- "github.com/apache/incubator-devlake/plugins/core"
- "github.com/apache/incubator-devlake/helpers/migrationhelper"
-)
-
-type addInitTables struct {}
-
-func (*addInitTables) Up(basicRes context.BasicRes) errors.Error {
- return migrationhelper.AutoMigrateTables(
- basicRes,
- // TO Add models
- )
-}
-
-func (*addInitTables) Version() uint64 {
- return {{ .Date }}000001
-}
-
-func (*addInitTables) Name() string {
- return "{{ .pluginName }} init schemas"
-}
diff --git
a/backend/generator/template/migrationscripts/archived/connection.go-template
b/backend/generator/template/migrationscripts/archived/connection.go-template
deleted file mode 100644
index bb5fb7d49..000000000
---
a/backend/generator/template/migrationscripts/archived/connection.go-template
+++ /dev/null
@@ -1,16 +0,0 @@
-/*
-Licensed to the Apache Software Foundation (ASF) under one or more
-contributor license agreements. See the NOTICE file distributed with
-this work for additional information regarding copyright ownership.
-The ASF licenses this file to You under the Apache License, Version 2.0
-(the "License"); you may not use this file except in compliance with
-the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
\ No newline at end of file
diff --git a/backend/generator/template/migrationscripts/migration.go-template
b/backend/generator/template/migrationscripts/migration.go-template
deleted file mode 100644
index 1502090ae..000000000
--- a/backend/generator/template/migrationscripts/migration.go-template
+++ /dev/null
@@ -1,41 +0,0 @@
-/*
-Licensed to the Apache Software Foundation (ASF) under one or more
-contributor license agreements. See the NOTICE file distributed with
-this work for additional information regarding copyright ownership.
-The ASF licenses this file to You under the Apache License, Version 2.0
-(the "License"); you may not use this file except in compliance with
-the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-package migrationscripts
-
-import (
- "github.com/apache/incubator-devlake/errors"
- "github.com/apache/incubator-devlake/plugins/core"
- "github.com/apache/incubator-devlake/helpers/migrationhelper"
-)
-
-type {{ .Purpose }} struct{}
-
-func (*{{ .Purpose }}) Up(basicRes context.BasicRes) errors.Error {
- return migrationhelper.AutoMigrateTables(
- basicRes,
- // TO Add models
- )
-}
-
-func (*{{ .Purpose }}) Version() uint64 {
- return {{ .Date }}{{ .Count }}
-}
-
-func (*{{ .Purpose }}) Name() string {
- return "UpdateSchemas for {{ .Purpose }}"
-}
diff --git
a/backend/generator/template/migrationscripts/migration_with_config.go-template
b/backend/generator/template/migrationscripts/migration_with_config.go-template
deleted file mode 100644
index c60725115..000000000
---
a/backend/generator/template/migrationscripts/migration_with_config.go-template
+++ /dev/null
@@ -1,47 +0,0 @@
-/*
-Licensed to the Apache Software Foundation (ASF) under one or more
-contributor license agreements. See the NOTICE file distributed with
-this work for additional information regarding copyright ownership.
-The ASF licenses this file to You under the Apache License, Version 2.0
-(the "License"); you may not use this file except in compliance with
-the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-package migrationscripts
-
-import (
- "github.com/apache/incubator-devlake/errors"
- "github.com/apache/incubator-devlake/plugins/core"
- "github.com/apache/incubator-devlake/helpers/migrationhelper"
-)
-
-type {{ .Purpose }} struct{
- config core.ConfigGetter
-}
-
-func (u *{{ .Purpose }}) SetConfigGetter(config core.ConfigGetter) {
- u.config = config
-}
-
-func (*{{ .Purpose }}) Up(basicRes context.BasicRes) errors.Error {
- return migrationhelper.AutoMigrateTables(
- basicRes,
- // TO Add models
- )
-}
-
-func (*{{ .Purpose }}) Version() uint64 {
- return {{ .Date }}{{ .Count }}
-}
-
-func (*{{ .Purpose }}) Name() string {
- return "UpdateSchemas for {{ .Purpose }}"
-}
diff --git a/backend/generator/template/migrationscripts/register.go-template
b/backend/generator/template/migrationscripts/register.go-template
deleted file mode 100644
index e6cee6c44..000000000
--- a/backend/generator/template/migrationscripts/register.go-template
+++ /dev/null
@@ -1,27 +0,0 @@
-/*
-Licensed to the Apache Software Foundation (ASF) under one or more
-contributor license agreements. See the NOTICE file distributed with
-this work for additional information regarding copyright ownership.
-The ASF licenses this file to You under the Apache License, Version 2.0
-(the "License"); you may not use this file except in compliance with
-the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-package migrationscripts
-
-import "github.com/apache/incubator-devlake/plugins/core"
-
-// All return all the migration scripts
-func All() []plugin.MigrationScript {
- return []plugin.MigrationScript{
- new(addInitTables),
- }
-}
diff --git a/backend/generator/template/plugin/api/blueprint.go-template
b/backend/generator/template/plugin/api/blueprint.go-template
deleted file mode 100644
index 11f9ca702..000000000
--- a/backend/generator/template/plugin/api/blueprint.go-template
+++ /dev/null
@@ -1,69 +0,0 @@
-/*
-Licensed to the Apache Software Foundation (ASF) under one or more
-contributor license agreements. See the NOTICE file distributed with
-this work for additional information regarding copyright ownership.
-The ASF licenses this file to You under the Apache License, Version 2.0
-(the "License"); you may not use this file except in compliance with
-the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-package api
-
-import (
- "encoding/json"
- "github.com/apache/incubator-devlake/core/errors"
- "github.com/apache/incubator-devlake/core/plugin"
- helper "github.com/apache/incubator-devlake/helpers/pluginhelper/api"
- "github.com/apache/incubator-devlake/plugins/{{ .plugin_name }}/tasks"
-)
-
-func MakePipelinePlan(subtaskMetas []plugin.SubTaskMeta, connectionId uint64,
scope []*plugin.BlueprintScopeV100) (plugin.PipelinePlan, errors.Error) {
- var err error
- plan := make(plugin.PipelinePlan, len(scope))
- for i, scopeElem := range scope {
- taskOptions := make(map[string]interface{})
- err = json.Unmarshal(scopeElem.Options, &taskOptions)
- if err != nil {
- return nil, errors.Default.Wrap(err, "error
unmarshalling task options")
- }
- taskOptions["connectionId"] = connectionId
-
- //TODO Add transformation rules to task options
-
- /*
- var transformationRules tasks.TransformationRules
- if len(scopeElem.Transformation) > 0 {
- err = json.Unmarshal(scopeElem.Transformation,
&transformationRules)
- if err != nil {
- return nil, err
- }
- }
- */
- //taskOptions["transformationRules"] = transformationRules
- _, err := tasks.DecodeAndValidateTaskOptions(taskOptions)
- if err != nil {
- return nil, err
- }
- // subtasks
- subtasks, err := helper.MakePipelinePlanSubtasks(subtaskMetas,
scopeElem.Entities)
- if err != nil {
- return nil, err
- }
- plan[i] = core.PipelineStage{
- {
- Plugin: "{{ .plugin_name }}",
- Subtasks: subtasks,
- Options: taskOptions,
- },
- }
- }
- return plan, nil
-}
diff --git a/backend/generator/template/plugin/api/connection.go-template
b/backend/generator/template/plugin/api/connection.go-template
deleted file mode 100644
index 05afe7662..000000000
--- a/backend/generator/template/plugin/api/connection.go-template
+++ /dev/null
@@ -1,167 +0,0 @@
-/*
-Licensed to the Apache Software Foundation (ASF) under one or more
-contributor license agreements. See the NOTICE file distributed with
-this work for additional information regarding copyright ownership.
-The ASF licenses this file to You under the Apache License, Version 2.0
-(the "License"); you may not use this file except in compliance with
-the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-package api
-
-import (
- "context"
- "fmt"
- "github.com/apache/incubator-devlake/core/errors"
- "net/http"
- "time"
-
- "github.com/apache/incubator-devlake/core/plugin"
- helper "github.com/apache/incubator-devlake/helpers/pluginhelper/api"
- "github.com/apache/incubator-devlake/plugins/{{ .plugin_name }}/models"
-)
-
-//TODO Please modify the following code to fit your needs
-// @Summary test {{ .pluginName }} connection
-// @Description Test {{ .pluginName }} Connection. endpoint: "https://dev.{{
.pluginName }}.com/{organization}/
-// @Tags plugins/{{ .pluginName }}
-// @Param body body models.{{ .PluginName }}Conn true "json body"
-// @Success 200 {object} {{ .PluginName }}TestConnResponse "Success"
-// @Failure 400 {string} errcode.Error "Bad Request"
-// @Failure 500 {string} errcode.Error "Internal Error"
-// @Router /plugins/{{ .pluginName }}/test [POST]
-func TestConnection(input *core.ApiResourceInput) (*plugin.ApiResourceOutput,
errors.Error) {
- // decode
- var err errors.Error
- var connection models.TestConnectionRequest
- if err = helper.Decode(input.Body, &connection, vld); err != nil {
- return nil, err
- }
- // test connection
- apiClient, err := api.NewApiClient(
- context.TODO(),
- connection.Endpoint,
- map[string]string{
- "Authorization": fmt.Sprintf("Bearer %v",
connection.Token),
- },
- 3*time.Second,
- connection.Proxy,
- basicRes,
- )
- if err != nil {
- return nil, err
- }
-
- res, err := apiClient.Get("user", nil, nil)
- if err != nil {
- return nil, err
- }
- resBody := &models.ApiUserResponse{}
- err = helper.UnmarshalResponse(res, resBody)
- if err != nil {
- return nil, err
- }
-
- if res.StatusCode != http.StatusOK {
- return nil,
errors.HttpStatus(res.StatusCode).New(fmt.Sprintf("unexpected status code: %d",
res.StatusCode))
- }
- body := {{ .PluginName }}TestConnResponse{}
- body.Success = true
- body.Message = "success"
- body.Connection = &connection
- // output
- return &plugin.ApiResourceOutput{Body: body, Status: 200}, nil
-}
-
-//TODO Please modify the folowing code to adapt to your plugin
-// @Summary create {{ .pluginName }} connection
-// @Description Create {{ .pluginName }} connection
-// @Tags plugins/{{ .pluginName }}
-// @Param body body models.{{ .PluginName }}Connection true "json body"
-// @Success 200 {object} models.{{ .PluginName }}Connection
-// @Failure 400 {string} errcode.Error "Bad Request"
-// @Failure 500 {string} errcode.Error "Internal Error"
-// @Router /plugins/{{ .pluginName }}/connections [POST]
-func PostConnections(input *core.ApiResourceInput) (*plugin.ApiResourceOutput,
errors.Error) {
- // update from request and save to database
- connection := &models.{{ .PluginName }}Connection{}
- err := connectionHelper.Create(connection, input)
- if err != nil {
- return nil, err
- }
- return &plugin.ApiResourceOutput{Body: connection, Status:
http.StatusOK}, nil
-}
-
-//TODO Please modify the folowing code to adapt to your plugin
-/*
-// @Summary patch {{ .pluginName }} connection
-// @Description Patch {{ .pluginName }} connection
-// @Tags plugins/{{ .pluginName }}
-// @Param body body models.{{ .PluginName }}Connection true "json body"
-// @Success 200 {object} models.{{ .PluginName }}Connection
-// @Failure 400 {string} errcode.Error "Bad Request"
-// @Failure 500 {string} errcode.Error "Internal Error"
-// @Router /plugins/{{ .pluginName }}/connections/{connectionId} [PATCH]
-func PatchConnection(input *core.ApiResourceInput) (*plugin.ApiResourceOutput,
errors.Error) {
- connection := &models.{{ .PluginName }}Connection{}
- err := connectionHelper.Patch(connection, input)
- if err != nil {
- return nil, err
- }
- return &plugin.ApiResourceOutput{Body: connection}, nil
-}
-
-// @Summary delete a {{ .pluginName }} connection
-// @Description Delete a {{ .pluginName }} connection
-// @Tags plugins/{{ .pluginName }}
-// @Success 200 {object} models.{{ .PluginName }}Connection
-// @Failure 400 {string} errcode.Error "Bad Request"
-// @Failure 500 {string} errcode.Error "Internal Error"
-// @Router /plugins/{{ .pluginName }}/connections/{connectionId} [DELETE]
-func DeleteConnection(input *core.ApiResourceInput)
(*plugin.ApiResourceOutput, errors.Error) {
- connection := &models.{{ .PluginName }}Connection{}
- err := connectionHelper.First(connection, input.Params)
- if err != nil {
- return nil, err
- }
- err = connectionHelper.Delete(connection)
- return &plugin.ApiResourceOutput{Body: connection}, err
-}
-
-// @Summary get all {{ .pluginName }} connections
-// @Description Get all {{ .pluginName }} connections
-// @Tags plugins/{{ .pluginName }}
-// @Success 200 {object} []models.{{ .PluginName }}Connection
-// @Failure 400 {string} errcode.Error "Bad Request"
-// @Failure 500 {string} errcode.Error "Internal Error"
-// @Router /plugins/{{ .pluginName }}/connections [GET]
-func ListConnections(input *core.ApiResourceInput) (*plugin.ApiResourceOutput,
errors.Error) {
- var connections []models.{{ .PluginName }}Connection
- err := connectionHelper.List(&connections)
- if err != nil {
- return nil, err
- }
- return &plugin.ApiResourceOutput{Body: connections, Status:
http.StatusOK}, nil
-}
-
-//TODO Please modify the folowing code to adapt to your plugin
-// @Summary get {{ .pluginName }} connection detail
-// @Description Get {{ .pluginName }} connection detail
-// @Tags plugins/{{ .pluginName }}
-// @Success 200 {object} models.{{ .PluginName }}Connection
-// @Failure 400 {string} errcode.Error "Bad Request"
-// @Failure 500 {string} errcode.Error "Internal Error"
-// @Router /plugins/{{ .pluginName }}/connections/{connectionId} [GET]
-func GetConnection(input *core.ApiResourceInput) (*plugin.ApiResourceOutput,
errors.Error) {
- connection := &models.{{ .PluginName }}Connection{}
- err := connectionHelper.First(connection, input.Params)
- return &plugin.ApiResourceOutput{Body: connection}, err
-}
\ No newline at end of file
diff --git a/backend/generator/template/plugin/api/init.go-template
b/backend/generator/template/plugin/api/init.go-template
deleted file mode 100644
index a981cea5e..000000000
--- a/backend/generator/template/plugin/api/init.go-template
+++ /dev/null
@@ -1,37 +0,0 @@
-/*
-Licensed to the Apache Software Foundation (ASF) under one or more
-contributor license agreements. See the NOTICE file distributed with
-this work for additional information regarding copyright ownership.
-The ASF licenses this file to You under the Apache License, Version 2.0
-(the "License"); you may not use this file except in compliance with
-the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-package api
-
-import (
- "github.com/apache/incubator-devlake/core/plugin"
- helper "github.com/apache/incubator-devlake/helpers/pluginhelper/api"
- "github.com/go-playground/validator/v10"
-)
-
-var vld *validator.Validate
-var connectionHelper *helper.ConnectionApiHelper
-var basicRes context.BasicRes
-
-func Init(br context.BasicRes) {
- basicRes = br
- vld = validator.New()
- connectionHelper = helper.NewConnectionHelper(
- basicRes,
- vld,
- )
-}
diff --git
a/backend/generator/template/plugin/impl/impl_complete_plugin.go-template
b/backend/generator/template/plugin/impl/impl_complete_plugin.go-template
deleted file mode 100644
index defafca36..000000000
--- a/backend/generator/template/plugin/impl/impl_complete_plugin.go-template
+++ /dev/null
@@ -1,134 +0,0 @@
-/*
-Licensed to the Apache Software Foundation (ASF) under one or more
-contributor license agreements. See the NOTICE file distributed with
-this work for additional information regarding copyright ownership.
-The ASF licenses this file to You under the Apache License, Version 2.0
-(the "License"); you may not use this file except in compliance with
-the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-package impl
-
-import (
- "fmt"
- "github.com/apache/incubator-devlake/errors"
- "github.com/apache/incubator-devlake/plugins/core"
- "github.com/apache/incubator-devlake/plugins/core"
- "github.com/apache/incubator-devlake/plugins/{{ .plugin_name }}/api"
- "github.com/apache/incubator-devlake/plugins/{{ .plugin_name }}/models"
- "github.com/apache/incubator-devlake/plugins/{{ .plugin_name
}}/models/migrationscripts"
- "github.com/apache/incubator-devlake/plugins/{{ .plugin_name }}/tasks"
- helper "github.com/apache/incubator-devlake/helpers/pluginhelper/api"
- "github.com/spf13/viper"
-)
-
-// make sure interface is implemented
-var _ plugin.PluginMeta = (*{{ .PluginName }})(nil)
-var _ plugin.PluginInit = (*{{ .PluginName }})(nil)
-var _ plugin.PluginTask = (*{{ .PluginName }})(nil)
-var _ plugin.PluginApi = (*{{ .PluginName }})(nil)
-var _ plugin.PluginBlueprintV100 = (*{{ .PluginName }})(nil)
-var _ plugin.CloseablePluginTask = (*{{ .PluginName }})(nil)
-
-
-
-type {{ .PluginName }} struct{}
-
-func (p {{ .PluginName }}) Description() string {
- return "collect some {{ .PluginName }} data"
-}
-
-func (p {{ .PluginName }}) Init(br context.BasicRes) errors.Error {
- api.Init(br)
- return nil
-}
-
-func (p {{ .PluginName }}) SubTaskMetas() []plugin.SubTaskMeta {
- // TODO add your sub task here
- return []plugin.SubTaskMeta{
- }
-}
-
-func (p {{ .PluginName }}) PrepareTaskData(taskCtx plugin.TaskContext, options
map[string]interface{}) (interface{}, errors.Error) {
- op, err := tasks.DecodeAndValidateTaskOptions(options)
- if err != nil {
- return nil, err
- }
- connectionHelper := helper.NewConnectionHelper(
- taskCtx,
- nil,
- )
- connection := &models.{{ .PluginName }}Connection{}
- err = connectionHelper.FirstById(connection, op.ConnectionId)
- if err != nil {
- return nil, errors.Default.Wrap(err, "unable to get {{ .PluginName }}
connection by the given connection ID")
- }
-
- apiClient, err := tasks.New{{ .PluginName }}ApiClient(taskCtx, connection)
- if err != nil {
- return nil, errors.Default.Wrap(err, "unable to get {{ .PluginName }}
API client instance")
- }
- taskData := &tasks.{{ .PluginName }}TaskData{
- Options: op,
- ApiClient: apiClient,
- }
- var createdDateAfter time.Time
- if op.CreatedDateAfter != "" {
- createdDateAfter, err =
errors.Convert01(time.Parse(time.RFC3339, op.CreatedDateAfter))
- if err != nil {
- return nil, errors.BadInput.Wrap(err, "invalid value
for `createdDateAfter`")
- }
- }
- if !createdDateAfter.IsZero() {
- taskData.CreatedDateAfter = &createdDateAfter
- logger.Debug("collect data updated createdDateAfter %s",
createdDateAfter)
- }
- return taskData, nil
-}
-
-// PkgPath information lost when compiled as plugin(.so)
-func (p {{ .PluginName }}) RootPkgPath() string {
- return "github.com/apache/incubator-devlake/plugins/{{ .plugin_name }}"
-}
-
-func (p {{ .PluginName }}) MigrationScripts() []plugin.MigrationScript {
- return migrationscripts.All()
-}
-
-func (p {{ .PluginName }}) ApiResources()
map[string]map[string]plugin.ApiResourceHandler {
- return map[string]map[string]plugin.ApiResourceHandler{
- "test": {
- "POST": api.TestConnection,
- },
- "connections": {
- "POST": api.PostConnections,
- "GET": api.ListConnections,
- },
- "connections/:connectionId": {
- "GET": api.GetConnection,
- "PATCH": api.PatchConnection,
- "DELETE": api.DeleteConnection,
- },
- }
-}
-
-func (p {{ .PluginName }}) MakePipelinePlan(connectionId uint64, scope
[]*plugin.BlueprintScopeV100) (plugin.PipelinePlan, errors.Error) {
- return api.MakePipelinePlan(plugin.SubTaskMetas(), connectionId, scope)
-}
-
-func (p {{ .PluginName }}) Close(taskCtx plugin.TaskContext) errors.Error {
- data, ok := taskCtx.GetData().(*tasks.{{ .PluginName }}TaskData)
- if !ok {
- return errors.Default.New(fmt.Sprintf("GetData failed when try
to close %+v", taskCtx))
- }
- data.ApiClient.Release()
- return nil
-}
diff --git a/backend/generator/template/plugin/models/connection.go-template
b/backend/generator/template/plugin/models/connection.go-template
deleted file mode 100644
index a7d7a20b7..000000000
--- a/backend/generator/template/plugin/models/connection.go-template
+++ /dev/null
@@ -1,51 +0,0 @@
-/*
-Licensed to the Apache Software Foundation (ASF) under one or more
-contributor license agreements. See the NOTICE file distributed with
-this work for additional information regarding copyright ownership.
-The ASF licenses this file to You under the Apache License, Version 2.0
-(the "License"); you may not use this file except in compliance with
-the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-package models
-
-import helper "github.com/apache/incubator-devlake/helpers/pluginhelper/api"
-
-//TODO Please modify the following code to fit your needs
-// This object conforms to what the frontend currently sends.
-type {{ .PluginName }}Connection struct {
- helper.RestConnection `mapstructure:",squash"`
- //TODO you may need to use helper.BasicAuth instead of helper.AccessToken
- helper.AccessToken `mapstructure:",squash"`
-}
-
-type TestConnectionRequest struct {
- Endpoint string `json:"endpoint"`
- Proxy string `json:"proxy"`
- helper.AccessToken `mapstructure:",squash"`
-}
-
-// This object conforms to what the frontend currently expects.
-type {{ .PluginName }}Response struct {
- Name string `json:"name"`
- ID int `json:"id"`
- {{ .PluginName }}Connection
-}
-
-// Using User because it requires authentication.
-type ApiUserResponse struct {
- Id int
- Name string `json:"name"`
-}
-
-func ({{ .PluginName }}Connection) TableName() string {
- return "_tool_{{ .plugin_name }}_connections"
-}
diff --git a/backend/generator/template/plugin/plugin_main.go-template
b/backend/generator/template/plugin/plugin_main.go-template
deleted file mode 100644
index f54349eaa..000000000
--- a/backend/generator/template/plugin/plugin_main.go-template
+++ /dev/null
@@ -1,90 +0,0 @@
-/*
-Licensed to the Apache Software Foundation (ASF) under one or more
-contributor license agreements. See the NOTICE file distributed with
-this work for additional information regarding copyright ownership.
-The ASF licenses this file to You under the Apache License, Version 2.0
-(the "License"); you may not use this file except in compliance with
-the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-package main
-
-import (
- "github.com/apache/incubator-devlake/core/errors"
- "github.com/apache/incubator-devlake/core/plugin"
- "github.com/apache/incubator-devlake/core/errors"
- "github.com/apache/incubator-devlake/plugins/{{ .plugin_name }}/tasks"
- "github.com/apache/incubator-devlake/runner"
- "github.com/mitchellh/mapstructure"
- "github.com/spf13/cobra"
- "github.com/spf13/viper"
-)
-
-// make sure interface is implemented
-var _ plugin.PluginMeta = (*{{ .PluginName }})(nil)
-var _ plugin.PluginInit = (*{{ .PluginName }})(nil)
-var _ plugin.PluginTask = (*{{ .PluginName }})(nil)
-var _ plugin.PluginApi = (*{{ .PluginName }})(nil)
-
-// PluginEntry exports a symbol for Framework to load
-var PluginEntry {{ .PluginName }} //nolint
-
-type {{ .PluginName }} struct{}
-
-func (p {{ .PluginName }}) Description() string {
- return "collect some {{ .PluginName }} data"
-}
-
-func (p {{ .PluginName }}) Init(br context.BasicRes) errors.Error {
- api.Init(br)
- return nil
-}
-
-func (p {{ .PluginName }}) SubTaskMetas() []plugin.SubTaskMeta {
- // TODO add your sub task here
- return []plugin.SubTaskMeta{
- }
-}
-
-func (p {{ .PluginName }}) PrepareTaskData(taskCtx plugin.TaskContext, options
map[string]interface{}) (interface{}, errors.Error) {
- var op tasks.{{ .PluginName }}Options
- if err := helper.Decode(input.Body, &connection, vld); err != nil {
- return nil, err
- }
- return &tasks.{{ .PluginName }}TaskData{
- Options: &op,
- }, nil
-}
-
-// PkgPath information lost when compiled as plugin(.so)
-func (p {{ .PluginName }}) RootPkgPath() string {
- return "github.com/apache/incubator-devlake/plugins/{{ .pluginName }}"
-}
-
-func (p {{ .PluginName }}) ApiResources()
map[string]map[string]plugin.ApiResourceHandler {
- return nil
-}
-
-// standalone mode for debugging
-func main() {
- cmd := &cobra.Command{Use: "{{ .pluginName }}"}
-
- // TODO add your cmd flag if necessary
- // yourFlag := cmd.Flags().IntP("yourFlag", "y", 8, "TODO add
description here")
- // _ = cmd.MarkFlagRequired("yourFlag")
-
- cmd.Run = func(cmd *cobra.Command, args []string) {
- runner.DirectRun(cmd, args, PluginEntry, map[string]interface{}{
- // TODO add more custom params here
- })
- }
- runner.RunCmd(cmd)
-}
diff --git
a/backend/generator/template/plugin/plugin_main_complete_plugin.go-template
b/backend/generator/template/plugin/plugin_main_complete_plugin.go-template
deleted file mode 100644
index 3d3259642..000000000
--- a/backend/generator/template/plugin/plugin_main_complete_plugin.go-template
+++ /dev/null
@@ -1,43 +0,0 @@
-/*
-Licensed to the Apache Software Foundation (ASF) under one or more
-contributor license agreements. See the NOTICE file distributed with
-this work for additional information regarding copyright ownership.
-The ASF licenses this file to You under the Apache License, Version 2.0
-(the "License"); you may not use this file except in compliance with
-the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-package main
-
-import (
- "github.com/apache/incubator-devlake/plugins/{{ .pluginName }}/impl"
- "github.com/apache/incubator-devlake/runner"
- "github.com/spf13/cobra"
-)
-
-// Export a variable named PluginEntry for Framework to search and load
-var PluginEntry impl.{{ .PluginName }} //nolint
-
-// standalone mode for debugging
-func main() {
- cmd := &cobra.Command{Use: "{{ .pluginName }}"}
-
- // TODO add your cmd flag if necessary
- // yourFlag := cmd.Flags().IntP("yourFlag", "y", 8, "TODO add
description here")
- // _ = cmd.MarkFlagRequired("yourFlag")
-
- cmd.Run = func(cmd *cobra.Command, args []string) {
- runner.DirectRun(cmd, args, PluginEntry, map[string]interface{}{
- // TODO add more custom params here
- })
- }
- runner.RunCmd(cmd)
-}
diff --git a/backend/generator/template/plugin/tasks/api_client.go-template
b/backend/generator/template/plugin/tasks/api_client.go-template
deleted file mode 100644
index f3344590f..000000000
--- a/backend/generator/template/plugin/tasks/api_client.go-template
+++ /dev/null
@@ -1,74 +0,0 @@
-/*
-Licensed to the Apache Software Foundation (ASF) under one or more
-contributor license agreements. See the NOTICE file distributed with
-this work for additional information regarding copyright ownership.
-The ASF licenses this file to You under the Apache License, Version 2.0
-(the "License"); you may not use this file except in compliance with
-the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-package tasks
-
-import (
- "fmt"
- "net/http"
- "strconv"
- "time"
-
- "github.com/apache/incubator-devlake/plugins/{{ .plugin_name }}/models"
- "github.com/apache/incubator-devlake/core/plugin"
- "github.com/apache/incubator-devlake/core/errors"
- helper "github.com/apache/incubator-devlake/helpers/pluginhelper/api"
-)
-
-func New{{ .PluginName }}ApiClient(taskCtx plugin.TaskContext, connection
*models.{{ .PluginName }}Connection) (*api.ApiAsyncClient, errors.Error) {
- // create synchronize api client so we can calculate api rate limit
dynamically
- headers := map[string]string{
- "Authorization": fmt.Sprintf("Bearer %v", connection.Token),
- }
- apiClient, err := api.NewApiClient(taskCtx.GetContext(),
connection.Endpoint, headers, 0, connection.Proxy, taskCtx)
- if err != nil {
- return nil, err
- }
- apiClient.SetAfterFunction(func(res *http.Response) errors.Error {
- if res.StatusCode == http.StatusUnauthorized {
- return
errors.HttpStatus(res.StatusCode).New("authentication failed, please check your
AccessToken")
- }
- return nil
- })
-
- // create rate limit calculator
- rateLimiter := &api.ApiRateLimitCalculator{
- UserRateLimitPerHour: connection.RateLimitPerHour,
- DynamicRateLimit: func(res *http.Response) (int, time.Duration,
errors.Error) {
- rateLimitHeader := res.Header.Get("RateLimit-Limit")
- if rateLimitHeader == "" {
- // use default
- return 0, 0, nil
- }
- rateLimit, err := strconv.Atoi(rateLimitHeader)
- if err != nil {
- return 0, 0, errors.Default.Wrap(err, "failed
to parse RateLimit-Limit header")
- }
- // seems like {{ .plugin-ame }} rate limit is on minute
basis
- return rateLimit, 1 * time.Minute, nil
- },
- }
- asyncApiClient, err := api.CreateAsyncApiClient(
- taskCtx,
- apiClient,
- rateLimiter,
- )
- if err != nil {
- return nil, err
- }
- return asyncApiClient, nil
-}
diff --git a/backend/generator/template/plugin/tasks/api_collector.go-template
b/backend/generator/template/plugin/tasks/api_collector.go-template
deleted file mode 100644
index d9eaeec53..000000000
--- a/backend/generator/template/plugin/tasks/api_collector.go-template
+++ /dev/null
@@ -1,75 +0,0 @@
-/*
-Licensed to the Apache Software Foundation (ASF) under one or more
-contributor license agreements. See the NOTICE file distributed with
-this work for additional information regarding copyright ownership.
-The ASF licenses this file to You under the Apache License, Version 2.0
-(the "License"); you may not use this file except in compliance with
-the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-package tasks
-
-import (
- "encoding/json"
- "net/http"
- "net/url"
- "strconv"
-
- "github.com/apache/incubator-devlake/core/plugin"
- "github.com/apache/incubator-devlake/core/errors"
- helper "github.com/apache/incubator-devlake/helpers/pluginhelper/api"
-)
-
-const RAW_{{ .COLLECTOR_DATA_NAME }}_TABLE = "{{ .plugin_name }}_{{
.collector_data_name }}"
-
-var _ plugin.SubTaskEntryPoint = Collect{{ .CollectorDataName }}
-
-func Collect{{ .CollectorDataName }}(taskCtx plugin.SubTaskContext)
errors.Error {
- rawDataSubTaskArgs, data := CreateRawDataSubTaskArgs(taskCtx, RAW_{{
.COLLECTOR_DATA_NAME }}_TABLE)
- logger := taskCtx.GetLogger()
-
- collectorWithState, err :=
helper.NewApiCollectorWithState(*rawDataSubTaskArgs, data.CreatedDateAfter)
- if err != nil {
- return err
- }
- incremental := collectorWithState.IsIncremental()
-
- err = collectorWithState.InitCollector(helper.ApiCollectorArgs{
- Incremental: incremental,
- ApiClient: data.ApiClient,
- // PageSize: 100,
- // TODO write which api would you want request
- UrlTemplate: "{{ .HttpPath }}",
- Query: func(reqData *helper.RequestData) (url.Values,
errors.Error) {
- query := url.Values{}
- input := reqData.Input.(*helper.DatePair)
- query.Set("start_time",
strconv.FormatInt(input.PairStartTime.Unix(), 10))
- query.Set("end_time",
strconv.FormatInt(input.PairEndTime.Unix(), 10))
- return query, nil
- },
- ResponseParser: func(res *http.Response) ([]json.RawMessage,
errors.Error) {
- // TODO decode result from api request
- return []json.RawMessage{}, nil
- },
- })
- if err != nil {
- return err
- }
- return collectorWithState.Execute()
-}
-
-var Collect{{ .CollectorDataName }}Meta = plugin.SubTaskMeta{
- Name: "Collect{{ .CollectorDataName }}",
- EntryPoint: Collect{{ .CollectorDataName }},
- EnabledByDefault: true,
- Description: "Collect {{ .CollectorDataName }} data from {{
.PluginName }} api",
- DomainTypes: []string{},
-}
diff --git a/backend/generator/template/plugin/tasks/extractor.go-template
b/backend/generator/template/plugin/tasks/extractor.go-template
deleted file mode 100644
index 8059c5f37..000000000
--- a/backend/generator/template/plugin/tasks/extractor.go-template
+++ /dev/null
@@ -1,56 +0,0 @@
-/*
-Licensed to the Apache Software Foundation (ASF) under one or more
-contributor license agreements. See the NOTICE file distributed with
-this work for additional information regarding copyright ownership.
-The ASF licenses this file to You under the Apache License, Version 2.0
-(the "License"); you may not use this file except in compliance with
-the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-package tasks
-
-import (
- "github.com/apache/incubator-devlake/core/errors"
- "github.com/apache/incubator-devlake/core/plugin"
- helper "github.com/apache/incubator-devlake/helpers/pluginhelper/api"
-)
-
-var _ plugin.SubTaskEntryPoint = Extract{{ .ExtractorDataName }}
-
-func Extract{{ .ExtractorDataName }}(taskCtx plugin.SubTaskContext)
errors.Error {
- rawDataSubTaskArgs, data := CreateRawDataSubTaskArgs(taskCtx, RAW_{{
.COLLECTOR_DATA_NAME }}_TABLE)
-
- extractor, err := helper.NewApiExtractor(helper.ApiExtractorArgs{
- RawDataSubTaskArgs: *rawDataSubTaskArgs,
-
- Extract: func(resData *helper.RawData) ([]interface{},
errors.Error) {
- extractedModels := make([]interface{}, 0)
- println(resData.Data)
- println(resData.Input)
- // TODO decode some db models from api result
- // extractedModels = append(extractedModels,
&models.XXXXXX)
- return extractedModels, nil
- },
- })
- if err != nil {
- return err
- }
-
- return extractor.Execute()
-}
-
-var Extract{{ .ExtractorDataName }}Meta = plugin.SubTaskMeta{
- Name: "Extract{{ .ExtractorDataName }}",
- EntryPoint: Extract{{ .ExtractorDataName }},
- EnabledByDefault: true,
- Description: "Extract raw data into tool layer table {{
.plugin_name }}_{{ .extractor_data_name }}",
- DomainTypes: []string{},
-}
diff --git a/backend/generator/template/plugin/tasks/shared.go-template
b/backend/generator/template/plugin/tasks/shared.go-template
deleted file mode 100644
index ee2ac613d..000000000
--- a/backend/generator/template/plugin/tasks/shared.go-template
+++ /dev/null
@@ -1,41 +0,0 @@
-/*
-Licensed to the Apache Software Foundation (ASF) under one or more
-contributor license agreements. See the NOTICE file distributed with
-this work for additional information regarding copyright ownership.
-The ASF licenses this file to You under the Apache License, Version 2.0
-(the "License"); you may not use this file except in compliance with
-the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-package tasks
-
-import (
- "github.com/apache/incubator-devlake/core/plugin"
- "github.com/apache/incubator-devlake/helpers/pluginhelper/api"
-)
-
-func CreateRawDataSubTaskArgs(taskCtx plugin.SubTaskContext, rawTable string)
(*api.RawDataSubTaskArgs, *{{ .PluginName }}TaskData) {
- data := taskCtx.GetData().(*{{ .PluginName }}TaskData)
- filteredData := *data
- filteredData.Options = &{{ .PluginName }}Options{}
- *filteredData.Options = *data.Options
- var params = {{ .PluginName }}ApiParams{
- ConnectionId: data.Options.ConnectionId,
- ProjectKey: data.Options.ProjectKey,
- HotspotKey: data.Options.HotspotKey,
- }
- rawDataSubTaskArgs := &api.RawDataSubTaskArgs{
- Ctx: taskCtx,
- Params: params,
- Table: rawTable,
- }
- return rawDataSubTaskArgs, &filteredData
-}
diff --git a/backend/generator/template/plugin/tasks/task_data.go-template
b/backend/generator/template/plugin/tasks/task_data.go-template
deleted file mode 100644
index fbb0250c3..000000000
--- a/backend/generator/template/plugin/tasks/task_data.go-template
+++ /dev/null
@@ -1,32 +0,0 @@
-/*
-Licensed to the Apache Software Foundation (ASF) under one or more
-contributor license agreements. See the NOTICE file distributed with
-this work for additional information regarding copyright ownership.
-The ASF licenses this file to You under the Apache License, Version 2.0
-(the "License"); you may not use this file except in compliance with
-the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-package tasks
-
-type {{ .PluginName }}ApiParams struct {
-}
-
-type {{ .PluginName }}Options struct {
- // TODO add some custom options here if necessary
- // options means some custom params required by plugin running.
- // Such As How many rows do your want
- // You can use it in subtasks, and you need to pass it to main.go and
pipelines.
-}
-
-type {{ .PluginName }}TaskData struct {
- Options *{{ .PluginName }}Options
-}
diff --git
a/backend/generator/template/plugin/tasks/task_data_complete_plugin.go-template
b/backend/generator/template/plugin/tasks/task_data_complete_plugin.go-template
deleted file mode 100644
index dcee683af..000000000
---
a/backend/generator/template/plugin/tasks/task_data_complete_plugin.go-template
+++ /dev/null
@@ -1,70 +0,0 @@
-/*
-Licensed to the Apache Software Foundation (ASF) under one or more
-contributor license agreements. See the NOTICE file distributed with
-this work for additional information regarding copyright ownership.
-The ASF licenses this file to You under the Apache License, Version 2.0
-(the "License"); you may not use this file except in compliance with
-the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-package tasks
-
-import (
- "fmt"
- "github.com/apache/incubator-devlake/core/errors"
- helper "github.com/apache/incubator-devlake/helpers/pluginhelper/api"
-)
-
-type {{ .PluginName }}ApiParams struct {
-}
-
-type {{ .PluginName }}Options struct {
- // TODO add some custom options here if necessary
- // options means some custom params required by plugin running.
- // Such As How many rows do your want
- // You can use it in subtasks, and you need to pass it to main.go and
pipelines.
- ConnectionId uint64 `json:"connectionId"`
- Tasks []string `json:"tasks,omitempty"`
- CreatedDateAfter string `json:"createdDateAfter"
mapstructure:"createdDateAfter,omitempty"`
-}
-
-type {{ .PluginName }}TaskData struct {
- Options *{{ .PluginName }}Options
- ApiClient *api.ApiAsyncClient
- CreatedDateAfter *time.Time
-}
-
-func DecodeAndValidateTaskOptions(options map[string]interface{}) (*{{
.PluginName }}Options, errors.Error) {
- var op {{ .PluginName }}Options
- if err := helper.Decode(options, &op, nil); err != nil {
- return nil, err
- }
- if op.ConnectionId == 0 {
- return nil, errors.Default.New("connectionId is invalid")
- }
- return &op, nil
-}
-
-func CreateRawDataSubTaskArgs(taskCtx plugin.SubTaskContext, rawTable string)
(*api.RawDataSubTaskArgs, *{{ .PluginName }}TaskData) {
- data := taskCtx.GetData().(*{{ .PluginName }}TaskData)
- filteredData := *data
- filteredData.Options = &{{ .PluginName }}Options{}
- *filteredData.Options = *data.Options
- var params = {{ .PluginName }}ApiParams{
- ConnectionId: data.Options.ConnectionId,
- }
- rawDataSubTaskArgs := &api.RawDataSubTaskArgs{
- Ctx: taskCtx,
- Params: params,
- Table: rawTable,
- }
- return rawDataSubTaskArgs, &filteredData
-}
diff --git a/backend/generator/util/template.go
b/backend/generator/util/template.go
deleted file mode 100644
index 3dc039d65..000000000
--- a/backend/generator/util/template.go
+++ /dev/null
@@ -1,103 +0,0 @@
-/*
-Licensed to the Apache Software Foundation (ASF) under one or more
-contributor license agreements. See the NOTICE file distributed with
-this work for additional information regarding copyright ownership.
-The ASF licenses this file to You under the Apache License, Version 2.0
-(the "License"); you may not use this file except in compliance with
-the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-package util
-
-import (
- "fmt"
- "github.com/iancoleman/strcase"
- "github.com/spf13/cobra"
- "os"
- "path/filepath"
- "regexp"
- "strings"
-)
-
-// GenerateAllFormatVar fill all format var into values
-func GenerateAllFormatVar(values map[string]string, baseVarName, baseValue
string) {
- values[strcase.ToLowerCamel(baseVarName)] =
strcase.ToLowerCamel(baseValue)
- values[strcase.ToCamel(baseVarName)] = strcase.ToCamel(baseValue)
- values[strcase.ToSnake(baseVarName)] = strcase.ToSnake(baseValue)
- values[strcase.ToScreamingSnake(baseVarName)] =
strcase.ToScreamingSnake(baseValue)
- values[strcase.ToKebab(baseVarName)] = strcase.ToKebab(baseValue)
- values[strcase.ToScreamingKebab(baseVarName)] =
strcase.ToScreamingKebab(baseValue)
-}
-
-// ReadTemplate read a file to string
-func ReadTemplate(templateFile string) string {
- f, err := os.ReadFile(templateFile)
- cobra.CheckErr(err)
- return string(f)
-}
-
-// WriteTemplates write some strings to files
-func WriteTemplates(path string, templates map[string]string) {
- err := os.MkdirAll(path, 0755)
- cobra.CheckErr(err)
- for name, template := range templates {
- err := os.MkdirAll(filepath.Dir(filepath.Join(path, name)),
0755)
- cobra.CheckErr(err)
- err = os.WriteFile(filepath.Join(path, name), []byte(template),
0600)
- cobra.CheckErr(err)
- println(filepath.Join(path, name), ` generated`)
- }
-}
-
-// ReplaceVarInFile replacte var into file without reading
-func ReplaceVarInFile(filename string, reg *regexp.Regexp, new string) {
- f, err := os.ReadFile(filename)
- cobra.CheckErr(err)
- f = reg.ReplaceAll(f, []byte(new))
-
- err = os.WriteFile(filename, f, 0777)
- cobra.CheckErr(err)
- println(filename, ` updated`)
-}
-
-// DetectExistVars filter the used vars in templates
-func DetectExistVars(templates map[string]string, values map[string]string)
(newValues map[string]string) {
- newValues = map[string]string{}
- for varName, value := range values {
- for _, template := range templates {
- if strings.Contains(template, varName) {
- newValues[varName] = value
- break
- }
- }
- }
- return newValues
-}
-
-// ReplaceVarInTemplates replace var with templates into templates
-func ReplaceVarInTemplates(templates map[string]string, valueMap
map[string]string) {
- for i, template := range templates {
- templates[i] = ReplaceVars(template, valueMap)
- }
-}
-
-// ReplaceVars will replace s with valueMap and return it
-func ReplaceVars(s string, valueMap map[string]string) string {
- for varName, value := range valueMap {
- s = ReplaceVar(s, varName, value)
- }
- return s
-}
-
-// ReplaceVar will replace s with value and return it
-func ReplaceVar(s, varName, value string) string {
- return strings.ReplaceAll(s, fmt.Sprintf(`{{ .%s }}`, varName), value)
-}
diff --git a/backend/python/README.md b/backend/python/README.md
index 148f96257..912f84d76 100644
--- a/backend/python/README.md
+++ b/backend/python/README.md
@@ -1,6 +1,6 @@
# Pydevlake
-Pydevlake is a framework for writing plugins plugins for
[DevLake](https://devlake.apache.org/). The framework source code
+Pydevlake is a framework for writing plugins for
[DevLake](https://devlake.apache.org/) in Python. The framework source code
can be found in [here](./pydevlake).
