[
https://issues.apache.org/jira/browse/COMDEV-462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiajie Zhong updated COMDEV-462:
--------------------------------
Description:
h1. About pydolphinscheduler
*PyDolphinScheduler* is Python API for [Apache
DolphinScheduler|https://dolphinscheduler.apache.org/], which allows you to
define your workflow by Python code, aka workflow-as-codes. You could see more
detail about PyDolphinScheduler in its document[4]. And all the source code
hold as the submodule in DolphinScheduler main codebase[5].
h1. The Goal
Make pydolphinscheduler's CLI more powerful, make it can operate the model of
DolphinScheduler, run pydolphinscheduler's code, visualize its DAG graph in the
terminal.
h1. Detail
Up to now, Apache DolphinScheduler Python API has CLI only with limited command
supported and our community wishes it to become a more powerful tool and
support as much command as possible(unless command has security issue).
It only supports `version` and `config` for now, which you could see more
detail in [1]
Basically, we think the following command is helpful for CLI and you could add
another command if it should be added(but may sure after discussing in the
community):
* `run <DAG name> [--example]`: Run local workflow DAG file or examples
build-in
* `users`: User's operation, CURD
* `projects`: Project's operation, CURD, grant to other users
* `tenants`: Tenant's operation, CURD
* `workflow`: Workflow's operation, CURD, name change, should also change the
local Python file name
* `visualize`: Show task graph in the terminal.
* etc...
Besides the functional addition, we should also consider the output part of CLI
which makes our output more clear and cool. We *may* consider using (we should
also find other interesting packages to do it):
* rich: For highlight, our output, or using some existing rich plugin like
`click-rich`
* tabulate: For the tables visualization in terminal
h1. What Can You Learn
We wish everyone joining GSoC could learn some things from the project. When
you finish this project, you could learn:
* How to write production-level Python codes and docs, you could improve your
Python syntax, how to write tests with `pytest` and `tox`, how to write a
document with `sphnix` and it related plugin, how to format your Python code
and the linter inside
* Adding knowledge about task scheduling system, what is it and what it
focuses, how it could be run
h1. If You Interested in It
If you want to take this ticket, you should
* (Must) Python skill, especially packages click, pytest and etc.
* Have a little knowledge of task scheduling systems.
* (Optional) Basic Java knowledge is better because Apache DolphinScheduler
core is written with Java and you may add some functional code to it.
h1. Mentors
* Calvin Kirs: Committer of Apache \{DolphinScheduler, SeaTunnel, Wayang},
DolphinScheduler PMC and SeaTunnel PPMC
* Jiajie Zhong: Committer of Apache \{Airflow, DolphinScheduler, SeaTunnel},
SeaTunnel PPMC
[1]: [https://dolphinscheduler.apache.org/python/cli.html]
[2]: [https://github.com/Textualize/rich]
[3]: [https://github.com/astanin/python-tabulate]
[4]: [https://dolphinscheduler.apache.org/python/index.html]
[5]:
[https://github.com/apache/dolphinscheduler/tree/dev/dolphinscheduler-python/pydolphinscheduler]
was:
h1. About pydolphinscheduler
*PyDolphinScheduler* is Python API for [Apache
DolphinScheduler|https://dolphinscheduler.apache.org/], which allows you to
define your workflow by Python code, aka workflow-as-codes. You could see more
detail about PyDolphinScheduler in its document[4]. And all the source code
hold as the submodule in DolphinScheduler main codebase[5].
h1. The Goal
Make pydolphinscheduler's CLI more powerful, make it can operate the model of
DolphinScheduler, run pydolphinscheduler's code, visualize its DAG graph in the
terminal.
h1. Detail
Up to now, Apache DolphinScheduler Python API has CLI only with limited command
supported and our community wishes it to become a more powerful tool and
support as much command as possible(unless command has security issue).
It only supports `version` and `config` for now, which you could see more
detail in [1]
Basically, we think the following command is helpful for CLI and you could add
another command if it should be added(but may sure after discussing in the
community):
* `run <DAG name> [--example]`: Run local workflow DAG file or examples
build-in
* `users`: User's operation, CURD
* `projects`: Project's operation, CURD, grant to other users
* `tenants`: Tenant's operation, CURD
* `workflow`: Workflow's operation, CURD, name change, should also change the
local Python file name
* `visualize`: Show task graph in the terminal.
* etc...
Besides the functional addition, we should also consider the output part of CLI
which makes our output more clear and cool. We *may* consider using (we should
also find other interesting packages to do it):
* rich: For highlight, our output, or using some existing rich plugin like
`click-rich`
* tabulate: For the tables visualization in terminal
h1. What Can You Learn
We wish everyone joining GSoC could learn some things from the project. When
you finish this project, you could learn:
* How to write production-level Python codes and docs, you could improve your
Python syntax, how to write tests with `pytest` and `tox`, how to write a
document with `sphnix` and it related plugin, how to format your Python code
and the linter inside
* Adding knowledge about task scheduling system, what is it and what it
focuses, how it could be run
h1. If You Interested in It
If you want to take this ticket, you should
* (Must) Python skill, especially packages click, pytest and etc.
* Have a little knowledge of task scheduling systems.
* (Optional) Basic Java knowledge is better because Apache DolphinScheduler
core is written with Java and you may add some functional code to it.
h1. Mentors
* Calvin Kirs: ASF member, Committer of Apache \{DolphinScheduler, SeaTunnel,
Wayang}, DolphinScheduler PMC and SeaTunnel PPMC
* Jiajie Zhong: Committer of Apache \{Airflow, DolphinScheduler, SeaTunnel},
SeaTunnel PPMC
[1]: [https://dolphinscheduler.apache.org/python/cli.html]
[2]: [https://github.com/Textualize/rich]
[3]: [https://github.com/astanin/python-tabulate]
[4]: [https://dolphinscheduler.apache.org/python/index.html]
[5]:
[https://github.com/apache/dolphinscheduler/tree/dev/dolphinscheduler-python/pydolphinscheduler]
> GSoC: Python API CLI enhancement
> --------------------------------
>
> Key: COMDEV-462
> URL: https://issues.apache.org/jira/browse/COMDEV-462
> Project: Community Development
> Issue Type: Task
> Components: GSoC/Mentoring ideas
> Reporter: Jiajie Zhong
> Priority: Major
> Labels: DolphinScheduler, full-time, gsoc2022
>
> h1. About pydolphinscheduler
> *PyDolphinScheduler* is Python API for [Apache
> DolphinScheduler|https://dolphinscheduler.apache.org/], which allows you to
> define your workflow by Python code, aka workflow-as-codes. You could see
> more detail about PyDolphinScheduler in its document[4]. And all the source
> code hold as the submodule in DolphinScheduler main codebase[5].
> h1. The Goal
> Make pydolphinscheduler's CLI more powerful, make it can operate the model of
> DolphinScheduler, run pydolphinscheduler's code, visualize its DAG graph in
> the terminal.
> h1. Detail
> Up to now, Apache DolphinScheduler Python API has CLI only with limited
> command supported and our community wishes it to become a more powerful tool
> and support as much command as possible(unless command has security issue).
> It only supports `version` and `config` for now, which you could see more
> detail in [1]
> Basically, we think the following command is helpful for CLI and you could
> add another command if it should be added(but may sure after discussing in
> the community):
> * `run <DAG name> [--example]`: Run local workflow DAG file or examples
> build-in
> * `users`: User's operation, CURD
> * `projects`: Project's operation, CURD, grant to other users
> * `tenants`: Tenant's operation, CURD
> * `workflow`: Workflow's operation, CURD, name change, should also change
> the local Python file name
> * `visualize`: Show task graph in the terminal.
> * etc...
> Besides the functional addition, we should also consider the output part of
> CLI which makes our output more clear and cool. We *may* consider using (we
> should also find other interesting packages to do it):
> * rich: For highlight, our output, or using some existing rich plugin like
> `click-rich`
> * tabulate: For the tables visualization in terminal
> h1. What Can You Learn
> We wish everyone joining GSoC could learn some things from the project. When
> you finish this project, you could learn:
> * How to write production-level Python codes and docs, you could improve
> your Python syntax, how to write tests with `pytest` and `tox`, how to write
> a document with `sphnix` and it related plugin, how to format your Python
> code and the linter inside
> * Adding knowledge about task scheduling system, what is it and what it
> focuses, how it could be run
> h1. If You Interested in It
> If you want to take this ticket, you should
> * (Must) Python skill, especially packages click, pytest and etc.
> * Have a little knowledge of task scheduling systems.
> * (Optional) Basic Java knowledge is better because Apache DolphinScheduler
> core is written with Java and you may add some functional code to it.
> h1. Mentors
> * Calvin Kirs: Committer of Apache \{DolphinScheduler, SeaTunnel, Wayang},
> DolphinScheduler PMC and SeaTunnel PPMC
> * Jiajie Zhong: Committer of Apache \{Airflow, DolphinScheduler, SeaTunnel},
> SeaTunnel PPMC
>
> [1]: [https://dolphinscheduler.apache.org/python/cli.html]
> [2]: [https://github.com/Textualize/rich]
> [3]: [https://github.com/astanin/python-tabulate]
> [4]: [https://dolphinscheduler.apache.org/python/index.html]
> [5]:
> [https://github.com/apache/dolphinscheduler/tree/dev/dolphinscheduler-python/pydolphinscheduler]
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@community.apache.org
For additional commands, e-mail: dev-h...@community.apache.org
