This is an automated email from the ASF dual-hosted git repository.
kezhenxu94 pushed a commit to branch doc/dev
in repository https://gitbox.apache.org/repos/asf/skywalking-python.git
commit 393d795f2a08769e7e20c09885b0d56fd85c5a9f
Author: kezhenxu94 <kezhenx...@163.com>
AuthorDate: Thu Aug 6 21:54:20 2020 +0800
[Doc] Add dev doc and reorganize the structure
closes https://github.com/apache/skywalking/issues/4993
---
CONTRIBUTING.md | 37 +++++++++++++++++++++
README.md | 61 +++++------------------------------
docs/Developer.md | 23 +++++++++++++
docs/EnvVars.md | 19 +++++++++++
docs/PluginTest.md | 5 +++
docs/Plugins.md | 14 ++++++++
requirements.txt | 26 +++++++++++----
tools/env/build_requirements_linux.sh | 6 ++--
tools/env/setup_linux.sh | 14 ++++++--
9 files changed, 140 insertions(+), 65 deletions(-)
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
new file mode 100644
index 0000000..afeea15
--- /dev/null
+++ b/CONTRIBUTING.md
@@ -0,0 +1,37 @@
+# Contributing to Apache SkyWalking Python
+
+Firstly, thanks for your interest in contributing! We hope that this will be a
pleasant experience for you,
+so that you will return to continue contributing.
+
+## Code of Conduct
+
+The project and everyone participating in it is governed by the Apache
software Foundation's [Code of
Conduct](http://www.apache.org/foundation/policies/conduct.html).
+By participating, you are expected to adhere to this code. If you are aware of
unacceptable behavior, please visit the [Reporting Guidelines
page](http://www.apache.org/foundation/policies/conduct.html#reporting-guidelines)
+and follow the instructions there.
+
+## Contribute
+
+Most of the contributions that we receive are code contributions, but you can
also contribute to the documentation or simply report solid bugs for us to fix.
+
+## Report a bug
+
+* **Ensure the bug was not already reported** by searching on GitHub under
[Issues](https://github.com/apache/skywalking/issues).
+
+* If you're unable to find an issue addressing the problem, [open a new
one](https://github.com/apache/skywalking/issues/new).
+Be sure to include a **title and clear description**, as much relevant
information as possible, and a **code sample** or an **executable test case**
demonstrating the expected behavior that is not happening.
+
+## Add a new feature or enhance an existing one
+
+_Before making any significant changes, please [open an
issue](https://github.com/apache/skywalking/issues)._
+Discussing your proposed changes ahead of time will make the contribution
process smooth for everyone.
+
+Once we've discussed your changes and you've got your code ready, make sure
that tests are passing and open your pull request. Your PR is most likely to be
accepted if it:
+
+* Update the README.md with details of changes to the interface.
+* Includes tests for new functionality.
+* References the original issue in description, e.g. "Resolves #123", "Closes
#123", or "Fixes #123", etc.
+* Has a [good commit
message](http://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html).
+
+## Questions about the source code
+
+* Join `#skywalking` channel at [Apache
Slack](https://s.apache.org/slack-invite)
diff --git a/README.md b/README.md
index fd8521c..f9e51c0 100755
--- a/README.md
+++ b/README.md
@@ -36,7 +36,7 @@ Refer to the [FAQ](docs/FAQ.md#q-how-to-build-from-sources).
SkyWalking Python SDK requires SkyWalking 8.0+ and Python 3.5+.
-> If you want to try out the latest features that're not released yet, please
refer to [the guide](docs/FAQ.md#q-how-to-build-from-sources) to build from
sources.
+> If you want to try out the latest features that are not released yet, please
refer to [the guide](docs/FAQ.md#q-how-to-build-from-sources) to build from
sources.
```python
from skywalking import agent, config
@@ -45,45 +45,13 @@ config.init(collector='127.0.0.1:11800', service='your
awesome service')
agent.start()
```
-Alternatively, you can also pass the configurations via environment variables
and you don't need to call `config.init`.
-
-The supported environment variables are as follows:
-
-Environment Variable | Description | Default
-| :--- | :--- | :--- |
-| `SW_AGENT_NAME` | The name of the Python service | `Python Service Name` |
-| `SW_AGENT_INSTANCE` | The name of the Python service instance | Randomly
generated |
-| `SW_AGENT_COLLECTOR_BACKEND_SERVICES` | The backend OAP server address |
`127.0.0.1:11800` |
-| `SW_AGENT_PROTOCOL` | The protocol to communicate with the backend OAP,
`http` or `grpc`, **we highly suggest using `grpc` in production as it's well
optimized than `http`** | `grpc` |
-| `SW_AGENT_AUTHENTICATION` | The authentication token to verify that the
agent is trusted by the backend OAP, as for how to configure the backend, refer
to [the
yaml](https://github.com/apache/skywalking/blob/4f0f39ffccdc9b41049903cc540b8904f7c9728e/oap-server/server-bootstrap/src/main/resources/application.yml#L155-L158).
| not set |
-| `SW_AGENT_LOGGING_LEVEL` | The logging level, could be one of `CRITICAL`,
`FATAL`, `ERROR`, `WARN`(`WARNING`), `INFO`, `DEBUG` | `INFO` |
-| `SW_AGENT_DISABLE_PLUGINS` | The name patterns in CSV pattern, plugins whose
name matches one of the pattern won't be installed | `''` |
-| `SW_MYSQL_TRACE_SQL_PARAMETERS` | Indicates whether to collect the sql
parameters or not | `False` |
-| `SW_MYSQL_SQL_PARAMETERS_MAX_LENGTH` | The maximum length of the collected
parameter, parameters longer than the specified length will be truncated |
`512` |
-| `SW_IGNORE_SUFFIX` | If the operation name of the first span is included in
this set, this segment should be ignored. |
`.jpg,.jpeg,.js,.css,.png,.bmp,.gif,.ico,.mp3,.mp4,.html,.svg` |
-| `SW_FLASK_COLLECT_HTTP_PARAMS`| This config item controls that whether the
Flask plugin should collect the parameters of the request.| `false` |
-| `SW_DJANGO_COLLECT_HTTP_PARAMS`| This config item controls that whether the
Django plugin should collect the parameters of the request.| `false` |
-| `SW_HTTP_PARAMS_LENGTH_THRESHOLD`| When `COLLECT_HTTP_PARAMS` is enabled,
how many characters to keep and send to the OAP backend, use negative values to
keep and send the complete parameters, NB. this config item is added for the
sake of performance. | `1024` |
-| `SW_CORRELATION_ELEMENT_MAX_NUMBER`|Max element count of the correlation
context.| `3` |
-| `SW_CORRELATION_VALUE_MAX_LENGTH`| Max value length of correlation context
element.| `128` |
+Alternatively, you can also pass the configurations via environment variables
(such as `SW_AGENT_NAME`, `SW_AGENT_COLLECTOR_BACKEND_SERVICES`, etc.) so that
you don't need to call `config.init`.
+All supported environment variables can be found [here](docs/EnvVars.md)
## Supported Libraries
-There're some built-in plugins that support automatic instrumentation of
Python libraries, the complete lists are as follow:
-
-Library | Plugin Name
-| :--- | :--- |
-| [http.server](https://docs.python.org/3/library/http.server.html) |
`sw_http_server` |
-| [urllib.request](https://docs.python.org/3/library/urllib.request.html) |
`sw_urllib_request` |
-| [requests](https://requests.readthedocs.io/en/master/) | `sw_requests` |
-| [Flask](https://flask.palletsprojects.com/en/1.1.x/) | `sw_flask` |
-| [PyMySQL](https://pymysql.readthedocs.io/en/latest/) | `sw_pymysql` |
-| [Django](https://www.djangoproject.com/) | `sw_django` |
-| [redis-py](https://github.com/andymccurdy/redis-py/) | `sw_redis` |
-| [kafka-python](https://kafka-python.readthedocs.io/en/master/) | `sw_kafka` |
-| [tornado](https://www.tornadoweb.org/en/stable/) | `sw_tornado` |
-| [pika](https://pika.readthedocs.io/en/stable/) | `sw_rabbitmq` |
+There are some built-in plugins (such as `http.server`, `Flask`, `Django`
etc.) that support automatic instrumentation of Python libraries, the complete
lists can be found [here](docs/Plugins.md)
## API
@@ -137,26 +105,13 @@ with
context.new_entry_span(op=str('https://github.com/apache/skywalking')) as s
some_method()
```
-## FAQs
-
-Check [the FAQ page](docs/FAQ.md) or add the FAQs there.
-
-## For Developers
+## Contributing
-### Steps to get an operational virtual environment:
+Before submitting a pull request or push a commit, please read our
[contributing](CONTRIBUTING.md) and [developer guide](docs/Developer.md).
-1. `git clone https://github.com/apache/skywalking-python.git`
-2. `cd skywalking-python/tools/env` (*make sure you actually go into the
directory since the scripts use relative paths*)
-3. Run the script for your relevant OS to create a virtual environment folder
in the project root (*skywalking-python/venv*) and install all the necessary
requirements
-
-**Make sure that when the `python` command is executed on your workstation,
the binary it references is python 3.5 or newer!**
-
-### Steps after contributing
+## FAQs
-If your PR introduces the need for a new non-standard library which needs to
be pulled via pip or if it removes the need for a previously-used library:
-1. navigate to `/path/to/skywalking/agent/tools/env`
-2. Execute the `build_requirements` script relevant to your OS.
-3. Double check the `requirements.txt` file in the project root to ensure that
the changes have been reflected.
+Check [the FAQ page](docs/FAQ.md) or add the FAQs there.
## License
Apache 2.0
diff --git a/docs/Developer.md b/docs/Developer.md
new file mode 100644
index 0000000..0051e68
--- /dev/null
+++ b/docs/Developer.md
@@ -0,0 +1,23 @@
+# Developers' Guide
+
+## Steps to get an operational virtual environment:
+
+1. `git clone https://github.com/apache/skywalking-python.git`
+1. Run the script(`setup-linux.sh`, `setup-windows.ps1`) for your relevant OS
to create a virtual environment folder in the project root
+(*skywalking-python/venv*) and install all the necessary requirements
+1. Set up your IDE to use the generated virtual environment of Python
+
+## Developing a new plugin
+
+You can always take [the existing plugins](../skywalking/plugins) as examples,
while there are some general ideas for all plugins.
+1. A plugin is a module under directory
[`skywalking/plugins/`](../skywalking/plugins) with an `install` method;
+1. Inside the `install` method, you find out the relevant method(s) of the
libraries that you plan to instrument, and create/close spans before/after
those method(s).
+1. Every plugin requires a corresponding test under
[`tests/plugin`](../tests/plugin) before it can be merged, refer to [the plugin
test guide](PluginTest.md) when writing a plugin test.
+1. Update [the supported list](Plugins.md).
+1. Add the environment variables to [the list](EnvVars.md) if any.
+
+## Steps after coding
+
+If your PR introduces the need for a new non-standard library which needs to
be pulled via pip or if it removes the need for a previously-used library:
+1. Execute the [`build_requirements`
script](../tools/env/build_requirements_linux.sh) relevant to your OS.
+1. Double check the `requirements.txt` file in the project root to ensure that
the changes have been reflected.
diff --git a/docs/EnvVars.md b/docs/EnvVars.md
new file mode 100644
index 0000000..7f1714d
--- /dev/null
+++ b/docs/EnvVars.md
@@ -0,0 +1,19 @@
+# Supported Environment Variables
+
+Environment Variable | Description | Default
+| :--- | :--- | :--- |
+| `SW_AGENT_NAME` | The name of the Python service | `Python Service Name` |
+| `SW_AGENT_INSTANCE` | The name of the Python service instance | Randomly
generated |
+| `SW_AGENT_COLLECTOR_BACKEND_SERVICES` | The backend OAP server address |
`127.0.0.1:11800` |
+| `SW_AGENT_PROTOCOL` | The protocol to communicate with the backend OAP,
`http` or `grpc`, **we highly suggest using `grpc` in production as it's well
optimized than `http`** | `grpc` |
+| `SW_AGENT_AUTHENTICATION` | The authentication token to verify that the
agent is trusted by the backend OAP, as for how to configure the backend, refer
to [the
yaml](https://github.com/apache/skywalking/blob/4f0f39ffccdc9b41049903cc540b8904f7c9728e/oap-server/server-bootstrap/src/main/resources/application.yml#L155-L158).
| unset |
+| `SW_AGENT_LOGGING_LEVEL` | The logging level, could be one of `CRITICAL`,
`FATAL`, `ERROR`, `WARN`(`WARNING`), `INFO`, `DEBUG` | `INFO` |
+| `SW_AGENT_DISABLE_PLUGINS` | The name patterns in CSV pattern, plugins whose
name matches one of the pattern won't be installed | `''` |
+| `SW_MYSQL_TRACE_SQL_PARAMETERS` | Indicates whether to collect the sql
parameters or not | `False` |
+| `SW_MYSQL_SQL_PARAMETERS_MAX_LENGTH` | The maximum length of the collected
parameter, parameters longer than the specified length will be truncated |
`512` |
+| `SW_IGNORE_SUFFIX` | If the operation name of the first span is included in
this set, this segment should be ignored. |
`.jpg,.jpeg,.js,.css,.png,.bmp,.gif,.ico,.mp3,.mp4,.html,.svg` |
+| `SW_FLASK_COLLECT_HTTP_PARAMS`| This config item controls that whether the
Flask plugin should collect the parameters of the request.| `false` |
+| `SW_DJANGO_COLLECT_HTTP_PARAMS`| This config item controls that whether the
Django plugin should collect the parameters of the request.| `false` |
+| `SW_HTTP_PARAMS_LENGTH_THRESHOLD`| When `COLLECT_HTTP_PARAMS` is enabled,
how many characters to keep and send to the OAP backend, use negative values to
keep and send the complete parameters, NB. this config item is added for the
sake of performance. | `1024` |
+| `SW_CORRELATION_ELEMENT_MAX_NUMBER`|Max element count of the correlation
context.| `3` |
+| `SW_CORRELATION_VALUE_MAX_LENGTH`| Max value length of correlation context
element.| `128` |
diff --git a/docs/PluginTest.md b/docs/PluginTest.md
index f267157..d49cd9c 100644
--- a/docs/PluginTest.md
+++ b/docs/PluginTest.md
@@ -30,3 +30,8 @@ If we want to test the plugin for the built-in library
`http`, we will:
1. Build a tested service, which sets up an http server by `http` library, and
exposes an http endpoint to be triggered in the test codes, say `/trigger`,
take [this provider service](../tests/plugin/sw_http/services/provider.py) as
example.
1. Compose a `docker-compose.yml` file, orchestrating the service built in
step 1 and the mock collector, take [this
`docker-compose.yml`](../tests/plugin/sw_http/docker-compose.yml) as example.
1. Write test codes to trigger the endpoint int step 1, and send the expected
data file to the mock collector to verify, take [this
test](../tests/plugin/sw_http/test_http.py) as example.
+
+## Notes
+
+Remember to add the library/module into the [setup.py](../setup.py)
`extras_require/test` so that other developers can have it installed
+after pulling your commits, and run test locally.
diff --git a/docs/Plugins.md b/docs/Plugins.md
new file mode 100644
index 0000000..f8c3d2b
--- /dev/null
+++ b/docs/Plugins.md
@@ -0,0 +1,14 @@
+# Supported Libraries
+
+Library | Plugin Name
+| :--- | :--- |
+| [http.server](https://docs.python.org/3/library/http.server.html) |
`sw_http_server` |
+| [urllib.request](https://docs.python.org/3/library/urllib.request.html) |
`sw_urllib_request` |
+| [requests](https://requests.readthedocs.io/en/master/) | `sw_requests` |
+| [Flask](https://flask.palletsprojects.com/en/1.1.x/) | `sw_flask` |
+| [PyMySQL](https://pymysql.readthedocs.io/en/latest/) | `sw_pymysql` |
+| [Django](https://www.djangoproject.com/) | `sw_django` |
+| [redis-py](https://github.com/andymccurdy/redis-py/) | `sw_redis` |
+| [kafka-python](https://kafka-python.readthedocs.io/en/master/) | `sw_kafka` |
+| [tornado](https://www.tornadoweb.org/en/stable/) | `sw_tornado` |
+| [pika](https://pika.readthedocs.io/en/stable/) | `sw_rabbitmq` |
diff --git a/requirements.txt b/requirements.txt
index 01424ca..4732bc5 100755
--- a/requirements.txt
+++ b/requirements.txt
@@ -1,23 +1,35 @@
+asgiref==3.2.10
blindspin==2.0.1
certifi==2020.6.20
chardet==3.0.4
+click==7.1.2
colorama==0.4.3
crayons==0.3.1
deprecation==2.1.0
+Django==3.1
docker==4.2.2
-grpcio==1.30.0
-grpcio-tools==1.30.0
+Flask==1.1.2
+grpcio==1.31.0
+grpcio-tools==1.31.0
idna==2.10
+itsdangerous==1.1.0
+Jinja2==2.11.2
+kafka-python==2.0.1
+MarkupSafe==1.1.1
packaging==20.4
-protobuf==3.12.2
-PyMySQL==0.9.3
+pika==1.1.0
+protobuf==3.12.4
+PyMySQL==0.10.0
pyparsing==2.4.7
-pypiwin32==223
-pywin32==228
+pytz==2020.1
+PyYAML==5.3.1
+redis==3.5.3
requests==2.24.0
six==1.15.0
+sqlparse==0.3.1
testcontainers==3.0.3
-urllib3==1.25.9
+tornado==6.0.4
+urllib3==1.25.10
websocket-client==0.57.0
Werkzeug==1.0.1
wrapt==1.12.1
diff --git a/tools/env/build_requirements_linux.sh
b/tools/env/build_requirements_linux.sh
index ba28139..9cdfea5 100755
--- a/tools/env/build_requirements_linux.sh
+++ b/tools/env/build_requirements_linux.sh
@@ -17,8 +17,10 @@
#!/bin/sh
+root_dir=$(dirname "$0")/../../
+
echo "Building requirements.txt"
-../../venv/bin /python -m pip freeze > ../../requirements.txt
+${root_dir}/venv/bin/python -m pip freeze > ${root_dir}/requirements.txt
-echo "requirements.txt created"
\ No newline at end of file
+echo "requirements.txt created"
diff --git a/tools/env/setup_linux.sh b/tools/env/setup_linux.sh
index a3cb289..39ea202 100755
--- a/tools/env/setup_linux.sh
+++ b/tools/env/setup_linux.sh
@@ -17,10 +17,18 @@
#!/bin/sh
+root_dir=$(dirname "$0")/../../
+
echo "Creating virtual environment"
-python3 -m venv '../../venv'
+
+python3 -m venv "${root_dir}/venv"
+
echo "Virtual env created"
-../../venv/bin/python -m pip install --upgrade pip
+source ${root_dir}/venv/bin/activate
+
+${root_dir}/venv/bin/python -m pip install --upgrade pip
+
echo "Pip upgrade complete. Installing packages from requirements.txt"
-../../venv/bin/python -m pip install -r ../../requirements.txt
+
+${root_dir}/venv/bin/python -m pip install -r ${root_dir}/requirements.txt
