spacewander commented on a change in pull request #19: URL: https://github.com/apache/apisix-java-plugin-runner/pull/19#discussion_r642164990
########## File path: docs/the-internal-of-apisix-java-plugin-runner.md ########## @@ -0,0 +1,58 @@ +The Internal of apisix java plugin runner + +This article explains the internal design of apisix-java-plugin-runner. + +## Table of Contents + +- [Overview](#overview) +- [Communication](#communication) +- [Serialization](#serialization) +- [Codec](#codec) + +## Overview + +The apisix-java-plugin-runner designed as a tcp server built using [reactor-netty](https://github.com/reactor/reactor-netty), +it provides a `PluginFilter` interface for users to implement. + +Users only need to focus on their business logic, not on the details of how the apisix java plugin runner communicates with APISIX. + +The inter-process communication between them is depicted by the following diagram. + + + +## Communication + +apisix-java-plugin-runner and APISIX use the Unix Domain Socket for inter-process communication, +so they need to be deployed in the same runtime environment. Review comment: the same runtime environment => the same instance ########## File path: docs/development.md ########## @@ -0,0 +1,120 @@ +Development + +This document explains how to get started to develop the apisix-java-plugin-runner. + +Prerequisites +------------- + +* JDK 8 +* APISIX 2.6.0 +* Clone the [apisix-java-plugin-runner](https://github.com/apache/apisix-java-plugin-runner) project. +* Refer to [Debug](how-it-works.md#debug) to build the debug environment. + +Install +------- + +```shell +cd /path/to/apisix-java-plugin-runner +./mvnw install +``` + +Write Filter +------------ + +Refer to the code in the [sample](https://github.com/apache/apisix-java-plugin-runner/tree/main/sample) +to learn how to extend `PluginFilter`, define the order, rewrite requests and stop requests. + +#### Code Location + +You need to put the code in [runner-plugin-sdk](https://github.com/apache/apisix-java-plugin-runner/tree/main/runner-plugin-sdk/src/main/java/org/apache/apisix/plugin/runner) +so that the `apisix-java-plugin-runner.jar` will contain the filter implementation class you wrote when you package it. + +#### The order of filter execution + +The order of execution of the filter in the runner is determined by the value returned by the `public int getOrder()` +function in the implementation class; the smaller the order number, the higher the order of execution. + +#### The name of filter execution + +The requests go through filters that are dynamically configured on APISIX. +For example, if the following configuration is done on APISIX + +```shell +curl http://127.0.0.1:9080/apisix/admin/routes/1 -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d ' +{ + "uri":"/hello", + "plugins":{ + "ext-plugin-pre-req":{ + "conf":[ + { + "name":"FooFilter", + "value":"bar" + } + ] + } + }, + "upstream":{ + "nodes":{ + "127.0.0.1:1980":1 + }, + "type":"roundrobin" + } +}' +``` + +apisix-java-plugin-runner will look for implementation classes named `FooFilter`, +and the name of each filter's implementation class is the return value of its overridden function `public String name()`. + + +#### Rewrite Request + +If you perform the following function call in the filter chain of the implementation class + +* request.setPath() +* request.setHeader() +* request.setArg() + +this means to rewrit the current request, the upstream server will receive +the relevant parameters rewritten here. + +#### Stop Request + +If you perform the following function call in the filter chain of the implementation class + +* response.setStatusCode() +* response.setHeader() +* response.setBody() + +this means to stop the current request, the client will receive +the relevant parameters generated here. + +Test +---- + +### Run Unit Test Suites + +```shell +cd /path/to/apisix-java-plugin-runner + ./mvnw test +``` + + +### Mimic practical environment + +If you want to mimic the practical environment, you need to configure the route on APISIX +by having the request go through the filter you want to test, for example + +```yaml Review comment: Should be json ########## File path: docs/the-internal-of-apisix-java-plugin-runner.md ########## @@ -0,0 +1,58 @@ +The Internal of apisix java plugin runner + +This article explains the internal design of apisix-java-plugin-runner. + +## Table of Contents + +- [Overview](#overview) +- [Communication](#communication) +- [Serialization](#serialization) +- [Codec](#codec) + +## Overview + +The apisix-java-plugin-runner designed as a tcp server built using [reactor-netty](https://github.com/reactor/reactor-netty), +it provides a `PluginFilter` interface for users to implement. + +Users only need to focus on their business logic, not on the details of how the apisix java plugin runner communicates with APISIX. + +The inter-process communication between them is depicted by the following diagram. + + + +## Communication + +apisix-java-plugin-runner and APISIX use the Unix Domain Socket for inter-process communication, +so they need to be deployed in the same runtime environment. + +apisix-java-plugin-runner is managed by APISIX. APISIX starts the apisix-java-plugin-runner when it starts and ends it when it +ends. if the apisix-java-plugin-runner quits in the middle, APISIX will restart it automatically. + +## Serialization + +Refer to [flatbuffers](https://github.com/google/flatbuffers) + +FlatBuffers is a cross platform serialization library architected for maximum memory efficiency. +It allows you to directly access serialized data without parsing/unpacking it first, while still having great forwards/backwards compatibility. Review comment: forward/backward compatibility ########## File path: docs/how-it-works.md ########## @@ -0,0 +1,86 @@ +# How It Works + +This article explains how apisix-java-plugin-runner collaborate with [Apache APISIX](https://apisix.apache.org) to run plugins written in java. + +## Run Mode + +apisix-java-plugin-runner can be run alone or bundled with Apache APISIX. +It depends on whether you need to debug it or run it. + +### Debug + +If you are developing a new plugin and need to debug the code, then you can run the main class +[PluginRunnerApplication](https://github.com/apache/apisix-java-plugin-runner/blob/main/runner-starter/src/main/java/org/apache/apisix/plugin/runner/PluginRunnerApplication.java), +and before start, you need to set the following two environment variables: + +- APISIX_LISTEN_ADDRESS: apisix-java-plugin-runner and APISIX for inter-process communication (Unix Domain Socket) socket type file address. + And do not need to actively create this file, apisix-java-plugin-runner will automatically create this file when it starts. +- APISIX_CONF_EXPIRE_TIME: the time that APISIX's configuration is cached in the apisix-java-plugin-runner process. + +For example, if you start apisix-java-plugin-runner as a jar package, pass the environment variables as follows + +```shell +java -jar -DAPISIX_LISTEN_ADDRESS=/tmp/runner.sock -DAPISIX_CONF_EXPIRE_TIME=3600 /path/to/apisix-java-plugin-runner.jar +``` + +and add the following configure in the `config.yaml` file of APISIX + +```yaml +ext-plugin: + path_for_test: /tmp/runner.sock + cmd: ['echo', 'call runner'] Review comment: We don't need to configure the `cmd` section as we don't need it to work. ########## File path: docs/development.md ########## @@ -0,0 +1,120 @@ +Development + +This document explains how to get started to develop the apisix-java-plugin-runner. + +Prerequisites +------------- + +* JDK 8 +* APISIX 2.6.0 +* Clone the [apisix-java-plugin-runner](https://github.com/apache/apisix-java-plugin-runner) project. +* Refer to [Debug](how-it-works.md#debug) to build the debug environment. + +Install +------- + +```shell +cd /path/to/apisix-java-plugin-runner +./mvnw install +``` + +Write Filter +------------ + +Refer to the code in the [sample](https://github.com/apache/apisix-java-plugin-runner/tree/main/sample) +to learn how to extend `PluginFilter`, define the order, rewrite requests and stop requests. + +#### Code Location + +You need to put the code in [runner-plugin-sdk](https://github.com/apache/apisix-java-plugin-runner/tree/main/runner-plugin-sdk/src/main/java/org/apache/apisix/plugin/runner) +so that the `apisix-java-plugin-runner.jar` will contain the filter implementation class you wrote when you package it. + +#### The order of filter execution + +The order of execution of the filter in the runner is determined by the value returned by the `public int getOrder()` +function in the implementation class; the smaller the order number, the higher the order of execution. + +#### The name of filter execution + +The requests go through filters that are dynamically configured on APISIX. +For example, if the following configuration is done on APISIX + +```shell +curl http://127.0.0.1:9080/apisix/admin/routes/1 -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d ' +{ + "uri":"/hello", + "plugins":{ + "ext-plugin-pre-req":{ + "conf":[ + { + "name":"FooFilter", + "value":"bar" + } + ] + } + }, + "upstream":{ + "nodes":{ + "127.0.0.1:1980":1 + }, + "type":"roundrobin" + } +}' +``` + +apisix-java-plugin-runner will look for implementation classes named `FooFilter`, +and the name of each filter's implementation class is the return value of its overridden function `public String name()`. + + +#### Rewrite Request + +If you perform the following function call in the filter chain of the implementation class + +* request.setPath() +* request.setHeader() +* request.setArg() + +this means to rewrit the current request, the upstream server will receive +the relevant parameters rewritten here. + +#### Stop Request + +If you perform the following function call in the filter chain of the implementation class + +* response.setStatusCode() +* response.setHeader() +* response.setBody() + +this means to stop the current request, the client will receive +the relevant parameters generated here. + +Test +---- + +### Run Unit Test Suites + +```shell +cd /path/to/apisix-java-plugin-runner + ./mvnw test +``` + + +### Mimic practical environment + +If you want to mimic the practical environment, you need to configure the route on APISIX +by having the request go through the filter you want to test, for example + +```yaml +"plugins":{ + "ext-plugin-pre-req":{ + "conf":[ + { + "name":"FooFilter", + "value":"bar" + } + ] + } +} +``` + +and then by making a request to APISIX to trigger the route. Review comment: `and then make a request ` ########## File path: docs/the-internal-of-apisix-java-plugin-runner.md ########## @@ -0,0 +1,58 @@ +The Internal of apisix java plugin runner + +This article explains the internal design of apisix-java-plugin-runner. + +## Table of Contents + +- [Overview](#overview) +- [Communication](#communication) +- [Serialization](#serialization) +- [Codec](#codec) + +## Overview + +The apisix-java-plugin-runner designed as a tcp server built using [reactor-netty](https://github.com/reactor/reactor-netty), Review comment: tcp => TCP ########## File path: docs/the-internal-of-apisix-java-plugin-runner.md ########## @@ -0,0 +1,58 @@ +The Internal of apisix java plugin runner + +This article explains the internal design of apisix-java-plugin-runner. + +## Table of Contents + +- [Overview](#overview) +- [Communication](#communication) +- [Serialization](#serialization) +- [Codec](#codec) + +## Overview + +The apisix-java-plugin-runner designed as a tcp server built using [reactor-netty](https://github.com/reactor/reactor-netty), +it provides a `PluginFilter` interface for users to implement. + +Users only need to focus on their business logic, not on the details of how the apisix java plugin runner communicates with APISIX. + +The inter-process communication between them is depicted by the following diagram. + + + +## Communication + +apisix-java-plugin-runner and APISIX use the Unix Domain Socket for inter-process communication, +so they need to be deployed in the same runtime environment. + +apisix-java-plugin-runner is managed by APISIX. APISIX starts the apisix-java-plugin-runner when it starts and ends it when it +ends. if the apisix-java-plugin-runner quits in the middle, APISIX will restart it automatically. + +## Serialization + +Refer to [flatbuffers](https://github.com/google/flatbuffers) + +FlatBuffers is a cross platform serialization library architected for maximum memory efficiency. +It allows you to directly access serialized data without parsing/unpacking it first, while still having great forwards/backwards compatibility. + +You can refer to the [ext-plugin.fbs](https://github.com/api7/ext-plugin-proto/blob/main/ext-plugin.fbs) + schema file to see how lua and java lay out the serialized objects. + +## Codec + +apisix-java-plugin-runner and APISIX use a private binary protocol for coding and decoding. +The protocol format is + +``` +1 byte of type + 3 bytes of length + data +``` + +The type can be 0 ~ 7, and the length can be [0, 8M). data length is determined by length. Review comment: data length is => The length of data is ########## File path: docs/the-internal-of-apisix-java-plugin-runner.md ########## @@ -0,0 +1,58 @@ +The Internal of apisix java plugin runner + +This article explains the internal design of apisix-java-plugin-runner. + +## Table of Contents + +- [Overview](#overview) +- [Communication](#communication) +- [Serialization](#serialization) +- [Codec](#codec) + +## Overview + +The apisix-java-plugin-runner designed as a tcp server built using [reactor-netty](https://github.com/reactor/reactor-netty), +it provides a `PluginFilter` interface for users to implement. + +Users only need to focus on their business logic, not on the details of how the apisix java plugin runner communicates with APISIX. + +The inter-process communication between them is depicted by the following diagram. + + + +## Communication + +apisix-java-plugin-runner and APISIX use the Unix Domain Socket for inter-process communication, +so they need to be deployed in the same runtime environment. + +apisix-java-plugin-runner is managed by APISIX. APISIX starts the apisix-java-plugin-runner when it starts and ends it when it +ends. if the apisix-java-plugin-runner quits in the middle, APISIX will restart it automatically. + +## Serialization + +Refer to [flatbuffers](https://github.com/google/flatbuffers) + +FlatBuffers is a cross platform serialization library architected for maximum memory efficiency. +It allows you to directly access serialized data without parsing/unpacking it first, while still having great forwards/backwards compatibility. + +You can refer to the [ext-plugin.fbs](https://github.com/api7/ext-plugin-proto/blob/main/ext-plugin.fbs) + schema file to see how lua and java lay out the serialized objects. Review comment: how lua and java lay out => how Lua and Java layout ########## File path: docs/how-it-works.md ########## @@ -0,0 +1,86 @@ +# How It Works + +This article explains how apisix-java-plugin-runner collaborate with [Apache APISIX](https://apisix.apache.org) to run plugins written in java. + +## Run Mode + +apisix-java-plugin-runner can be run alone or bundled with Apache APISIX. +It depends on whether you need to debug it or run it. + +### Debug + +If you are developing a new plugin and need to debug the code, then you can run the main class +[PluginRunnerApplication](https://github.com/apache/apisix-java-plugin-runner/blob/main/runner-starter/src/main/java/org/apache/apisix/plugin/runner/PluginRunnerApplication.java), +and before start, you need to set the following two environment variables: + +- APISIX_LISTEN_ADDRESS: apisix-java-plugin-runner and APISIX for inter-process communication (Unix Domain Socket) socket type file address. + And do not need to actively create this file, apisix-java-plugin-runner will automatically create this file when it starts. +- APISIX_CONF_EXPIRE_TIME: the time that APISIX's configuration is cached in the apisix-java-plugin-runner process. + +For example, if you start apisix-java-plugin-runner as a jar package, pass the environment variables as follows + +```shell +java -jar -DAPISIX_LISTEN_ADDRESS=/tmp/runner.sock -DAPISIX_CONF_EXPIRE_TIME=3600 /path/to/apisix-java-plugin-runner.jar +``` + +and add the following configure in the `config.yaml` file of APISIX + +```yaml +ext-plugin: + path_for_test: /tmp/runner.sock + cmd: ['echo', 'call runner'] +``` + +The `/tmp/runner.sock` is the address of the file where apisix-java-plugin-runner +and APISIX communicate between processes and must be consistent. + +Note: If you see some error logs like + +``` +phase_func(): failed to connect to the unix socket unix:/tmp/runner.sock: permission denied Review comment: Why there is permission denied when the sock file is created by runner itself? ########## File path: docs/how-it-works.md ########## @@ -0,0 +1,86 @@ +# How It Works + +This article explains how apisix-java-plugin-runner collaborate with [Apache APISIX](https://apisix.apache.org) to run plugins written in java. + +## Run Mode + +apisix-java-plugin-runner can be run alone or bundled with Apache APISIX. +It depends on whether you need to debug it or run it. + +### Debug + +If you are developing a new plugin and need to debug the code, then you can run the main class +[PluginRunnerApplication](https://github.com/apache/apisix-java-plugin-runner/blob/main/runner-starter/src/main/java/org/apache/apisix/plugin/runner/PluginRunnerApplication.java), +and before start, you need to set the following two environment variables: + +- APISIX_LISTEN_ADDRESS: apisix-java-plugin-runner and APISIX for inter-process communication (Unix Domain Socket) socket type file address. + And do not need to actively create this file, apisix-java-plugin-runner will automatically create this file when it starts. +- APISIX_CONF_EXPIRE_TIME: the time that APISIX's configuration is cached in the apisix-java-plugin-runner process. + +For example, if you start apisix-java-plugin-runner as a jar package, pass the environment variables as follows + +```shell +java -jar -DAPISIX_LISTEN_ADDRESS=/tmp/runner.sock -DAPISIX_CONF_EXPIRE_TIME=3600 /path/to/apisix-java-plugin-runner.jar Review comment: The environment variable APISIX_LISTEN_ADDRESS set by APISIX is "unix:/tmp/runner.sock", see https://github.com/apache/apisix/blob/09e298302cbc8514d8240c300974746c9f85845c/apisix/plugins/ext-plugin/helper.lua#L45 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
