AydarZaynutdinov commented on a change in pull request #16285: URL: https://github.com/apache/beam/pull/16285#discussion_r774450253
########## File path: playground/backend/README.md ########## @@ -21,47 +21,81 @@ ## About -Apache Beam is an open-source, unified model for defining parallel processing pipelines for batch and streaming data. -It provides a portable API layer for building sophisticated data-parallel processing pipelines that may be executed -across a diversity of execution engines, or runners. - -Beam Playground helps facilitate trying out and adopting Apache Beam by providing a very quick way for prospective Beam -users to see and run examples of Apache Beam pipelines in a web interface that requires no setup. +Beam Playground helps facilitate trying out and adopting [Apache Beam](https://beam.apache.org/) by providing a very +quick way for prospective Beam users to see and run examples of Apache Beam pipelines in a web interface that requires +no setup. ## Getting Started See [playground/README.md](../README.md) for details on requirements and setup. This section describes what is needed to run the backend application. -- Generating models from proto file -- Go commands to run/test application locally + +- Go commands to run/test the backend locally +- Set up environment variables to run the backend locally +- Running the backend via Docker ### Go commands to run/test application locally -The following command is used to build and serve the backend locally: +Go to the backend directory: +```shell +$ cd backend ``` -go run ./cmd/server/server.go + +The following command is used to build and serve the backend locally: + +```shell +$ go run ./cmd/server/server.go ``` Run the following command to generate a release build file: -``` -go build ./cmd/server/server.go +```shell +$ go build ./cmd/server/server.go ``` Playground tests may be run using this command: -``` -go test ... -v +```shell +$ go test ... -v ``` The full list of commands can be found [here](https://pkg.go.dev/cmd/go). -## Running the server app +### Set up environment variables to run the backend locally + +These environment variables should be set to run the backend locally: -To run the server using Docker images there are `Docker` files in the `containers` folder for Go and Java languages. -Each of them processes the corresponding SDK, so the backend with Go SDK will work with Go examples/katas/tests only. +- `BEAM_SDK` - is the SDK which backend could process (`SDK_GO` / `SDK_JAVA` / `SDK_PYTHON`) +- `APP_WORK_DIR` - is the directory where all folders will be placed to process each code processing request +- `PREPARED_MOD_DIR` - is the directory where prepared go.mod and go.sum files are placed. It is used only for Go SDK + +There are also environment variables which are needed for the deployment of Apache Beam Playground. These variables have +default value and there is no need to set them up to launch locally: + +- `SERVER_IP` - is the IP address of the backend server (default value = `localhost`) +- `SERVER_PORT` - is the PORT of the backend server (default value = `8080`) +- `CACHE_TYPE` - is a type of the cache service which is used for the backend server. If it is set as a `remote`, then + the backend server will use Redis to keep all cache values (default value = `local`) +- `CACHE_ADDRESS` - is an address of the Redis server. It is used only when `CACHE_TYPE=remote` (default value + = `localhost:6379`) +- `BEAM_PATH` - it is the place where all required for the Java SDK libs are placed + (default value = `/opt/apache/beam/jars/*`) +- `KEY_EXPIRATION_TIME` - is the expiration time of the keys in the cache (default value = `15 min`) +- `PIPELINE_EXPIRATION_TIMEOUT` - is the expiration time of the code processing (default value = `15 min`) +- `PROTOCOL_TYPE` - is the type of the backend server protocol. It could be `TCP` or `HTTP` (default value = `HTTP`) +- `NUM_PARALLEL_JOBS` - is the max number of the code processing requests which could be processed on the backend server + at the same time (default value = `20`). This value is used to check readiness of the backend server. If server + processes max number of the code processing requests then need to deploy a new instance. Review comment: I've updated this part. The main idea is `readiness` endpoint returns a response with status `200` if the instance could process more code processing requests. In other cases, this endpoint returns status `423`. This endpoint is used by the load balancer. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
