[
https://issues.apache.org/jira/browse/BEAM-12941?focusedWorklogId=690278&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-690278
]
ASF GitHub Bot logged work on BEAM-12941:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 03/Dec/21 19:23
Start Date: 03/Dec/21 19:23
Worklog Time Spent: 10m
Work Description: emilymye commented on a change in pull request #16039:
URL: https://github.com/apache/beam/pull/16039#discussion_r762177355
##########
File path: playground/DEPLOY.md
##########
@@ -0,0 +1,95 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+## Manual App Install
+To install Playground applications into existed GCP account you'll need to
follow several steps, in this document just simple set of commands will be
provided that will guide you over install process.
+
+Before starting that process, you will need to create GCP Artifact Registry
instance to store there docker files that would be deployed. Guide to install
that registry could be found in
[that](https://github.com/apache/beam/blob/master/playground/terraform/README.md)
document.
+### Common steps
+First of all you will need to get service account key in json format, that can
be done by following
[that](https://cloud.google.com/iam/docs/creating-managing-service-account-keys)
document. Let's assume that you saved that key into file named key.json.
+Next step is to authentificate your account in GCP docker registry, this can
be done by running command:
+```bash
+$ cat /root/dt-key.json | docker login -u _json_key --password-stdin
REGISTRY_NAME
Review comment:
Can we recommend following
https://cloud.google.com/container-registry/docs/advanced-authentication (more
specifically
https://cloud.google.com/container-registry/docs/advanced-authentication#gcloud-helper)
##########
File path: playground/DEPLOY.md
##########
@@ -0,0 +1,95 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+## Manual App Install
+To install Playground applications into existed GCP account you'll need to
follow several steps, in this document just simple set of commands will be
provided that will guide you over install process.
+
+Before starting that process, you will need to create GCP Artifact Registry
instance to store there docker files that would be deployed. Guide to install
that registry could be found in
[that](https://github.com/apache/beam/blob/master/playground/terraform/README.md)
document.
+### Common steps
+First of all you will need to get service account key in json format, that can
be done by following
[that](https://cloud.google.com/iam/docs/creating-managing-service-account-keys)
document. Let's assume that you saved that key into file named key.json.
+Next step is to authentificate your account in GCP docker registry, this can
be done by running command:
+```bash
+$ cat /root/dt-key.json | docker login -u _json_key --password-stdin
REGISTRY_NAME
+```
+you will need to replace `REGISTRY_NAME` to actual registry address that can
be found in Google Cloud console of via gcloud tool.
+
+Next step is to specify path to json key to be used to deploy applications to
Google App Engine. This could be done by command
+```bash
+$ export GOOGLE_APPLICATION_CREDENTIALS=PATH_TO_KEY_JSON_FILE
+```
+replace `PATH_TO_KEY_JSON_FILE` with actual path to key.json file
+
+### Deploy backend application
+
+Now you can start deploying Playground backend application.
+First of all you will need to build and push image to docker registry, this
can be done by command
+```bash
+$ ./gradlew playground:backend:containers:java:dockerTagsPush
-Pdocker-repository-root='REGISTRY_NAME'
-Pbase-image='apache/beam_java8_sdk:BEAM_VERSION' -Pdocker-tag="BACKEND_TAG"
+```
+replace placeholders by
+* `REGISTRY_NAME` with actual registry address, same as was defined in first
command(docker login)
+* `BEAM_VERSION` with version of Apache Beam that you want to use
+* `BACKEND_TAG` tag that playground backend docker image will be set and
pushed to repo
+
+And final step for deploying backend is deploy via terraform, to run that you
will need to run commands
+
+```bash
+$ cd playground/terraform/applications/backend/
+$ terraform init
+$ terraform apply -auto-approve -var="project_id=GCP_PROJECT_ID"
-var="docker_registry_address=REGISTRY_NAME"
-var="docker_image_tag=BACKEND_TAG" -var="service_name=BACKEND_SERVICE_NAME"
+```
+
+replace placeholdes by
Review comment:
```suggestion
Replace the following variables:
```
##########
File path: playground/DEPLOY.md
##########
@@ -0,0 +1,95 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+## Manual App Install
+To install Playground applications into existed GCP account you'll need to
follow several steps, in this document just simple set of commands will be
provided that will guide you over install process.
+
+Before starting that process, you will need to create GCP Artifact Registry
instance to store there docker files that would be deployed. Guide to install
that registry could be found in
[that](https://github.com/apache/beam/blob/master/playground/terraform/README.md)
document.
+### Common steps
+First of all you will need to get service account key in json format, that can
be done by following
[that](https://cloud.google.com/iam/docs/creating-managing-service-account-keys)
document. Let's assume that you saved that key into file named key.json.
+Next step is to authentificate your account in GCP docker registry, this can
be done by running command:
+```bash
+$ cat /root/dt-key.json | docker login -u _json_key --password-stdin
REGISTRY_NAME
+```
+you will need to replace `REGISTRY_NAME` to actual registry address that can
be found in Google Cloud console of via gcloud tool.
+
+Next step is to specify path to json key to be used to deploy applications to
Google App Engine. This could be done by command
+```bash
+$ export GOOGLE_APPLICATION_CREDENTIALS=PATH_TO_KEY_JSON_FILE
+```
+replace `PATH_TO_KEY_JSON_FILE` with actual path to key.json file
+
+### Deploy backend application
+
+Now you can start deploying Playground backend application.
+First of all you will need to build and push image to docker registry, this
can be done by command
+```bash
+$ ./gradlew playground:backend:containers:java:dockerTagsPush
-Pdocker-repository-root='REGISTRY_NAME'
-Pbase-image='apache/beam_java8_sdk:BEAM_VERSION' -Pdocker-tag="BACKEND_TAG"
+```
+replace placeholders by
+* `REGISTRY_NAME` with actual registry address, same as was defined in first
command(docker login)
Review comment:
```suggestion
* `REGISTRY_NAME`: The AR repository root URL created earlier to host Docker
images
```
(I would also maybe change REGISTRY_NAME to REPOSITORY_ROOT or similar
##########
File path: playground/DEPLOY.md
##########
@@ -0,0 +1,95 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+## Manual App Install
+To install Playground applications into existed GCP account you'll need to
follow several steps, in this document just simple set of commands will be
provided that will guide you over install process.
+
+Before starting that process, you will need to create GCP Artifact Registry
instance to store there docker files that would be deployed. Guide to install
that registry could be found in
[that](https://github.com/apache/beam/blob/master/playground/terraform/README.md)
document.
+### Common steps
+First of all you will need to get service account key in json format, that can
be done by following
[that](https://cloud.google.com/iam/docs/creating-managing-service-account-keys)
document. Let's assume that you saved that key into file named key.json.
+Next step is to authentificate your account in GCP docker registry, this can
be done by running command:
+```bash
+$ cat /root/dt-key.json | docker login -u _json_key --password-stdin
REGISTRY_NAME
+```
+you will need to replace `REGISTRY_NAME` to actual registry address that can
be found in Google Cloud console of via gcloud tool.
+
+Next step is to specify path to json key to be used to deploy applications to
Google App Engine. This could be done by command
+```bash
+$ export GOOGLE_APPLICATION_CREDENTIALS=PATH_TO_KEY_JSON_FILE
+```
+replace `PATH_TO_KEY_JSON_FILE` with actual path to key.json file
+
+### Deploy backend application
+
+Now you can start deploying Playground backend application.
+First of all you will need to build and push image to docker registry, this
can be done by command
+```bash
+$ ./gradlew playground:backend:containers:java:dockerTagsPush
-Pdocker-repository-root='REGISTRY_NAME'
-Pbase-image='apache/beam_java8_sdk:BEAM_VERSION' -Pdocker-tag="BACKEND_TAG"
+```
+replace placeholders by
+* `REGISTRY_NAME` with actual registry address, same as was defined in first
command(docker login)
+* `BEAM_VERSION` with version of Apache Beam that you want to use
+* `BACKEND_TAG` tag that playground backend docker image will be set and
pushed to repo
+
+And final step for deploying backend is deploy via terraform, to run that you
will need to run commands
+
+```bash
+$ cd playground/terraform/applications/backend/
+$ terraform init
+$ terraform apply -auto-approve -var="project_id=GCP_PROJECT_ID"
-var="docker_registry_address=REGISTRY_NAME"
-var="docker_image_tag=BACKEND_TAG" -var="service_name=BACKEND_SERVICE_NAME"
Review comment:
I think these variables should be one per line
##########
File path: playground/DEPLOY.md
##########
@@ -0,0 +1,95 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+## Manual App Install
+To install Playground applications into existed GCP account you'll need to
follow several steps, in this document just simple set of commands will be
provided that will guide you over install process.
Review comment:
I think maybe we can improve this guide by changing the guide to the
following steps (which I think are the requirements for manually installing the
Playground app)
1) Building and pushing the backend docker image file to Google Artifact
Registry (AR)
2) Building and pushing frontend config files and docker image file to AR
3) Deploy services via terraform.
Where each section describes the commands and variables and links to GCP
guides if needed.
##########
File path: playground/DEPLOY.md
##########
@@ -0,0 +1,95 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+## Manual App Install
+To install Playground applications into existed GCP account you'll need to
follow several steps, in this document just simple set of commands will be
provided that will guide you over install process.
+
+Before starting that process, you will need to create GCP Artifact Registry
instance to store there docker files that would be deployed. Guide to install
that registry could be found in
[that](https://github.com/apache/beam/blob/master/playground/terraform/README.md)
document.
+### Common steps
+First of all you will need to get service account key in json format, that can
be done by following
[that](https://cloud.google.com/iam/docs/creating-managing-service-account-keys)
document. Let's assume that you saved that key into file named key.json.
+Next step is to authentificate your account in GCP docker registry, this can
be done by running command:
+```bash
+$ cat /root/dt-key.json | docker login -u _json_key --password-stdin
REGISTRY_NAME
+```
+you will need to replace `REGISTRY_NAME` to actual registry address that can
be found in Google Cloud console of via gcloud tool.
Review comment:
Having the doc link
(https://cloud.google.com/artifact-registry/docs/manage-repos#view) or example
here would be helpful (either the gcloud command like `gcloud artifacts
repositories list` or something like
`us-central1-docker.pkg.dev/my-project/playground-repository`).
I think it would also be better to have this link to the actual guides from
GCP documentation to create AR repositories like
https://cloud.google.com/artifact-registry/docs/manage-repos
##########
File path: playground/DEPLOY.md
##########
@@ -0,0 +1,95 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+## Manual App Install
+To install Playground applications into existed GCP account you'll need to
follow several steps, in this document just simple set of commands will be
provided that will guide you over install process.
+
+Before starting that process, you will need to create GCP Artifact Registry
instance to store there docker files that would be deployed. Guide to install
that registry could be found in
[that](https://github.com/apache/beam/blob/master/playground/terraform/README.md)
document.
+### Common steps
+First of all you will need to get service account key in json format, that can
be done by following
[that](https://cloud.google.com/iam/docs/creating-managing-service-account-keys)
document. Let's assume that you saved that key into file named key.json.
+Next step is to authentificate your account in GCP docker registry, this can
be done by running command:
+```bash
+$ cat /root/dt-key.json | docker login -u _json_key --password-stdin
REGISTRY_NAME
+```
+you will need to replace `REGISTRY_NAME` to actual registry address that can
be found in Google Cloud console of via gcloud tool.
+
+Next step is to specify path to json key to be used to deploy applications to
Google App Engine. This could be done by command
+```bash
+$ export GOOGLE_APPLICATION_CREDENTIALS=PATH_TO_KEY_JSON_FILE
+```
+replace `PATH_TO_KEY_JSON_FILE` with actual path to key.json file
+
+### Deploy backend application
+
+Now you can start deploying Playground backend application.
+First of all you will need to build and push image to docker registry, this
can be done by command
+```bash
+$ ./gradlew playground:backend:containers:java:dockerTagsPush
-Pdocker-repository-root='REGISTRY_NAME'
-Pbase-image='apache/beam_java8_sdk:BEAM_VERSION' -Pdocker-tag="BACKEND_TAG"
+```
+replace placeholders by
Review comment:
```suggestion
Replace the following variables:
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 690278)
Time Spent: 2h 50m (was: 2h 40m)
> [Playground] Application CI/CD scripts
> --------------------------------------
>
> Key: BEAM-12941
> URL: https://issues.apache.org/jira/browse/BEAM-12941
> Project: Beam
> Issue Type: New Feature
> Components: beam-playground
> Reporter: Alex Kosolapov
> Assignee: Sergey Kalinin
> Priority: P3
> Labels: beam-playground-devops, beam-playground-sprint-2,
> beam-playground-sprint-3
> Time Spent: 2h 50m
> Remaining Estimate: 0h
>
> As a Playground maintainer, I want to have CI/CD pipelines in Apache Beam
> GitHub to automatically run tests in the Beam Playground project and deploy
> Beam Playground components to GCP infrastructure.
> Acceptance criteria:
> # CI/CD pipelines have been implemented using GitHub Actions and Jenkins
> # CI pipelines check style using linter for backend
> # CI pipelines check style using linter for frontend
> # CI pipelines run tests for the Beam Playground project
> # CI/CD pipelines built artifacts and docker images
> # CD pipelines deployed Beam Playground components to GCP
> # CI/CD pipelines integrated with Apache Beam CI/CD
> # Draft Deployment instruction/FAQ
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
