gaborgsomogyi commented on a change in pull request #18746: URL: https://github.com/apache/flink/pull/18746#discussion_r838422656
########## File path: docs/content/docs/security/overview.md ########## @@ -0,0 +1,87 @@ +--- +title: "Overview" +weight: 1 +type: docs +--- +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> + +# Security Overview + +Frameworks that process data are sensitive components; you must use authentication and encryption to +secure your data and data sources. Apache Flink supports authentication with [Kerberos](https://web.mit.edu/kerberos/) +and can be configured to encrypt all network communication with [SSL/TLS](https://www.ssl.com/faqs/faq-what-is-ssl/). + +{{< hint info >}} +SSL and TLS are both encryption protocols for the transport layer, used for encrypting data between +the client and server. The TLS protocol is the successor to SSL. Flink enforces TLS by default, but +you can also switch back to SSL. +{{< /hint >}} + +## What needs to be secured in Flink? + +When we talk about security for Flink, we generally make a distinction between securing the internal +communication within the Flink cluster (i.e. between the TaskManagers, between the TaskManagers and +the JobManager) and securing the external communication between the cluster and the outside world. + +Internally, [Netty](https://netty.io) is used for the TCP connections used for data exchange among +the TaskManagers, and [Akka](https://akka.io) is used for RPC between the JobManager and the TaskManagers. + +Externally, HTTP is used for pretty much everything, except that some external services used as sources +or sinks may use some other network protocol. + +## What is supported by Flink? + +Security enhancement features by the Flink community make it easy to access secured data, protect +associated credentials, and increase overall security in a Flink cluster. The following security +measures are currently supported: + +- Authentication of connections between Flink processes +- Encryption of data transferred between Flink processes using SSL (Note that there is a performance + degradation when SSL is enabled, the magnitude of which depends on the CPU type, the JVM implementation, + and the key size.) +- Authorization of read / write operations by clients +- Authorization is pluggable and integration with external authorization services is supported + +It is worth noting that security needs to be manually configured because the overall philosophy in +Flink is to have defaults that work out-of-the-box, without any configuration. Thus, non-secured +clusters are supported, as well as a mix of authenticated, unauthenticated, encrypted and unencrypted +clients. + +## Where are its flaws? How should they be addressed? + +Web UI unsecure Review comment: These 2 statement lines are a little bit oddly formed. If I would a be a user not sure I would get what does it actually mean. ########## File path: docs/content/docs/security/overview.md ########## @@ -0,0 +1,87 @@ +--- +title: "Overview" +weight: 1 +type: docs +--- +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> + +# Security Overview + +Frameworks that process data are sensitive components; you must use authentication and encryption to +secure your data and data sources. Apache Flink supports authentication with [Kerberos](https://web.mit.edu/kerberos/) +and can be configured to encrypt all network communication with [SSL/TLS](https://www.ssl.com/faqs/faq-what-is-ssl/). + +{{< hint info >}} +SSL and TLS are both encryption protocols for the transport layer, used for encrypting data between +the client and server. The TLS protocol is the successor to SSL. Flink enforces TLS by default, but +you can also switch back to SSL. +{{< /hint >}} + +## What needs to be secured in Flink? + +When we talk about security for Flink, we generally make a distinction between securing the internal +communication within the Flink cluster (i.e. between the TaskManagers, between the TaskManagers and +the JobManager) and securing the external communication between the cluster and the outside world. + +Internally, [Netty](https://netty.io) is used for the TCP connections used for data exchange among +the TaskManagers, and [Akka](https://akka.io) is used for RPC between the JobManager and the TaskManagers. + +Externally, HTTP is used for pretty much everything, except that some external services used as sources +or sinks may use some other network protocol. + +## What is supported by Flink? + +Security enhancement features by the Flink community make it easy to access secured data, protect +associated credentials, and increase overall security in a Flink cluster. The following security +measures are currently supported: + +- Authentication of connections between Flink processes +- Encryption of data transferred between Flink processes using SSL (Note that there is a performance + degradation when SSL is enabled, the magnitude of which depends on the CPU type, the JVM implementation, + and the key size.) +- Authorization of read / write operations by clients +- Authorization is pluggable and integration with external authorization services is supported + +It is worth noting that security needs to be manually configured because the overall philosophy in +Flink is to have defaults that work out-of-the-box, without any configuration. Thus, non-secured +clusters are supported, as well as a mix of authenticated, unauthenticated, encrypted and unencrypted +clients. + +## Where are its flaws? How should they be addressed? + +Web UI unsecure +not secure by default + +## What needs to be solved outside of Flink? + +connectors Review comment: Nit: Shouldn't this be `Connectors` to follow the same pattern which we have in other chapters (capital first letter)? ########## File path: docs/content/docs/security/running-cluster.md ########## @@ -0,0 +1,193 @@ +--- +title: Incorporating Security Features in a Running Cluster +weight: 4 +type: docs +aliases: +--- +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> + +# Incorporating Security Features in a Running Cluster + +This guide describes how Flink security works in the context of various [deployment modes]({{< ref "docs/deployment/overview" >}}), +[file systems]({{< ref "docs/deployment/filesystems/overview" >}}), [connectors]({{< ref "docs/connectors" >}}), +and [state backends]({{< ref "docs/ops/state/state_backends" >}}). + +## Running a Secure Flink Cluster in Standalone Mode + +Standalone mode is the most barebone way of deploying a Flink cluster where Flink processes are launched +as processes on the operating system. You can learn more [here]({{< ref "docs/deployment/resource-providers/standalone/overview" >}}). + +Incorporating security in this mode involves the following steps: +1. Add [security-related configuration options]({{< ref "docs/deployment/config" >}}#auth-with-external-systems)) + to the Flink configuration file (`flink-conf.yaml`) on all cluster nodes. +2. Ensure that the Kerberos keytab file exists at the path indicated by the `security.kerberos.login.keytab` + setting on all cluster nodes. +3. Deploy the Flink cluster as normal. + +{{< hint info >}} +`HADOOP_TOKEN_FILE_LOCATION` is an important environment variable that specifies the location of your +credential files/tokens. The service instances uses this location to find the file to load the credentials +and run the tasks. Launched containers must load the delegation tokens from this location, and use them +(including renewals) until they can no longer be renewed. +{{< /hint >}} + +You need to have an appropriate Java keystore and truststore accessible from each node in the Flink +cluster. For standalone setups, this means copying the files to each node, or adding them to a shared +mounted directory. For externally facing REST endpoints, the common name or subject alternative names +in the certificate should match the node's hostname and IP address. + +Keystores and truststores can be generated using the [keytool utility](https://docs.oracle.com/javase/8/docs/technotes/tools/unix/keytool.html). + +## Running a Secure Flink Cluster in Kubernetes + +Flink’s native Kubernetes integration allows you to directly deploy Flink on a running Kubernetes cluster. +You can learn more [here]({{< ref "docs/deployment/resource-providers/native_kubernetes" >}}). + +Incorporating security in this mode involves the following steps: +1. Add [security-related configuration options]({{< ref "docs/deployment/config" >}}#auth-with-external-systems)) + to the Flink configuration file (`flink-conf.yaml`) on the client node. +2. Ensure that the Kerberos keytab file exists at the path as indicated by the `security.kerberos.login.keytab` + on the client node. +3. Deploy the Flink cluster as normal. + +### Setting up Kerberos + +When deploying Flink on Kubernetes, the Kerberos keytab file is automatically copied from the client +node to the Flink containers. + +To enable Kerberos authentication, the Kerberos configuration file is also required. This file should +be present in the cluster nodes / classpath. If you want Flink to upload a new file, you need to configure +the `security.kerberos.krb5-conf.path` setting to indicate the path of the Kerberos configuration file +and Flink will copy this file to its containers/pods. + +{{< hint info >}} +`HADOOP_TOKEN_FILE_LOCATION` is an important environment variable that specifies the location of your +credential files/tokens. The service instances uses this location to find the file to load the credentials +and run the tasks. Launched containers must load the delegation tokens from this location, and use them +(including renewals) until they can no longer be renewed. +{{< /hint >}} + +### Setting up keystores and truststores + +You need to have an appropriate Java keystore and truststore accessible from each node in the Flink +cluster. For container-based setups, this means adding the keystore and truststore files to the container +images. For externally facing REST endpoints, the common name or subject alternative names in the +certificate should match the node's hostname and IP address. + +## Running a Secure Flink Cluster in YARN + +Flink services are submitted to YARN’s ResourceManager, which spawns containers on machines managed +by YARN NodeManagers. Flink deploys its JobManager and TaskManager instances into such containers. +You can learn more [here]({{< ref "docs/deployment/resource-providers/yarn" >}}). + +Incorporating security in this mode involves the following steps: +1. Add [security-related configuration options]({{< ref "docs/deployment/config" >}}#auth-with-external-systems)) + to the Flink configuration file (`flink-conf.yaml`) on the client node. +2. Ensure that the Kerberos keytab file exists at the path as indicated by the `security.kerberos.login.keytab` + on the client node. +3. Deploy the Flink cluster as normal. + +### Setting up Kerberos + +When deploying Flink on YARN, the Kerberos keytab is automatically copied from the client node to the +Flink containers. + +To enable Kerberos authentication, the Kerberos configuration file is also required. This file can be Review comment: This is not yet answered what does that mean: `This file can be either fetched from the cluster environment` ########## File path: docs/content/docs/security/ssl.md ########## @@ -0,0 +1,367 @@ +--- +title: "Encryption and Authentication using SSL/TLS" +weight: 3 +type: docs +aliases: + - /deployment/security/ssl.html + - /ops/security-ssl.html +--- +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> + +# Encryption and Authentication using SSL/TLS for Cluster Components + +Flink supports mutual authentication (when two parties authenticate each other at the same time) and +encryption of network communication with SSL for internal and external communication. + +**By default, SSL/TLS authentication and encryption is not enabled** (to have defaults work out-of-the-box). + +This guide will explain internal vs external connectivity, and provide instructions on how to enable +SSL/TLS authentication and encryption for network communication with and between Flink processes. We +will go through steps such as generating certificates, setting up TrustStores and KeyStores, and +configuring cipher suites. + +For how-tos and tips for different deployment environments (i.e. standalone clusters, Kubernetes, YARN), +check out the section on [Incorporating Security Features in a Running Cluster]({{< ref "docs/security/running-cluster" >}}). + +## Internal and External Communication + +There are two types of network connections to authenticate and encrypt: internal and external. + +{{< img src="/fig/ssl_internal_external.svg" alt="Internal and External Connectivity" width=75% >}} + +For more flexibility, security for internal and external connectivity can be enabled and configured +separately. + +### Internal Connectivity + +Flink internal communication refers to all connections made between Flink processes. These include: + +- Control messages: RPC between JobManager / TaskManager / Dispatcher / ResourceManager +- Transfers on the data plane: connections between TaskManagers to exchange data during shuffles, + broadcasts, redistribution, etc +- Blob service communication: distribution of libraries and other artifacts + +All internal connections are SSL authenticated and encrypted. The connections use **mutual authentication**, +meaning both server and client side of each connection need to present the certificate to each other. +The certificate acts as a shared secret and can be embedded into container images or attached to your +deployment setup. These connections run Flink custom protocols. Users never connect directly to internal +connectivity endpoints. + +### External Connectivity + +Flink external communication refers to all connections made from the outside to Flink processes. +This includes: +- communication with the Dispatcher to submit Flink jobs (session clusters) +- communication of the Flink CLI with the JobManager to inspect and modify a running Flink job/application + +These connections are exposed via REST/HTTP endpoints (and are used by Flink's web UI). Some +application-specific external services such as sources and sinks may use some other network protocol. + +The server will, by default, accept connections from any client, meaning that the REST endpoint does +not authenticate the client. These REST endpoints, however, can be configured to require SSL encryption +and mutual authentication. + +However, the recommended approach is setting up and configuring a dedicated proxy service (a "sidecar +proxy") that controls access to the REST endpoint. This involves binding the REST endpoint to the +loopback interface (or the pod-local interface in Kubernetes) and starting a REST proxy that authenticates +and forwards the requests to Flink. Examples for proxies that Flink users have deployed are [Envoy Proxy](https://www.envoyproxy.io/) +or [NGINX with MOD_AUTH](http://nginx.org/en/docs/http/ngx_http_auth_request_module.html). + +The rationale behind delegating authentication to a proxy is that such proxies offer a wide variety +of authentication options and thus better integration into existing infrastructures. + +## Queryable State + +Connections to the [queryable state]({{< ref "docs/dev/datastream/fault-tolerance/queryable_state" >}}) +endpoints is currently not authenticated or encrypted. + +## SSL Setups + +{{< img src="/fig/ssl_mutual_auth.svg" alt="SSL Mutual Authentication" width=75% >}} Review comment: This is still not working. Not sure what is the issue but having such issue is something we need to fix I think. <img width="427" alt="Screenshot 2022-02-16 at 12 51 40" src="https://user-images.githubusercontent.com/18561820/154259238-2e7e4f58-5a1a-4083-9e7e-a3590ce8a088.png";> ########## File path: docs/content/docs/security/ssl.md ########## @@ -0,0 +1,367 @@ +--- +title: "Encryption and Authentication using SSL/TLS" +weight: 3 +type: docs +aliases: + - /deployment/security/ssl.html + - /ops/security-ssl.html +--- +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> + +# Encryption and Authentication using SSL/TLS for Cluster Components + +Flink supports mutual authentication (when two parties authenticate each other at the same time) and +encryption of network communication with SSL for internal and external communication. + +**By default, SSL/TLS authentication and encryption is not enabled** (to have defaults work out-of-the-box). + +This guide will explain internal vs external connectivity, and provide instructions on how to enable +SSL/TLS authentication and encryption for network communication with and between Flink processes. We +will go through steps such as generating certificates, setting up TrustStores and KeyStores, and +configuring cipher suites. + +For how-tos and tips for different deployment environments (i.e. standalone clusters, Kubernetes, YARN), +check out the section on [Incorporating Security Features in a Running Cluster]({{< ref "docs/security/running-cluster" >}}). + +## Internal and External Communication + +There are two types of network connections to authenticate and encrypt: internal and external. + +{{< img src="/fig/ssl_internal_external.svg" alt="Internal and External Connectivity" width=75% >}} + +For more flexibility, security for internal and external connectivity can be enabled and configured +separately. + +### Internal Connectivity + +Flink internal communication refers to all connections made between Flink processes. These include: + +- Control messages: RPC between JobManager / TaskManager / Dispatcher / ResourceManager +- Transfers on the data plane: connections between TaskManagers to exchange data during shuffles, + broadcasts, redistribution, etc +- Blob service communication: distribution of libraries and other artifacts + +All internal connections are SSL authenticated and encrypted. The connections use **mutual authentication**, +meaning both server and client side of each connection need to present the certificate to each other. +The certificate acts as a shared secret and can be embedded into container images or attached to your +deployment setup. These connections run Flink custom protocols. Users never connect directly to internal +connectivity endpoints. + +### External Connectivity + +Flink external communication refers to all connections made from the outside to Flink processes. +This includes: +- communication with the Dispatcher to submit Flink jobs (session clusters) +- communication of the Flink CLI with the JobManager to inspect and modify a running Flink job/application + +These connections are exposed via REST/HTTP endpoints (and are used by Flink's web UI). Some +application-specific external services such as sources and sinks may use some other network protocol. + +The server will, by default, accept connections from any client, meaning that the REST endpoint does +not authenticate the client. These REST endpoints, however, can be configured to require SSL encryption +and mutual authentication. + +However, the recommended approach is setting up and configuring a dedicated proxy service (a "sidecar +proxy") that controls access to the REST endpoint. This involves binding the REST endpoint to the +loopback interface (or the pod-local interface in Kubernetes) and starting a REST proxy that authenticates +and forwards the requests to Flink. Examples for proxies that Flink users have deployed are [Envoy Proxy](https://www.envoyproxy.io/) +or [NGINX with MOD_AUTH](http://nginx.org/en/docs/http/ngx_http_auth_request_module.html). + +The rationale behind delegating authentication to a proxy is that such proxies offer a wide variety +of authentication options and thus better integration into existing infrastructures. + +## Queryable State + +Connections to the [queryable state]({{< ref "docs/dev/datastream/fault-tolerance/queryable_state" >}}) +endpoints is currently not authenticated or encrypted. + +## SSL Setups + +{{< img src="/fig/ssl_mutual_auth.svg" alt="SSL Mutual Authentication" width=75% >}} + +Each participant has a keystore and a truststore, which are files. + +A keystore contains a certificate (which contains a public key) and a private key. A truststore +contains trusted certificates and certificate chains/authorities. + +Establishing encrypted, authenticated communication is a multi-step process, shown in the figure. +Certificates are exchanged and validated against the truststore, after which the two parties can +safely communicate. + +### Typical SSL Setup in Flink + +For mutually authenticated internal connections, note that: + +- a keystore and a truststore can contain the same dedicated certificate +- the same file can be used for both keystore and truststore +- wildcard hostnames or addresses can be used + +For internal communication between servers in a Flink cluster, a secure setup can be established with +a single, self-signed certificate that all parties use as both their keystore and truststore. You can +also use this approach for external communication when establishing mutual authentication for communication +between clients and the Flink Master. + +### Configuring Keystores and Truststores + +The SSL configuration requires configuring a keystore and a truststore such that the truststore trusts +the keystore's certificate. + +You can use the [keytool utility](https://docs.oracle.com/javase/8/docs/technotes/tools/unix/keytool.html) +to generate keys, certificates, keystores, and truststores: + +```bash + keytool -genkeypair -alias flink.internal -keystore internal.keystore \ Review comment: Is there a specific reason why this keytool command is broken with `\` and all the others not? ########## File path: docs/content/docs/security/running-cluster.md ########## @@ -0,0 +1,193 @@ +--- +title: Incorporating Security Features in a Running Cluster +weight: 4 +type: docs +aliases: +--- +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> + +# Incorporating Security Features in a Running Cluster + +This guide describes how Flink security works in the context of various [deployment modes]({{< ref "docs/deployment/overview" >}}), +[file systems]({{< ref "docs/deployment/filesystems/overview" >}}), [connectors]({{< ref "docs/connectors" >}}), +and [state backends]({{< ref "docs/ops/state/state_backends" >}}). + +## Running a Secure Flink Cluster in Standalone Mode + +Standalone mode is the most barebone way of deploying a Flink cluster where Flink processes are launched +as processes on the operating system. You can learn more [here]({{< ref "docs/deployment/resource-providers/standalone/overview" >}}). + +Incorporating security in this mode involves the following steps: +1. Add [security-related configuration options]({{< ref "docs/deployment/config" >}}#auth-with-external-systems)) + to the Flink configuration file (`flink-conf.yaml`) on all cluster nodes. +2. Ensure that the Kerberos keytab file exists at the path indicated by the `security.kerberos.login.keytab` + setting on all cluster nodes. +3. Deploy the Flink cluster as normal. + +{{< hint info >}} +`HADOOP_TOKEN_FILE_LOCATION` is an important environment variable that specifies the location of your +credential files/tokens. The service instances uses this location to find the file to load the credentials +and run the tasks. Launched containers must load the delegation tokens from this location, and use them +(including renewals) until they can no longer be renewed. +{{< /hint >}} + +You need to have an appropriate Java keystore and truststore accessible from each node in the Flink +cluster. For standalone setups, this means copying the files to each node, or adding them to a shared +mounted directory. For externally facing REST endpoints, the common name or subject alternative names +in the certificate should match the node's hostname and IP address. + +Keystores and truststores can be generated using the [keytool utility](https://docs.oracle.com/javase/8/docs/technotes/tools/unix/keytool.html). + +## Running a Secure Flink Cluster in Kubernetes + +Flink’s native Kubernetes integration allows you to directly deploy Flink on a running Kubernetes cluster. +You can learn more [here]({{< ref "docs/deployment/resource-providers/native_kubernetes" >}}). + +Incorporating security in this mode involves the following steps: +1. Add [security-related configuration options]({{< ref "docs/deployment/config" >}}#auth-with-external-systems)) + to the Flink configuration file (`flink-conf.yaml`) on the client node. +2. Ensure that the Kerberos keytab file exists at the path as indicated by the `security.kerberos.login.keytab` + on the client node. +3. Deploy the Flink cluster as normal. + +### Setting up Kerberos + +When deploying Flink on Kubernetes, the Kerberos keytab file is automatically copied from the client +node to the Flink containers. + +To enable Kerberos authentication, the Kerberos configuration file is also required. This file should +be present in the cluster nodes / classpath. If you want Flink to upload a new file, you need to configure +the `security.kerberos.krb5-conf.path` setting to indicate the path of the Kerberos configuration file +and Flink will copy this file to its containers/pods. + +{{< hint info >}} +`HADOOP_TOKEN_FILE_LOCATION` is an important environment variable that specifies the location of your +credential files/tokens. The service instances uses this location to find the file to load the credentials +and run the tasks. Launched containers must load the delegation tokens from this location, and use them Review comment: `Launched containers can load the delegation tokens from this location` because Flink can fetch it in the initial phase too. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
![https://user-images.githubusercontent.com/18561820/154259238-2e7e4f58-5a1a-4083-9e7e-a3590ce8a088.png"](https://user-images.githubusercontent.com/18561820/154259238-2e7e4f58-5a1a-4083-9e7e-a3590ce8a088.png"){kind=link}