damccorm commented on code in PR #30430: URL: https://github.com/apache/beam/pull/30430#discussion_r1515134717
########## examples/java/webapis/src/main/java/org/apache/beam/examples/webapis/UsingHttpClientExample.java: ########## @@ -0,0 +1,68 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.beam.examples.webapis; Review Comment: Minor nit: in the future, it would be helpful to split PRs like this in 2; it makes it easier to review (and likely would drive down the time to review completion since it requires less time carved out) ########## website/www/site/content/en/documentation/io/built-in/webapis.md: ########## @@ -0,0 +1,441 @@ +--- +title: "Web Apis I/O connector" +--- +<!-- +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + +http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +--> + +[Built-in I/O Transforms](/documentation/io/built-in/) + +# Web APIs I/O connector + +{{< language-switcher java py go >}} + +The Beam SDKs include a built-in transform, called RequestResponseIO that can read from and write to Web APIs such as +REST or gRPC. + +## Before you start + +{{< paragraph class="language-java" >}} +To use RequestResponseIO, add the Maven artifact dependency to your `pom.xml` file. See +[Maven Central](https://central.sonatype.com/artifact/org.apache.beam/beam-sdks-java-io-rrio) for available versions. +{{< /paragraph >}} + +{{< highlight java >}} +<dependency> + <groupId>org.apache.beam</groupId> + <artifactId>beam-sdks-java-io-rrio</artifactId> + <version>{{< param release_latest >}}</version> +</dependency> +{{< /highlight >}} + +{{< paragraph class="language-py" >}} +To use RequestResponseIO, install the Beam SDK by running `pip install apache-beam` +{{< /paragraph >}} + +{{< paragraph class="language-go" wrap="span" >}} +At this time the Go SDK implementation of RequestResponseIO is not available. See tracker issue: +https://github.com/apache/beam/issues/30423. +{{< /paragraph >}} + +## Additional resources + +{{< paragraph class="language-java" wrap="span" >}} +* [RequestResponseIO source code](https://github.com/apache/beam/tree/master/sdks/java/io/rrio) +* [RequestResponseIO Javadoc](https://beam.apache.org/releases/javadoc/current/org/apache/beam/io/requestresponse/RequestResponseIO.html) +{{< /paragraph >}} + +{{< paragraph class="language-py" wrap="span" >}} +* [RequestResponseIO source code](https://github.com/apache/beam/blob/master/sdks/python/apache_beam/io/requestresponse.py) +* [RequestResponseIO PyDoc](https://beam.apache.org/releases/pydoc/current/apache_beam.io.requestresponse.html) +{{< /paragraph >}} + +{{< paragraph class="language-go" wrap="span" >}} +TODO: see https://github.com/apache/beam/issues/30423. +{{< /paragraph >}} + +## RequestResponseIO basics + +### Minimal code + +The minimal code needed to read from or write to Web APIs is: + +{{< paragraph class="language-java" wrap="span" >}} +1. [Caller](https://beam.apache.org/releases/javadoc/current/org/apache/beam/io/requestresponse/Caller.html) implementation. +2. Instantiate [RequestResponseIO](https://beam.apache.org/releases/javadoc/current/org/apache/beam/io/requestresponse/RequestResponseIO.html). +{{< /paragraph >}} + +{{< paragraph class="language-py" wrap="span" >}} +1. [Caller](https://beam.apache.org/releases/pydoc/current/apache_beam.io.requestresponse.html#apache_beam.io.requestresponse.Caller) implementation. +2. Instantiate [RequestResponseIO](https://beam.apache.org/releases/pydoc/current/apache_beam.io.requestresponse.html#apache_beam.io.requestresponse.RequestResponseIO). +{{< /paragraph >}} + +{{< paragraph class="language-go" wrap="span" >}} +TODO: see https://github.com/apache/beam/issues/30423. +{{< /paragraph >}} + +#### Implementing the Caller + +{{< paragraph class="language-java" >}} +[Caller](https://beam.apache.org/releases/javadoc/current/org/apache/beam/io/requestresponse/Caller.html) requires +only one method override. Review Comment: It would be helpful to describe the purpose of the `call` function here ########## buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy: ########## @@ -760,6 +760,7 @@ class BeamModulePlugin implements Plugin<Project> { google_cloud_platform_libraries_bom : "com.google.cloud:libraries-bom:26.32.0", google_cloud_spanner : "com.google.cloud:google-cloud-spanner", // google_cloud_platform_libraries_bom sets version google_cloud_spanner_test : "com.google.cloud:google-cloud-spanner:$google_cloud_spanner_version:tests", + google_cloud_vertexai : "com.google.cloud:google-cloud-vertexai", // google_cloud_platform_libraries_bom sets version Review Comment: ```suggestion google_cloud_vertexai : "com.google.cloud:google-cloud-vertexai", // google_cloud_platform_libraries_bom sets version ``` Spacing nit ########## website/www/site/content/en/documentation/io/built-in/webapis.md: ########## @@ -0,0 +1,441 @@ +--- +title: "Web Apis I/O connector" +--- +<!-- +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + +http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +--> + +[Built-in I/O Transforms](/documentation/io/built-in/) + +# Web APIs I/O connector + +{{< language-switcher java py go >}} + +The Beam SDKs include a built-in transform, called RequestResponseIO that can read from and write to Web APIs such as +REST or gRPC. Review Comment: I think it would be helpful to call out some of the features this includes here (throttling, caching, etc...) ########## website/www/site/content/en/documentation/io/built-in/webapis.md: ########## @@ -0,0 +1,441 @@ +--- +title: "Web Apis I/O connector" +--- +<!-- +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + +http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +--> + +[Built-in I/O Transforms](/documentation/io/built-in/) + +# Web APIs I/O connector + +{{< language-switcher java py go >}} + +The Beam SDKs include a built-in transform, called RequestResponseIO that can read from and write to Web APIs such as +REST or gRPC. + +## Before you start + +{{< paragraph class="language-java" >}} +To use RequestResponseIO, add the Maven artifact dependency to your `pom.xml` file. See +[Maven Central](https://central.sonatype.com/artifact/org.apache.beam/beam-sdks-java-io-rrio) for available versions. +{{< /paragraph >}} + +{{< highlight java >}} +<dependency> + <groupId>org.apache.beam</groupId> + <artifactId>beam-sdks-java-io-rrio</artifactId> + <version>{{< param release_latest >}}</version> +</dependency> +{{< /highlight >}} + +{{< paragraph class="language-py" >}} +To use RequestResponseIO, install the Beam SDK by running `pip install apache-beam` +{{< /paragraph >}} Review Comment: Can we add a note that currently there are no python examples on the website (tracked by https://github.com/apache/beam/issues/30422), but an E2E example does exist in the beam repo at https://github.com/apache/beam/blob/master/sdks/python/apache_beam/io/requestresponse_it_test.py -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
