damondouglas opened a new issue, #24206: URL: https://github.com/apache/beam/issues/24206
### What needs to happen? ### What work does this Task describe? This task adds instance method and two static methods to [PubsubClient](https://beam.apache.org/releases/javadoc/current/org/apache/beam/sdk/io/gcp/pubsub/PubsubClient.html). ```java // org.apache.beam.sdk.io.gcp.pubsub.PubsubClientPubsubClient import org.apache.beam.sdk.schemas.Schema; public Schema getSchema(SchemaPath schemaPath) throws IOException; ``` To support the aforementioned, this task additionally adds: 1. SchemaPath ```java public static class SchemaPath { // Supports the projects/<project>/schemas/<schema> resource path } ``` 2. fromSchema methods ```java // org.apache.beam.sdk.io.gcp.pubsub.PubsubClientPubsubClient import org.apache.beam.sdk.schemas.Schema; static Schema fromPubsubSchema(com.google.api.services.pubsub.model.Schema pubsubSchema) { /* Converts Pub/Sub model Schema to Beam Schema; for use by PubsubJsonClient. */ } static Schema fromPubsubSchema(com.google.pubsub.v1.Schema pubsubSchema) { /* Converts Pub/Sub model Schema to Beam Schema; for use by PubsubGrpcClient. */ } ``` 3. Override getSchema in PubsubClient subclasses ```java // org.apache.beam.sdk.io.gcp.pubsub.PubsubGrpcClient import org.apache.beam.sdk.schemas.Schema; @Override public Schema getSchema(SchemaPath schemaPath) throws IOException { ... } ``` ```java // org.apache.beam.sdk.io.gcp.pubsub.PubsubJsonClient import org.apache.beam.sdk.schemas.Schema; @Override public Schema getSchema(SchemaPath schemaPath) throws IOException { ... } ``` ```java // org.apache.beam.sdk.io.gcp.pubsub.PubsubTestClient import org.apache.beam.sdk.schemas.Schema; @Override public Schema getSchema(SchemaPath schemaPath) throws IOException { ... } ``` 4. Add schema to PubsubTestClient.State ```java import org.apache.beam.sdk.schemas.Schema; private static class State { /** Expected Pub/Sub mapped Beam Schema. */ @Nullable Schema schema; } ``` ### What value may result from this Task's output? Querying from and converting the Pub/Sub Schema [1] to a Beam Schema[2] would allow us to validate that both schemas match to prevent potential errors. This supports the design goals of Pub/Sub schemas to facilitate a contract between publisher and subscriber and facilitate a single source of truth for inter-team production and consumption. ### Ready and Done Measures #### Ready No blockers #### Done - [ ] Tests detect errors in resource path generation for a Google Cloud resource path - [ ] Tests detect errors in converting from Beam Schema to Pub/Sub model Schema - [ ] Tests detect errors in converting from Beam Schema to Pub/Sub proto Schema - [ ] Tests detect errors in querying source Pub/Sub schema - [ ] Integration tests detect errors in querying source Pub/Sub schema from provisioned resources ### Issue Priority Priority: 3 ### Issue Component Component: io-java-gcp -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
