[ https://issues.apache.org/jira/browse/BEAM-7336?focusedWorklogId=393185&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-393185 ]
ASF GitHub Bot logged work on BEAM-7336: ---------------------------------------- Author: ASF GitHub Bot Created on: 26/Feb/20 05:24 Start Date: 26/Feb/20 05:24 Worklog Time Spent: 10m Work Description: iemejia commented on pull request #10966: [WIP][BEAM-7336] Add schema inferring for KafkaIO when reading Avro values URL: https://github.com/apache/beam/pull/10966#discussion_r384277134 ########## File path: sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/KafkaIO.java ########## @@ -845,11 +853,23 @@ public void setValueDeserializer(String valueDeserializer) { return toBuilder().setConsumerConfig(config).build(); } - /** Returns a {@link PTransform} for PCollection of {@link KV}, dropping Kafka metatdata. */ + /** + * Returns a {@link PTransform} for {@link PCollection} of {@link KV}, dropping Kafka metadata. + */ public PTransform<PBegin, PCollection<KV<K, V>>> withoutMetadata() { return new TypedWithoutMetadata<>(this); } + /** + * Returns a {@link PTransform} for {@link PCollection} of {@link GenericRecord}, dropping Kafka + * keys and metadata. It has to be used only with {@link + * ConfluentSchemaRegistryDeserializerProvider}. See {@link KafkaIO} for more information on + * usage and configuration of reader. + */ + public PTransform<PBegin, PCollection<GenericRecord>> withAvroSchemaValues() { Review comment: We do not need an special PTransform for this. Notice that both SqlTransform and most Schema-based PTransforms do require PCollection<T> but with the specific condition that this PCollection should be encoded using a `SchemaCoder`. So we can just use the existing methods that produce `PCollection<K>` and `PCollection<V>`. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking ------------------- Worklog Id: (was: 393185) Time Spent: 40m (was: 0.5h) > KafkaIO should support inferring schemas when reading Avro > ---------------------------------------------------------- > > Key: BEAM-7336 > URL: https://issues.apache.org/jira/browse/BEAM-7336 > Project: Beam > Issue Type: Sub-task > Components: io-java-kafka > Reporter: Reuven Lax > Assignee: Alexey Romanenko > Priority: Major > Time Spent: 40m > Remaining Estimate: 0h > > PubSubIO already supports this. > It would also be nice to be able to look up Avro schemas in the Kafka schema > registry. -- This message was sent by Atlassian Jira (v8.3.4#803005)