[
https://issues.apache.org/jira/browse/BEAM-7336?focusedWorklogId=393230&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-393230
]
ASF GitHub Bot logged work on BEAM-7336:
----------------------------------------
Author: ASF GitHub Bot
Created on: 26/Feb/20 08:09
Start Date: 26/Feb/20 08:09
Worklog Time Spent: 10m
Work Description: iemejia commented on issue #10966: [WIP][BEAM-7336] Add
schema inferring for KafkaIO when reading Avro values
URL: https://github.com/apache/beam/pull/10966#issuecomment-591293909
Some extra things:
- We should probably annotate the two infer methods in KafkaIO as
`@Experimental(Kind.SCHEMAS)`
- The `getCoder(CoderRegistry)` method should now receive a `SchemaRegistry`
too, you can get that one from the input like this: `SchemaRegistry
schemaRegistry = input.getPipeline().getSchemaRegistry();`
- We can simplify the test, because the only thing the code should care
about is that if we infer a Beam `SchemaCoder` and it is available then the
returned PCollection has a Coder that is an instance of `SchemaCoder`. The fact
that you can do SQL queries or Schema-based PTransforms from a `PCollection`
encoded with a `SchemaCoder` is already given and it is a nice guarantee of
Reuven's work, so we do not need to double test that here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 393230)
Time Spent: 1h 20m (was: 1h 10m)
> KafkaIO should support inferring schemas when reading Avro
> ----------------------------------------------------------
>
> Key: BEAM-7336
> URL: https://issues.apache.org/jira/browse/BEAM-7336
> Project: Beam
> Issue Type: Sub-task
> Components: io-java-kafka
> Reporter: Reuven Lax
> Assignee: Alexey Romanenko
> Priority: Major
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> PubSubIO already supports this.
> It would also be nice to be able to look up Avro schemas in the Kafka schema
> registry.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
