[
https://issues.apache.org/jira/browse/BEAM-11338?focusedWorklogId=522798&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-522798
]
ASF GitHub Bot logged work on BEAM-11338:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 10/Dec/20 17:06
Start Date: 10/Dec/20 17:06
Worklog Time Spent: 10m
Work Description: ccciudatu commented on pull request #13428:
URL: https://github.com/apache/beam/pull/13428#issuecomment-742657119
@TheNeuralBit I'm trying to use KafkaIO to process thrift input and output
with some beam "canonical" processing on Rows in between.
In my opinion, the first step towards simplifying the use of the various
schema providers available would be to allow registering the top of the class
hierarchy with a given provider, rather than every individual subtype used with
a pipeline. E.g.
pipeline.getSchemaRegistry().registerSchemaProvider(Message.class, new
ProtoMessageSchema());
pipeline.getSchemaRegistry().registerSchemaProvider(TBase.class,
ThriftSchema.provider());
Looking up the schema for any proto message type should resolve to what's
registered for `Message`, unless something more specific is defined for a given
type.
I'd be glad to give it a try, if you think this is worth it.
P.S.
I added some test cases for thrift "default values" in this PR, along with
the proper union mapping that I mentioned. Sorry for the plethora of amends and
force pushes, I just wanted to keep all changes since your last review in one
commit.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 522798)
Time Spent: 2h 40m (was: 2.5h)
> Beam schema for thrift data
> ---------------------------
>
> Key: BEAM-11338
> URL: https://issues.apache.org/jira/browse/BEAM-11338
> Project: Beam
> Issue Type: New Feature
> Components: io-java-utilities
> Reporter: Costi Ciudatu
> Priority: P2
> Labels: thrift
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> Define a SchemaProvider that can handle thrift objects, the same way as
> ProtoMessageSchema handles protobuf.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
