[
https://issues.apache.org/jira/browse/BEAM-7274?focusedWorklogId=345862&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-345862
]
ASF GitHub Bot logged work on BEAM-7274:
----------------------------------------
Author: ASF GitHub Bot
Created on: 19/Nov/19 08:36
Start Date: 19/Nov/19 08:36
Worklog Time Spent: 10m
Work Description: alexvanboxel commented on issue #8690: [BEAM-7274]
Implement the Protobuf schema provider
URL: https://github.com/apache/beam/pull/8690#issuecomment-555392864
> Another comment: this delegates everything to Proto's reflection API,
which can be quite inefficient. Compare with AvroSchema where we delegate
straight to generated classes when they exist.
**Totally agree, but** I wanted first a solution that works for every
use-case. Optimization can happen afterward. I still have **plans for iterative
improvements**, certainly related to protobuf options first, then optimize for
performance (but I have to make performance tests for this to measure this)
> Reflection might be necessary for the case of DynamicMessage (though in
that case I think we should use RowWithStorage instead of RowWithGetters), but
shouldn't be necessary when we have generated classes.
RowWithStorage could be a solution when the Row is materialized (with the
getToRowFunction), but having the reflection API is still needed it you use the
schema without materializing. Example just you the filter on a column function
on a schema aware collection but keeping the original type, you don't want to
materialize the a big proto just to filter on 1 column. Again, I like to
measure this, but we need a base line.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 345862)
Time Spent: 9h 50m (was: 9h 40m)
> Protobuf Beam Schema support
> ----------------------------
>
> Key: BEAM-7274
> URL: https://issues.apache.org/jira/browse/BEAM-7274
> Project: Beam
> Issue Type: Improvement
> Components: sdk-java-core
> Reporter: Alex Van Boxel
> Assignee: Alex Van Boxel
> Priority: Minor
> Time Spent: 9h 50m
> Remaining Estimate: 0h
>
> Add support for the new Beam Schema to the Protobuf extension.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
