[
https://issues.apache.org/jira/browse/BEAM-4081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16491833#comment-16491833
]
Kenneth Knowles commented on BEAM-4081:
---------------------------------------
Digging into the code, I see there are a number of issues and probably some
standard ways to resolve them.
1. We need to be able to convert Beam schemas to something Calcite understands.
This should live in Beam SQL. Right now it is partially addressed by Beam type
with metadata. When converting back, the SQL statement should retain enough
metadata that Beam does not need to propagate it (as when you compile a typed
language to assembly, etc, the higher-level language can handle managing the
metadata).
2. We need to convert Calcite types to Beam schemas. This is fine to do by just
an embedding.
3. A lot of things are done with type names instead of types. I'm fixing that
up.
I'm working through it. There are a lot of coupled decisions that make basic
changes pretty sweeping.
> Review of schema metadata vs schema types
> -----------------------------------------
>
> Key: BEAM-4081
> URL: https://issues.apache.org/jira/browse/BEAM-4081
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-java-core
> Reporter: Kenneth Knowles
> Priority: Major
>
> The Schema basic types have a place for metadata that can say "this int is
> really millis since epoch". This deserves some careful design review and
> perhaps some of these need to be promoted to basic types.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
