[jira] [Assigned] (FLINK-17909) Make the GenericInMemoryCatalog to hold the serialized meta data to uncover more potential bugs
[
https://issues.apache.org/jira/browse/FLINK-17909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jark Wu reassigned FLINK-17909:
---
Assignee: (was: Jark Wu)
> Make the GenericInMemoryCatalog to hold the serialized meta data to uncover
> more potential bugs
> ---
>
> Key: FLINK-17909
> URL: https://issues.apache.org/jira/browse/FLINK-17909
> Project: Flink
> Issue Type: Improvement
> Components: Table SQL / API, Table SQL / Planner
>Reporter: Jark Wu
>Priority: Major
> Fix For: 1.11.0
>
>
> Currently, the builtin {{GenericInMemoryCatalog}} hold the meta objects in
> HashMap. However, this lead to many bugs when users switch to some persisted
> catalogs, e.g. Hive Metastore. For example, FLINK-17189, FLINK-17868,
> FLINK-16021.
> That is because the builtin {{GenericInMemoryCatalog}} doesn't cover the
> important path of serialization and deserialization of meta data. We missed
> some important meta information (PK, time attributes) when serialization and
> deserialization which lead to bugs.
> So I propose to hold the serialized meta data in {{GenericInMemoryCatalog}}
> to cover the serialization and deserializtion path. The serialized meta data
> may be in the {{Map}} properties format.
> We may lose some performance here, but {{GenericInMemoryCatalog}} is mostly
> used in demo/experiment/testing, so I think it's fine.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (FLINK-17909) Make the GenericInMemoryCatalog to hold the serialized meta data to uncover more potential bugs
[
https://issues.apache.org/jira/browse/FLINK-17909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jark Wu reassigned FLINK-17909:
---
Assignee: Jark Wu
> Make the GenericInMemoryCatalog to hold the serialized meta data to uncover
> more potential bugs
> ---
>
> Key: FLINK-17909
> URL: https://issues.apache.org/jira/browse/FLINK-17909
> Project: Flink
> Issue Type: Improvement
> Components: Table SQL / API, Table SQL / Planner
>Reporter: Jark Wu
>Assignee: Jark Wu
>Priority: Major
> Fix For: 1.11.0
>
>
> Currently, the builtin {{GenericInMemoryCatalog}} hold the meta objects in
> HashMap. However, this lead to many bugs when users switch to some persisted
> catalogs, e.g. Hive Metastore. For example, FLINK-17189, FLINK-17868,
> FLINK-16021.
> That is because the builtin {{GenericInMemoryCatalog}} doesn't cover the
> important path of serialization and deserialization of meta data. We missed
> some important meta information (PK, time attributes) when serialization and
> deserialization which lead to bugs.
> So I propose to hold the serialized meta data in {{GenericInMemoryCatalog}}
> to cover the serialization and deserializtion path. The serialized meta data
> may be in the {{Map}} properties format.
> We may lose some performance here, but {{GenericInMemoryCatalog}} is mostly
> used in demo/experiment/testing, so I think it's fine.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
