[jira] [Updated] (FLINK-17909) Add a TestCatalog to hold the serialized meta data to uncover more potential bugs

2021-04-02 Thread Jark Wu (Jira) 


 [ 
https://issues.apache.org/jira/browse/FLINK-17909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jark Wu updated FLINK-17909:

Fix Version/s: (was: 1.13.0)

> Add a TestCatalog to hold the serialized meta data to uncover more potential 
> bugs
> -
>
> Key: FLINK-17909
> URL: https://issues.apache.org/jira/browse/FLINK-17909
> Project: Flink
>  Issue Type: Improvement
>  Components: Table SQL / API, Table SQL / Planner
>Reporter: Jark Wu
>Priority: Major
>
> Currently, the builtin {{GenericInMemoryCatalog}} hold the meta objects in 
> HashMap. However, this lead to many bugs when users switch to some persisted 
> catalogs, e.g. Hive Metastore. For example, FLINK-17189, FLINK-17868, 
> FLINK-16021. 
> That is because the builtin {{GenericInMemoryCatalog}} doesn't cover the 
> important path of serialization and deserialization of meta data. We missed 
> some important meta information (PK, time attributes) when serialization and 
> deserialization which lead to bugs. 
> So I propose to add a {{TestCatalog}} to hold the serialized meta data to 
> cover the serialization and deserializtion path. The serialized meta data may 
> be in the {{Map}} properties format. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (FLINK-17909) Add a TestCatalog to hold the serialized meta data to uncover more potential bugs

2020-11-16 Thread Leonard Xu (Jira) 


 [ 
https://issues.apache.org/jira/browse/FLINK-17909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Leonard Xu updated FLINK-17909:
---
Fix Version/s: (was: 1.12.0)
   1.13.0

> Add a TestCatalog to hold the serialized meta data to uncover more potential 
> bugs
> -
>
> Key: FLINK-17909
> URL: https://issues.apache.org/jira/browse/FLINK-17909
> Project: Flink
>  Issue Type: Improvement
>  Components: Table SQL / API, Table SQL / Planner
>Reporter: Jark Wu
>Priority: Major
> Fix For: 1.13.0
>
>
> Currently, the builtin {{GenericInMemoryCatalog}} hold the meta objects in 
> HashMap. However, this lead to many bugs when users switch to some persisted 
> catalogs, e.g. Hive Metastore. For example, FLINK-17189, FLINK-17868, 
> FLINK-16021. 
> That is because the builtin {{GenericInMemoryCatalog}} doesn't cover the 
> important path of serialization and deserialization of meta data. We missed 
> some important meta information (PK, time attributes) when serialization and 
> deserialization which lead to bugs. 
> So I propose to add a {{TestCatalog}} to hold the serialized meta data to 
> cover the serialization and deserializtion path. The serialized meta data may 
> be in the {{Map}} properties format. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (FLINK-17909) Add a TestCatalog to hold the serialized meta data to uncover more potential bugs

2020-05-27 Thread Danny Chen (Jira) 


 [ 
https://issues.apache.org/jira/browse/FLINK-17909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Danny Chen updated FLINK-17909:
---
Fix Version/s: (was: 1.11.0)
   1.12.0

> Add a TestCatalog to hold the serialized meta data to uncover more potential 
> bugs
> -
>
> Key: FLINK-17909
> URL: https://issues.apache.org/jira/browse/FLINK-17909
> Project: Flink
>  Issue Type: Improvement
>  Components: Table SQL / API, Table SQL / Planner
>Reporter: Jark Wu
>Priority: Major
> Fix For: 1.12.0
>
>
> Currently, the builtin {{GenericInMemoryCatalog}} hold the meta objects in 
> HashMap. However, this lead to many bugs when users switch to some persisted 
> catalogs, e.g. Hive Metastore. For example, FLINK-17189, FLINK-17868, 
> FLINK-16021. 
> That is because the builtin {{GenericInMemoryCatalog}} doesn't cover the 
> important path of serialization and deserialization of meta data. We missed 
> some important meta information (PK, time attributes) when serialization and 
> deserialization which lead to bugs. 
> So I propose to add a {{TestCatalog}} to hold the serialized meta data to 
> cover the serialization and deserializtion path. The serialized meta data may 
> be in the {{Map}} properties format. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (FLINK-17909) Add a TestCatalog to hold the serialized meta data to uncover more potential bugs

2020-05-25 Thread Jark Wu (Jira) 


 [ 
https://issues.apache.org/jira/browse/FLINK-17909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jark Wu updated FLINK-17909:

Description: 
Currently, the builtin {{GenericInMemoryCatalog}} hold the meta objects in 
HashMap. However, this lead to many bugs when users switch to some persisted 
catalogs, e.g. Hive Metastore. For example, FLINK-17189, FLINK-17868, 
FLINK-16021. 

That is because the builtin {{GenericInMemoryCatalog}} doesn't cover the 
important path of serialization and deserialization of meta data. We missed 
some important meta information (PK, time attributes) when serialization and 
deserialization which lead to bugs. 

So I propose to add a {{TestCatalog}} to hold the serialized meta data to cover 
the serialization and deserializtion path. The serialized meta data may be in 
the {{Map}} properties format. 



  was:
Currently, the builtin {{GenericInMemoryCatalog}} hold the meta objects in 
HashMap. However, this lead to many bugs when users switch to some persisted 
catalogs, e.g. Hive Metastore. For example, FLINK-17189, FLINK-17868, 
FLINK-16021. 

That is because the builtin {{GenericInMemoryCatalog}} doesn't cover the 
important path of serialization and deserialization of meta data. We missed 
some important meta information (PK, time attributes) when serialization and 
deserialization which lead to bugs. 

So I propose to hold the serialized meta data in {{GenericInMemoryCatalog}} to 
cover the serialization and deserializtion path. The serialized meta data may 
be in the {{Map}} properties format. 

We may lose some performance here, but {{GenericInMemoryCatalog}} is mostly 
used in demo/experiment/testing, so I think it's fine. 







> Add a TestCatalog to hold the serialized meta data to uncover more potential 
> bugs
> -
>
> Key: FLINK-17909
> URL: https://issues.apache.org/jira/browse/FLINK-17909
> Project: Flink
>  Issue Type: Improvement
>  Components: Table SQL / API, Table SQL / Planner
>Reporter: Jark Wu
>Priority: Major
> Fix For: 1.11.0
>
>
> Currently, the builtin {{GenericInMemoryCatalog}} hold the meta objects in 
> HashMap. However, this lead to many bugs when users switch to some persisted 
> catalogs, e.g. Hive Metastore. For example, FLINK-17189, FLINK-17868, 
> FLINK-16021. 
> That is because the builtin {{GenericInMemoryCatalog}} doesn't cover the 
> important path of serialization and deserialization of meta data. We missed 
> some important meta information (PK, time attributes) when serialization and 
> deserialization which lead to bugs. 
> So I propose to add a {{TestCatalog}} to hold the serialized meta data to 
> cover the serialization and deserializtion path. The serialized meta data may 
> be in the {{Map}} properties format. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (FLINK-17909) Add a TestCatalog to hold the serialized meta data to uncover more potential bugs

2020-05-25 Thread Jark Wu (Jira) 


 [ 
https://issues.apache.org/jira/browse/FLINK-17909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jark Wu updated FLINK-17909:

Summary: Add a TestCatalog to hold the serialized meta data to uncover more 
potential bugs  (was: Make the GenericInMemoryCatalog to hold the serialized 
meta data to uncover more potential bugs)

> Add a TestCatalog to hold the serialized meta data to uncover more potential 
> bugs
> -
>
> Key: FLINK-17909
> URL: https://issues.apache.org/jira/browse/FLINK-17909
> Project: Flink
>  Issue Type: Improvement
>  Components: Table SQL / API, Table SQL / Planner
>Reporter: Jark Wu
>Priority: Major
> Fix For: 1.11.0
>
>
> Currently, the builtin {{GenericInMemoryCatalog}} hold the meta objects in 
> HashMap. However, this lead to many bugs when users switch to some persisted 
> catalogs, e.g. Hive Metastore. For example, FLINK-17189, FLINK-17868, 
> FLINK-16021. 
> That is because the builtin {{GenericInMemoryCatalog}} doesn't cover the 
> important path of serialization and deserialization of meta data. We missed 
> some important meta information (PK, time attributes) when serialization and 
> deserialization which lead to bugs. 
> So I propose to hold the serialized meta data in {{GenericInMemoryCatalog}} 
> to cover the serialization and deserializtion path. The serialized meta data 
> may be in the {{Map}} properties format. 
> We may lose some performance here, but {{GenericInMemoryCatalog}} is mostly 
> used in demo/experiment/testing, so I think it's fine. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)