[ 
https://issues.apache.org/jira/browse/FLINK-41522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated FLINK-41522:
-----------------------------------
    Labels: pull-request-available  (was: )

> FileCatalogStore changes YAML-sensitive string options when reloading catalogs
> ------------------------------------------------------------------------------
>
>                 Key: FLINK-41522
>                 URL: https://issues.apache.org/jira/browse/FLINK-41522
>             Project: Flink
>          Issue Type: Bug
>          Components: Table SQL / API
>    Affects Versions: 2.4.0
>         Environment: Flink 2.4
>            Reporter: Hao Yu
>            Priority: Minor
>              Labels: pull-request-available
>
> h2. Description
> FileCatalogStore does not preserve catalog option strings that require YAML 
> quoting or escaping.
> For example, a generic_in_memory catalog configured with 
> default-database="123" initially uses the database name 123. After its 
> descriptor is persisted and loaded from a new FileCatalogStore instance, the 
> default database becomes '123', including literal single quotation marks.
> Other affected values include boolean-looking strings such as "true", 
> null-looking strings such as "null", "*", empty strings, multiline strings, 
> and strings containing ": ".
> This affects file-backed catalog persistence and reuse across sessions. An 
> already initialized catalog can continue to work with the original options, 
> so the changed value may only become visible when the catalog is loaded again.
> h2. Reproduction
> Configure a file-backed catalog store:
> table.catalog-store.kind: file
> table.catalog-store.file.path: /tmp/flink-catalog-store
> Create a catalog:
> CREATE CATALOG demo WITH (
> 'type' = 'generic_in_memory',
> 'default-database' = '123'
> );
> Load the catalog in a fresh session using the same catalog-store directory, 
> then inspect its databases:
> USE CATALOG demo;
> SHOW DATABASES;
> The underlying failure has been reproduced with a deterministic regression 
> test that:
> Stores a CatalogDescriptor in FileCatalogStore.
> Closes the store and opens a new instance pointing to the same directory.
> Retrieves the descriptor and creates the catalog through FactoryUtil.
> Checks the default database name and whether database "123" exists.
> The full SQL Gateway session sequence above has not been exercised as an 
> end-to-end process test.
> h2. Expected behavior
> Catalog option strings should survive persistence unchanged.
> The reloaded catalog should have default database "123", without additional 
> quotation marks.
> Actual behavior
> The default database assertion fails:
> expected: "123"
> but was: "'123'"
> The single quotation marks become part of the configuration value rather than 
> serving as YAML syntax.
> h2. Root cause
> FileCatalogStore.storeCatalog passes Configuration.toFileWritableMap() to 
> YAMLMapper.writeValue().
> toFileWritableMap() already converts each value into YAML scalar text. 
> YAMLMapper then serializes that text again, treating the quoting or escaping 
> introduced by the first conversion as part of the string.
> getCatalog() decodes only the outer YAML layer and passes the resulting 
> strings to Configuration.fromMap(), retaining the extra quoting or escaping.
> h2. Compatibility
> The proposed fix corrects newly persisted catalog definitions. It does not 
> automatically repair definitions already written with extra YAML encoding. 
> Affected definitions may need to be recreated from their original option 
> values.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to