[
https://issues.apache.org/jira/browse/FLINK-41522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-41522:
-----------------------------------
Labels: pull-request-available (was: )
> FileCatalogStore changes YAML-sensitive string options when reloading catalogs
> ------------------------------------------------------------------------------
>
> Key: FLINK-41522
> URL: https://issues.apache.org/jira/browse/FLINK-41522
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / API
> Affects Versions: 2.4.0
> Environment: Flink 2.4
> Reporter: Hao Yu
> Priority: Minor
> Labels: pull-request-available
>
> h2. Description
> FileCatalogStore does not preserve catalog option strings that require YAML
> quoting or escaping.
> For example, a generic_in_memory catalog configured with
> default-database="123" initially uses the database name 123. After its
> descriptor is persisted and loaded from a new FileCatalogStore instance, the
> default database becomes '123', including literal single quotation marks.
> Other affected values include boolean-looking strings such as "true",
> null-looking strings such as "null", "*", empty strings, multiline strings,
> and strings containing ": ".
> This affects file-backed catalog persistence and reuse across sessions. An
> already initialized catalog can continue to work with the original options,
> so the changed value may only become visible when the catalog is loaded again.
> h2. Reproduction
> Configure a file-backed catalog store:
> table.catalog-store.kind: file
> table.catalog-store.file.path: /tmp/flink-catalog-store
> Create a catalog:
> CREATE CATALOG demo WITH (
> 'type' = 'generic_in_memory',
> 'default-database' = '123'
> );
> Load the catalog in a fresh session using the same catalog-store directory,
> then inspect its databases:
> USE CATALOG demo;
> SHOW DATABASES;
> The underlying failure has been reproduced with a deterministic regression
> test that:
> Stores a CatalogDescriptor in FileCatalogStore.
> Closes the store and opens a new instance pointing to the same directory.
> Retrieves the descriptor and creates the catalog through FactoryUtil.
> Checks the default database name and whether database "123" exists.
> The full SQL Gateway session sequence above has not been exercised as an
> end-to-end process test.
> h2. Expected behavior
> Catalog option strings should survive persistence unchanged.
> The reloaded catalog should have default database "123", without additional
> quotation marks.
> Actual behavior
> The default database assertion fails:
> expected: "123"
> but was: "'123'"
> The single quotation marks become part of the configuration value rather than
> serving as YAML syntax.
> h2. Root cause
> FileCatalogStore.storeCatalog passes Configuration.toFileWritableMap() to
> YAMLMapper.writeValue().
> toFileWritableMap() already converts each value into YAML scalar text.
> YAMLMapper then serializes that text again, treating the quoting or escaping
> introduced by the first conversion as part of the string.
> getCatalog() decodes only the outer YAML layer and passes the resulting
> strings to Configuration.fromMap(), retaining the extra quoting or escaping.
> h2. Compatibility
> The proposed fix corrects newly persisted catalog definitions. It does not
> automatically repair definitions already written with extra YAML encoding.
> Affected definitions may need to be recreated from their original option
> values.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)