Hi Hongshun, Thanks for your question. Answer your question below:
> Does this mean that users are still required to provide initial credentials (e.g., username and password) when creating the catalog? There are two types of connection: system and catalog. System connection is not bound to catalog and can be used to create catalog. But in the FLIP, we only proposed temporary system connection which is in memory. I updated the FLIP to reflect it. > On the other hand, if a table is created using a catalog that already has an associated connection, it seems no additional connection properties need to be specified — which is great. But then, why do we need per-connection credentials at all? Could the catalog-level connection be sufficient in most cases? I think the credential for tables in the catalog may not be the same as the catalog credential itself. For example, kafka table credentials is not likely to be the credential you use to connect to your catalog. There are also models/functions in catalog which can also have other credentials. Thanks, Hao On Tue, Aug 5, 2025 at 7:54 PM Hongshun Wang <loserwang1...@gmail.com> wrote: > Hi Mayank, > Sorry for joining this FLIP discussion late. I can see that it > significantly reduces complexity and improves security when creating tables. > > However, I have a question regarding its usage in the context of catalogs: > > In the current FLIP, when creating a connection within a catalog, we use: > CREATE [TEMPORARY] [SYSTEM] CONNECTION [IF NOT EXISTS] > [catalog_name.][db_name.]connection_name > [COMMENT connection_comment] > WITH ( > 'type' = 'basic' | 'bearer' | 'oauth' | 'custom_type', > 'url' = 'endpoint_url', > 'username' = 'user_name', -- Will be extracted to SecretStore > 'password' = 'user_password', -- Will be extracted to SecretStore > ... > ) > > But we still need to define the catalog itself with some connection > information upfront. Does this mean that users are still required to > provide initial credentials (e.g., username and password) when creating the > catalog? > > If so, isn't this inconsistent with the goal of centralizing and securing > credentials via the SecretStore? > > > On the other hand, if a table is created using a catalog that already has > an associated connection, it seems no additional connection properties need > to be specified — which is great. But then, why do we need per-connection > credentials at all? Could the catalog-level connection be sufficient in > most cases? > > > Best > Hongshun > > > 2025年4月29日 23:07，Mayank Juneja <mayankjunej...@gmail.com> 写道： > > > > Hi all, > > > > I would like to open up for discussion a new FLIP-529 [1]. > > > > Motivation: > > Currently, Flink SQL handles external connectivity by defining endpoints > > and credentials in table configuration. This approach prevents > reusability > > of these connections and makes table definition less secure by exposing > > sensitive information. > > We propose the introduction of a new "connection" resource in Flink. This > > will be a pluggable resource configured with a remote endpoint and > > associated access key. Once defined, connections can be reused across > table > > definitions, and eventually for model definition (as discussed in > FLIP-437) > > for inference, enabling seamless and secure integration with external > > systems. > > The connection resource will provide a new, optional way to manage > external > > connectivity in Flink. Existing methods for table definitions will remain > > unchanged. > > > > [1] https://cwiki.apache.org/confluence/x/cYroF > > > > Best Regards, > > Mayank Juneja > >
