[
https://issues.apache.org/jira/browse/FLINK-12771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16863762#comment-16863762
]
Dawid Wysakowicz commented on FLINK-12771:
------------------------------------------
Hi [~xuefuz],
Ad.1 No questions about that. I agree.
Ad.2 Disagree, isn't the reason for having HiveCatalog to communicate with Hive
and store objects in HiveMetastore? What does it mean
> Hive catalog supports temporary objects
How does Hive metastore support temporary objects?
Ad.3 I think it should. How can a user tell if a table will be stored and
persisted between sessions in a HiveMetastore or it will be just a temporary
thing? If I were the user and succesfully registered a table in a HiveCatalog I
would be super surprised that it is not available once I restart my session.
Ad.4 First I agree the story behind registering tables is not explored enough.
We miss a lot of pieces here. But I think there is a huge flaw in this design
that user cannot tell if a Table will or will not be stored.
Ad.5 This is the same argument as 2&3.
> Support ConnectorCatalogTable in HiveCatalog
> --------------------------------------------
>
> Key: FLINK-12771
> URL: https://issues.apache.org/jira/browse/FLINK-12771
> Project: Flink
> Issue Type: Sub-task
> Components: Connectors / Hive
> Reporter: Bowen Li
> Assignee: Bowen Li
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.9.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Currently {{HiveCatalog}} does not support {{ConnectorCatalogTable}}. There's
> a major drawback on this when it comes to real use cases, that is when Table
> API users set a {{HiveCatalog}} as their default catalog (which is very
> likely), they cannot create or use any inline table sources/sinks with their
> default catalog any more. It's really inconvenient for Table API users to use
> Flink for exploration, experiment, and production.
> There are several workaround in this case. E.g. users have to switch their
> default catalog, but that misses our original intention of having a default
> {{HiveCatalog}}; or users can register their inline source/sinks to Flink's
> default catalog which is a in memory catalog, but that not only require users
> to type full path of a table but also requires users to be aware of the
> Flink's default catalog, default db, and their names. In short, none of the
> workaround seems to be reasonable and user friendly.
> From another perspective, Hive has the concept of temporary tables that are
> stored in memory of Hive metastore client and are removed when client is shut
> down. In Flink, {{ConnectorCatalogTable}} can be seen as a type of
> session-based temporary table, and {{HiveCatalog}} (potentially any catalog
> implementations) can store it in memory. By introducing the concept of temp
> table, we could greatly eliminate frictions for users and raise their
> experience and productivity.
> Thus, we propose adding a simple in memory map for {{ConnectorCatalogTable}}
> in {{HiveCatalog}} to allow users create and use inline source/sink when
> their default catalog is a {{HiveCatalog}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
