[
https://issues.apache.org/jira/browse/IMPALA-12375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17755727#comment-17755727
]
Wenzhe Zhou edited comment on IMPALA-12375 at 8/17/23 11:04 PM:
----------------------------------------------------------------
Data source objects are saved in-memory
[cache|https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/catalog/Catalog.java#L93-L94]
in Catalog. They are NOT [persisted to the
metastore|https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/catalog/Catalog.java#L261-L267]
by original design. All data source properties are stored as [table
properties|https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/catalog/DataSourceTable.java#L41-L48]
(persisted in the metastore) so that the DataSource catalog objects are not
needed in order to scan data source tables.
Hive don't have data source object. All properties are specified as table
properties when creating table with JDBC storage.
To make data source object persistent, we need to add new APIs in HMS to
support this Impala specific objects.
was (Author: wzhou):
Data source objects are saved in-memory
[cache|https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/catalog/Catalog.java#L93-L94]
in Catalog. They are [persisted to the
metastore|https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/catalog/Catalog.java#L261-L267]
by original design. All data source properties are stored as [table
properties|https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/catalog/DataSourceTable.java#L41-L48]
(persisted in the metastore) so that the DataSource catalog objects are not
needed in order to scan data source tables.
Hive don't have data source object. All properties are specified as table
properties when creating table with JDBC storage.
To make data source object persistent, we need to add new APIs in HMS to
support this Impala specific objects.
> DataSource ojects are not persistent
> ------------------------------------
>
> Key: IMPALA-12375
> URL: https://issues.apache.org/jira/browse/IMPALA-12375
> Project: IMPALA
> Issue Type: Sub-task
> Components: Backend, Catalog, Frontend
> Reporter: Wenzhe Zhou
> Assignee: Wenzhe Zhou
> Priority: Major
>
> DataSource ojects which are created with "CREATE DATA SOURCE" statements are
> not persistent. The objects are not shown in "show data sources" after the
> mini-cluster is restarted.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
