[
https://issues.apache.org/jira/browse/HIVE-25900?focusedWorklogId=717764&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-717764
]
ASF GitHub Bot logged work on HIVE-25900:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 31/Jan/22 06:51
Start Date: 31/Jan/22 06:51
Worklog Time Spent: 10m
Work Description: kasakrisz commented on a change in pull request #2984:
URL: https://github.com/apache/hive/pull/2984#discussion_r795383856
##########
File path:
ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveMaterializedViewsRegistry.java
##########
@@ -97,23 +98,83 @@
private static final Logger LOG =
LoggerFactory.getLogger(HiveMaterializedViewsRegistry.class);
private static final String CLASS_NAME =
HiveMaterializedViewsRegistry.class.getName();
- /* Singleton */
- private static final HiveMaterializedViewsRegistry SINGLETON = new
HiveMaterializedViewsRegistry();
+ /**
+ * Registry for materialized views. The goal of this cache is to avoid
parsing and creating
+ * logical plans for the materialized views at query runtime. When a query
arrives, we will
+ * just need to consult this cache and extract the logical plans for the
views (which had
+ * already been parsed) from it. This cache lives in HS2.
+ */
+ public interface MaterializedViewsRegistry {
- private final MaterializedViewsCache materializedViewsCache = new
MaterializedViewsCache();
+ /**
+ * Adds a newly created materialized view to the cache.
Review comment:
Changed cache to registry, since this is more like a registry.
##########
File path:
ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveMaterializedViewsRegistry.java
##########
@@ -97,23 +98,83 @@
private static final Logger LOG =
LoggerFactory.getLogger(HiveMaterializedViewsRegistry.class);
private static final String CLASS_NAME =
HiveMaterializedViewsRegistry.class.getName();
- /* Singleton */
- private static final HiveMaterializedViewsRegistry SINGLETON = new
HiveMaterializedViewsRegistry();
+ /**
+ * Registry for materialized views. The goal of this cache is to avoid
parsing and creating
+ * logical plans for the materialized views at query runtime. When a query
arrives, we will
+ * just need to consult this cache and extract the logical plans for the
views (which had
+ * already been parsed) from it. This cache lives in HS2.
+ */
+ public interface MaterializedViewsRegistry {
- private final MaterializedViewsCache materializedViewsCache = new
MaterializedViewsCache();
+ /**
+ * Adds a newly created materialized view to the cache.
+ */
+ default void createMaterializedView(HiveConf conf, Table
materializedViewTable) { }
- /* Whether the cache has been initialized or not. */
- private final AtomicBoolean initialized = new AtomicBoolean(false);
+ /**
+ * Update the materialized view in the registry (if existing materialized
view matches).
+ */
+ default void refreshMaterializedView(HiveConf conf, Table
materializedViewTable) {
+
+ }
- private HiveMaterializedViewsRegistry() {
+ /**
+ * Update the materialized view in the registry (if existing materialized
view matches).
+ */
+ default void refreshMaterializedView(HiveConf conf, Table
oldMaterializedViewTable, Table materializedViewTable) {
+
+ }
+
+ /**
+ * Removes the materialized view from the cache (based on table object
equality), if exists.
+ */
+ default void dropMaterializedView(Table materializedViewTable) { }
+
+ /**
+ * Removes the materialized view from the cache (based on qualified name),
if exists.
+ */
+ default void dropMaterializedView(String dbName, String tableName) { }
+
+ /**
+ * Returns all the materialized views enabled for Calcite based rewriting
in the cache.
+ *
+ * @return the collection of materialized views, or the empty collection
if none
+ */
+ default List<HiveRelOptMaterialization> getRewritingMaterializedViews() {
return Collections.emptyList(); }
+
+ /**
+ * Returns the materialized views in the cache for the given database.
+ *
+ * @return the collection of materialized views, or the empty collection
if none
+ */
Review comment:
Fixed
##########
File path:
ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveMaterializedViewsRegistry.java
##########
@@ -97,23 +98,83 @@
private static final Logger LOG =
LoggerFactory.getLogger(HiveMaterializedViewsRegistry.class);
private static final String CLASS_NAME =
HiveMaterializedViewsRegistry.class.getName();
- /* Singleton */
- private static final HiveMaterializedViewsRegistry SINGLETON = new
HiveMaterializedViewsRegistry();
+ /**
+ * Registry for materialized views. The goal of this cache is to avoid
parsing and creating
+ * logical plans for the materialized views at query runtime. When a query
arrives, we will
+ * just need to consult this cache and extract the logical plans for the
views (which had
+ * already been parsed) from it. This cache lives in HS2.
+ */
+ public interface MaterializedViewsRegistry {
- private final MaterializedViewsCache materializedViewsCache = new
MaterializedViewsCache();
+ /**
+ * Adds a newly created materialized view to the cache.
+ */
+ default void createMaterializedView(HiveConf conf, Table
materializedViewTable) { }
- /* Whether the cache has been initialized or not. */
- private final AtomicBoolean initialized = new AtomicBoolean(false);
+ /**
+ * Update the materialized view in the registry (if existing materialized
view matches).
+ */
+ default void refreshMaterializedView(HiveConf conf, Table
materializedViewTable) {
+
+ }
- private HiveMaterializedViewsRegistry() {
+ /**
+ * Update the materialized view in the registry (if existing materialized
view matches).
+ */
+ default void refreshMaterializedView(HiveConf conf, Table
oldMaterializedViewTable, Table materializedViewTable) {
+
+ }
+
+ /**
+ * Removes the materialized view from the cache (based on table object
equality), if exists.
+ */
+ default void dropMaterializedView(Table materializedViewTable) { }
+
+ /**
+ * Removes the materialized view from the cache (based on qualified name),
if exists.
+ */
+ default void dropMaterializedView(String dbName, String tableName) { }
+
+ /**
+ * Returns all the materialized views enabled for Calcite based rewriting
in the cache.
+ *
+ * @return the collection of materialized views, or the empty collection
if none
+ */
+ default List<HiveRelOptMaterialization> getRewritingMaterializedViews() {
return Collections.emptyList(); }
+
+ /**
+ * Returns the materialized views in the cache for the given database.
+ *
+ * @return the collection of materialized views, or the empty collection
if none
+ */
+ default HiveRelOptMaterialization getRewritingMaterializedView(
+ String dbName, String viewName,
EnumSet<HiveRelOptMaterialization.RewriteAlgorithm> scope) {
+ return null;
+ }
+
+ default List<HiveRelOptMaterialization>
getRewritingMaterializedViews(String querySql) {
+ return Collections.emptyList();
+ }
+
+ default boolean isEmpty() {
+ return true;
+ }
Review comment:
Added.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 717764)
Time Spent: 50m (was: 40m)
> Materialized view registry does not clean non existing views at refresh
> -----------------------------------------------------------------------
>
> Key: HIVE-25900
> URL: https://issues.apache.org/jira/browse/HIVE-25900
> Project: Hive
> Issue Type: Bug
> Components: Materialized views
> Reporter: Krisztian Kasa
> Assignee: Krisztian Kasa
> Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> CBO plans of materialized views which are enabled for query rewrite are
> cached in HS2 (MaterializedViewsCache, HiveMaterializedViewsRegistry)
> The registry is refreshed periodically from HMS:
> {code:java}
> set hive.server2.materializedviews.registry.refresh.period=1500s;
> {code}
> This functionality is required when multiple HS2 instances are used in a
> cluster: MV drop operation is served by one of the HS2 instances and the
> registry is updated at that time in that instance. However other HS2
> instances still cache the non-existent view and need to be refreshed by the
> updater thread.
> Currently the updater thread adds new entries, refresh existing ones but does
> not remove the outdated entries.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
