[
https://issues.apache.org/jira/browse/HIVE-24259?focusedWorklogId=517536&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-517536
]
ASF GitHub Bot logged work on HIVE-24259:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 28/Nov/20 14:20
Start Date: 28/Nov/20 14:20
Worklog Time Spent: 10m
Work Description: sankarh commented on a change in pull request #1610:
URL: https://github.com/apache/hive/pull/1610#discussion_r532042079
##########
File path:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/cache/SharedCache.java
##########
@@ -134,17 +134,8 @@ public int getPosition() {
}
}
- private enum MemberName {
- TABLE_COL_STATS_CACHE,
- PARTITION_CACHE,
- PARTITION_COL_STATS_CACHE,
- AGGR_COL_STATS_CACHE,
- PRIMARY_KEY_CACHE,
- FOREIGN_KEY_CACHE,
- NOTNULL_CONSTRAINT_CACHE,
- UNIQUE_CONSTRAINT_CACHE,
- DEFAULT_CONSTRAINT_CACHE,
- CHECK_CONSTRAINT_CACHE
+ public enum MemberName {
+ TABLE_COL_STATS_CACHE, PARTITION_CACHE, PARTITION_COL_STATS_CACHE,
AGGR_COL_STATS_CACHE, PRIMARY_KEY_CACHE, FOREIGN_KEY_CACHE,
NOTNULL_CONSTRAINT_CACHE, UNIQUE_CONSTRAINT_CACHE, DEFAULT_CONSTRAINT_CACHE,
CHECK_CONSTRAINT_CACHE, ALL_TABLE_CONSTRAINT
Review comment:
Keep one member per line to be more readable.
##########
File path:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/cache/SharedCache.java
##########
@@ -309,12 +302,19 @@ public int getObjectSize(Class<?> clazz, Object obj) {
private Map<String, SQLDefaultConstraint> defaultConstraintCache = new
ConcurrentHashMap<>();
private Map<String, SQLCheckConstraint> checkConstraintCache = new
ConcurrentHashMap<>();
+ private boolean pk = false;
Review comment:
Using too many flags is complex. Check if we can simplify it using below
logic.
1) Fetch all constraints and update cache -> If this update is successful,
then set "isAllConstraintsSet=true"
2) If partially successful due to memory constraint, then set
"isAllConstraintsSet=false" and set bit position of completed constraint "long
constraintStatusBits". We can assign one bit per constraintType and generally
in the order in which we update it. It means, if we encounter a bit which is
unset, then rest of the bits are unset too.
3) We never refresh individual constraints. Always try to update all
constraints by using same method which follow above logic to set the flag.
Shall remove all individual update(CachedStore.updateTablePrimaryKeys...) and
refresh(SharedCache.refreshPrimaryKeysInCache...) methods for each types. This
ensure, we always cache a valid snapshot.
4) When read any constraint, first check isAllConstraintsSet==true. If true,
then assume all are valid and return the cached entry immediately. If false,
then just the corresponding bit in "constraintStatusBits" and if true, then
return the cached entry or else return null and invoke RawStore.
Note: We can also get rid of isAllConstraintsSet flag by using
constraintStatusBits = -1 for "true" scenario.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 517536)
Time Spent: 1h 50m (was: 1h 40m)
> [CachedStore] Optimise get constraints call by removing redundant table check
> ------------------------------------------------------------------------------
>
> Key: HIVE-24259
> URL: https://issues.apache.org/jira/browse/HIVE-24259
> Project: Hive
> Issue Type: Sub-task
> Reporter: Ashish Sharma
> Assignee: Ashish Sharma
> Priority: Minor
> Labels: pull-request-available
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Description -
> Problem -
> 1. Redundant check if table is present or not
> 2. Currently in order to get all constraint form the cachedstore. 6 different
> call is made with in the cached store. Which led to 6 different call to raw
> store
>
> DOD
> 1. Check only once if table exit in cached store.
> 2. Instead of calling individual constraint in cached store. Add a method
> which return all constraint at once and if data is not consistent then fall
> back to rawstore.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
