[
https://issues.apache.org/jira/browse/KYLIN-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16338745#comment-16338745
]
Zhong Yanghong commented on KYLIN-2894:
---------------------------------------
The previous design for sql response signature may not be correct in case that
some segment is deleted.
The refined signature should contain the following info:
* Segment signature: name + last build time
* Cube signature: name + status(DISABLED, ENABLED) + Set of segment signature
* Hybrid signature: Set of cube signature
* Realization signature: Cube signature or hybrid signature
* SQL response signature: Set of realization signature
> Change the query cache expiration strategy by signature checking
> ----------------------------------------------------------------
>
> Key: KYLIN-2894
> URL: https://issues.apache.org/jira/browse/KYLIN-2894
> Project: Kylin
> Issue Type: Sub-task
> Components: Query Engine
> Reporter: Zhong Yanghong
> Assignee: Zhong Yanghong
> Priority: Major
> Fix For: v2.3.0
>
>
> Currently to invalid query cache, {{CacheService}} will either invoke
> {{cleanDataCache}} or {{cleanAllDataCache}}. Both methods will clear all of
> the query cache, which is very inefficient. In eBay PROD environment, there's
> around 400 cubing jobs per day, which means the query cache will be cleared
> very 4 minutes. Then we introduced a signature based cache invalidation
> strategy. The basic idea is as follows:
> * Add a signature for {{SQLResponse}}, here we choose the cube last build time
> * When fetch {{SQLResponse}} for cache, first check whether the signature is
> consistent. If not, this cached value is overdue and will be invalidate.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
