[jira] [Updated] (KYLIN-2894) Change the query cache expiration strategy by signature checking
[ https://issues.apache.org/jira/browse/KYLIN-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI updated KYLIN-2894: Fix Version/s: v2.6.0 > Change the query cache expiration strategy by signature checking > > > Key: KYLIN-2894 > URL: https://issues.apache.org/jira/browse/KYLIN-2894 > Project: Kylin > Issue Type: Sub-task > Components: Query Engine >Reporter: Zhong Yanghong >Assignee: Wang Ken >Priority: Major > Fix For: v2.6.0 > > > Currently to invalid query cache, {{CacheService}} will either invoke > {{cleanDataCache}} or {{cleanAllDataCache}}. Both methods will clear all of > the query cache, which is very inefficient. In eBay PROD environment, there's > around 400 cubing jobs per day, which means the query cache will be cleared > very 4 minutes. Then we introduced a signature based cache invalidation > strategy. The basic idea is as follows: > * Add a signature for {{SQLResponse}}, here we choose the cube last build time > * When fetch {{SQLResponse}} for cache, first check whether the signature is > consistent. If not, this cached value is overdue and will be invalidate. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-2894) Change the query cache expiration strategy by signature checking
[ https://issues.apache.org/jira/browse/KYLIN-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu updated KYLIN-2894: - Fix Version/s: (was: v2.3.0) > Change the query cache expiration strategy by signature checking > > > Key: KYLIN-2894 > URL: https://issues.apache.org/jira/browse/KYLIN-2894 > Project: Kylin > Issue Type: Sub-task > Components: Query Engine >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong >Priority: Major > > Currently to invalid query cache, {{CacheService}} will either invoke > {{cleanDataCache}} or {{cleanAllDataCache}}. Both methods will clear all of > the query cache, which is very inefficient. In eBay PROD environment, there's > around 400 cubing jobs per day, which means the query cache will be cleared > very 4 minutes. Then we introduced a signature based cache invalidation > strategy. The basic idea is as follows: > * Add a signature for {{SQLResponse}}, here we choose the cube last build time > * When fetch {{SQLResponse}} for cache, first check whether the signature is > consistent. If not, this cached value is overdue and will be invalidate. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-2894) Change the query cache expiration strategy by signature checking
[ https://issues.apache.org/jira/browse/KYLIN-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu updated KYLIN-2894: - Fix Version/s: v2.3.0 > Change the query cache expiration strategy by signature checking > > > Key: KYLIN-2894 > URL: https://issues.apache.org/jira/browse/KYLIN-2894 > Project: Kylin > Issue Type: Sub-task > Components: Query Engine >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong >Priority: Major > Fix For: v2.3.0 > > > Currently to invalid query cache, {{CacheService}} will either invoke > {{cleanDataCache}} or {{cleanAllDataCache}}. Both methods will clear all of > the query cache, which is very inefficient. In eBay PROD environment, there's > around 400 cubing jobs per day, which means the query cache will be cleared > very 4 minutes. Then we introduced a signature based cache invalidation > strategy. The basic idea is as follows: > * Add a signature for {{SQLResponse}}, here we choose the cube last build time > * When fetch {{SQLResponse}} for cache, first check whether the signature is > consistent. If not, this cached value is overdue and will be invalidate. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-2894) Change the query cache expiration strategy by signature checking
[ https://issues.apache.org/jira/browse/KYLIN-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong updated KYLIN-2894: -- Fix Version/s: (was: v2.2.0) > Change the query cache expiration strategy by signature checking > > > Key: KYLIN-2894 > URL: https://issues.apache.org/jira/browse/KYLIN-2894 > Project: Kylin > Issue Type: Sub-task > Components: Query Engine >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > > Currently to invalid query cache, {{CacheService}} will either invoke > {{cleanDataCache}} or {{cleanAllDataCache}}. Both methods will clear all of > the query cache, which is very inefficient. In eBay PROD environment, there's > around 400 cubing jobs per day, which means the query cache will be cleared > very 4 minutes. Then we introduced a signature based cache invalidation > strategy. The basic idea is as follows: > * Add a signature for {{SQLResponse}}, here we choose the cube last build time > * When fetch {{SQLResponse}} for cache, first check whether the signature is > consistent. If not, this cached value is overdue and will be invalidate. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2894) Change the query cache expiration strategy by signature checking
[ https://issues.apache.org/jira/browse/KYLIN-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong updated KYLIN-2894: -- Affects Version/s: (was: v2.1.0) > Change the query cache expiration strategy by signature checking > > > Key: KYLIN-2894 > URL: https://issues.apache.org/jira/browse/KYLIN-2894 > Project: Kylin > Issue Type: Sub-task > Components: Query Engine >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > > Currently to invalid query cache, {{CacheService}} will either invoke > {{cleanDataCache}} or {{cleanAllDataCache}}. Both methods will clear all of > the query cache, which is very inefficient. In eBay PROD environment, there's > around 400 cubing jobs per day, which means the query cache will be cleared > very 4 minutes. Then we introduced a signature based cache invalidation > strategy. The basic idea is as follows: > * Add a signature for {{SQLResponse}}, here we choose the cube last build time > * When fetch {{SQLResponse}} for cache, first check whether the signature is > consistent. If not, this cached value is overdue and will be invalidate. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2894) Change the query cache expiration strategy by signature checking
[ https://issues.apache.org/jira/browse/KYLIN-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong updated KYLIN-2894: -- Issue Type: Sub-task (was: Improvement) Parent: KYLIN-2895 > Change the query cache expiration strategy by signature checking > > > Key: KYLIN-2894 > URL: https://issues.apache.org/jira/browse/KYLIN-2894 > Project: Kylin > Issue Type: Sub-task > Components: Query Engine >Affects Versions: v2.1.0 >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > Fix For: v2.2.0 > > > Currently to invalid query cache, {{CacheService}} will either invoke > {{cleanDataCache}} or {{cleanAllDataCache}}. Both methods will clear all of > the query cache, which is very inefficient. In eBay PROD environment, there's > around 400 cubing jobs per day, which means the query cache will be cleared > very 4 minutes. Then we introduced a signature based cache invalidation > strategy. The basic idea is as follows: > * Add a signature for {{SQLResponse}}, here we choose the cube last build time > * When fetch {{SQLResponse}} for cache, first check whether the signature is > consistent. If not, this cached value is overdue and will be invalidate. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2894) Change the query cache expiration strategy by signature checking
[ https://issues.apache.org/jira/browse/KYLIN-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong updated KYLIN-2894: -- Fix Version/s: v2.2.0 > Change the query cache expiration strategy by signature checking > > > Key: KYLIN-2894 > URL: https://issues.apache.org/jira/browse/KYLIN-2894 > Project: Kylin > Issue Type: Improvement > Components: Query Engine >Affects Versions: v2.1.0 >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > Fix For: v2.2.0 > > > Currently to invalid query cache, {{CacheService}} will either invoke > {{cleanDataCache}} or {{cleanAllDataCache}}. Both methods will clear all of > the query cache, which is very inefficient. In eBay PROD environment, there's > around 400 cubing jobs per day, which means the query cache will be cleared > very 4 minutes. Then we introduced a signature based cache invalidation > strategy. The basic idea is as follows: > * Add a signature for {{SQLResponse}}, here we choose the cube last build time > * When fetch {{SQLResponse}} for cache, first check whether the signature is > consistent. If not, this cached value is overdue and will be invalidate. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2894) Change the query cache expiration strategy by signature checking
[ https://issues.apache.org/jira/browse/KYLIN-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong updated KYLIN-2894: -- Affects Version/s: v2.1.0 > Change the query cache expiration strategy by signature checking > > > Key: KYLIN-2894 > URL: https://issues.apache.org/jira/browse/KYLIN-2894 > Project: Kylin > Issue Type: Improvement > Components: Query Engine >Affects Versions: v2.1.0 >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > Fix For: v2.2.0 > > > Currently to invalid query cache, {{CacheService}} will either invoke > {{cleanDataCache}} or {{cleanAllDataCache}}. Both methods will clear all of > the query cache, which is very inefficient. In eBay PROD environment, there's > around 400 cubing jobs per day, which means the query cache will be cleared > very 4 minutes. Then we introduced a signature based cache invalidation > strategy. The basic idea is as follows: > * Add a signature for {{SQLResponse}}, here we choose the cube last build time > * When fetch {{SQLResponse}} for cache, first check whether the signature is > consistent. If not, this cached value is overdue and will be invalidate. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2894) Change the query cache expiration strategy by signature checking
[ https://issues.apache.org/jira/browse/KYLIN-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong updated KYLIN-2894: -- Summary: Change the query cache expiration strategy by signature checking (was: Change the query cache invalidation strategy by signature checking) > Change the query cache expiration strategy by signature checking > > > Key: KYLIN-2894 > URL: https://issues.apache.org/jira/browse/KYLIN-2894 > Project: Kylin > Issue Type: Improvement > Components: Query Engine >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > > Currently to invalid query cache, {{CacheService}} will either invoke > {{cleanDataCache}} or {{cleanAllDataCache}}. Both methods will clear all of > the query cache, which is very inefficient. In eBay PROD environment, there's > around 400 cubing jobs per day, which means the query cache will be cleared > very 4 minutes. Then we introduced a signature based cache invalidation > strategy. The basic idea is as follows: > * Add a signature for {{SQLResponse}}, here we choose the cube last build time > * When fetch {{SQLResponse}} for cache, first check whether the signature is > consistent. If not, this cached value is overdue and will be invalidate. -- This message was sent by Atlassian JIRA (v6.4.14#64029)