BKRV created KYLIN-4655:
---
Summary: Load HFile to HBase Table step is loading the files over
and over
Key: KYLIN-4655
URL: https://issues.apache.org/jira/browse/KYLIN-4655
Project: Kylin
Issue Type:
Agreed with Xiaoxiang. I think it's a good proposal. Maintaining system
availability is important. But we need to verify whether the solution you
provided is feasible and do more tests.
在 2020/7/22 10:43，“Xiaoxiang Yu” 写入:
Hi,
It is a fancy idea from my side. Since it is a major
Zhong Yanghong created KYLIN-4653:
-
Summary: Don't set capacity for the LinkedBlockingQueue of
BlockingReservoir
Key: KYLIN-4653
URL: https://issues.apache.org/jira/browse/KYLIN-4653
Project: Kylin
create jira KYLIN-4654
在 2020-07-22 05:39:49，"chuxiao" 写道：
>1.读写锁改成分桶和自旋锁，允许瞬时的脏读。
>2.更新元数据时不再广播自己，即修改元数据的进程。需要靠广播刷新的缓存，在更新操作时同步刷新。
>3.缓存更新细粒度，按最小原子进行更新，考虑记录变更的版本号时间戳
>依次往下修改，直到满足设计目标。
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>在 2020-07-22 05:27:24，"chuxiao" 写道：
chuxiao created KYLIN-4654:
--
Summary: new metadata read/write and reload mechanism
Key: KYLIN-4654
URL: https://issues.apache.org/jira/browse/KYLIN-4654
Project: Kylin
Issue Type: Improvement
Hi,
It is a fancy idea from my side. Since it is a major behavior change for
Kylin core system and may have impact to all components, please provided us
your detailed design documentation (better in English ^_^).
I think you may implement your idea and verify if it is works as you
Zhong Yanghong created KYLIN-4652:
-
Summary: Fix sum(case when)
Key: KYLIN-4652
URL: https://issues.apache.org/jira/browse/KYLIN-4652
Project: Kylin
Issue Type: Bug
Reporter:
1.读写锁改成分桶和自旋锁，允许瞬时的脏读。
2.更新元数据时不再广播自己，即修改元数据的进程。需要靠广播刷新的缓存，在更新操作时同步刷新。
3.缓存更新细粒度，按最小原子进行更新，考虑记录变更的版本号时间戳
依次往下修改，直到满足设计目标。
在 2020-07-22 05:27:24，"chuxiao" 写道：
>kylin当前的缓存机制，比较适合cube数几十上百，不频繁更新的场景。
kylin当前的缓存机制，比较适合cube数几十上百，不频繁更新的场景。
当有2千以上的cube，每天更新几百个，metadata上的读写锁，任何修改全更新的广播机制，导致一旦有持续的元数据更新，整个集群响应时间大幅拉长，建模和查询频繁出现超时。
KYLIN-4169缓解了这个问题，但还不够。
我想重新设计缓存功能，设计目标是在单个项目1万个cube，每天新建/删除一千个的场景下，建模和查询不会出现频繁超时，保持系统可用性。
