数据回填(write-back)目前是不支持的,也不在计划上。 做到更高效的历史数据更新,就是要尽量不重刷无变化的数据,而变化的数据总要重新刷新。Kylin刷新的单位是Segment。如果不需要“支付”“开票”“创建”之间的关联计算,可以把这些字段拆在不同的Cube中,分别做Partition字段,刷新的时候要重刷新旧两个时间段。但这个可能的潜在风险是不同Cube之间,数据的不一致。
一般来说,数据的更新具备冷热属性,即热数据经常更新,冷数据不更新,因此可以针对冷热,设计不同规模的Segment,减少刷新时的复杂度。 With Warm regards Billy Liu Yun, Henry (BJ/DEL) <[email protected]> 于2019年9月3日周二 下午12:54写道: > 这个问题非常具有普遍意义。我司在开发类似财务指标平台的时候也遇到类似问题。 > > > > 我们不仅需要从Cube里面查询多维度财务指标数据(例如:公司A-华北区-产品A-已支付订单数-10000 > 单),有些没有前端系统的汇报单位还要实时填报多维数据(例如:公司A-西南区-产品A-已支付订单数- > ???单)。西南区的负责人填报后需要能够实时汇总到更上一层的区域当中,以供合并查询。 > > > > 过去我们在Hyperion Essbase中是可以在多维数据库中进行填报的。现在要转型到Kylin,不知填报问题如何解决?请各位大师指点。 > > > > 谢谢! > > 运宁 > > > > *From:* 王刚 [mailto:[email protected]] > *Sent:* Tuesday, September, 03, 2019 11:53 AM > *To:* [email protected] > *Cc:* [email protected] > *Subject:* Re: 事实表历史数据的更新,如何更新cube > > > > 补充一下这个场景: > > > > 有事实表orders,以创建时间作为分区,有已支付订单数,已开票订单数两个指标,都以订单创建时间作为统计分区。 > > > > > > order1这条数据,20190801和20190901分别对时间分区为20190701的数据支付时间和开票时间做了更新。 > > > > 那么20190701这个时间分区的已支付订单数和已开票订单数两个指标都需要更新。 > > > > 在这个场景中,我们如何进行cube的更新呢 > > > > > > 在 2019年9月3日,11:44,王刚 <[email protected]> 写道: > > > > Hi All > > 我是苏宁财务平台的研发,我们在财务指标平台升级计划中正在考虑平台选型,kylin作为考察目标之一。 > > 在目前的测试步骤中,遇到了事实表历史数据更新的问题,请教一下各位developer。 > > 举例hive事实表orders,以创建时间作为时间分区 > > 订单号 > > 创建时间 > > 支付时间 > > 开票时间 > > Order1 > > 20190701 > > 20190801 > > 20190901 > > Order2 > > 20190901 > > 20190901 > > 20190901 > > > > 表中order2的时间分区一致且不更新,比较容易计算cube。在order1这条数据中,20190801和20190901 > 分别对时间分区为20190701的数据支付时间和开票时间做了更新,因此支付和开票相关的统计指标也需要更新。 > > 请问如何配置cube和增量更新方式,能够最高效实现事实表历史数据的指标更新呢? > > kylin新手,诚心请教,烦请各位不吝赐家,多谢。 > > > > **************************************************************************** > Email Disclaimer > ---------------- > The information in this email is confidential and may be legally > privileged. > It is intended solely for the addressee. Access to this email by anyone > else > is unauthorised. > > If you have received this email in error, please reply to the sender and > notify him/her of this and delete the email (including all its enclosures) > and destroy all copies of it. If you are not the intended recipient, any > disclosure, copying, distribution or any action taken or omitted to be > taken in reliance on it, is prohibited and may be unlawful. > > Any opinions, conclusions, advice or statements contained in this email > should not be relied upon unless they are confirmed in writing on hard copy > letterhead. Opinions, conclusions and other information in this email and > any attachments that do not relate to the official business of the firm are > neither given nor endorsed by it. > > There is no guarantee that email communications are secure or error-free, > as information could be intercepted, corrupted, amended, lost, destroyed, > arrive late or incomplete, or contain viruses. > > KPMG, a Hong Kong partnership and KPMG Huazhen LLP, a People's Republic > of China partnership, are member firms of the KPMG network of independent > member firms affiliated with KPMG International Cooperative. > > KPMG and KPMG Huazhen LLP, together with various > affiliated entities operate in the People's Republic of China including > Hong Kong SAR and Macau SAR, as KPMG China. > > > KPMG International Cooperative is a Swiss entity. Member firms of the KPMG > network of independent firms are affiliated with KPMG International > Cooperative. > KPMG International Cooperative provides no client services. > > Web sites: kpmg.com/cn > > ***************************************************************************** >
