[
https://issues.apache.org/jira/browse/KYLIN-750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14558061#comment-14558061
]
Shaofeng SHI commented on KYLIN-750:
------------------------------------
The first version has been developed (see MergeCuboidFromHBaseJob.java in
branch 0.8.0); While to gain better performance and minimal the impact to HBase
server, we need seek the way to run MR over snapshot files; see
https://hbase.apache.org/apidocs/org/apache/hadoop/hbase/mapreduce/TableSnapshotInputFormat.html
Currently hbase doesn't support running MR over multiple snapshots; there is an
open JIRA with it:
https://issues.apache.org/jira/browse/HBASE-13356
We need figure out how to do the cube merge with this way.
> Merge cube segments from HBase table
> ------------------------------------
>
> Key: KYLIN-750
> URL: https://issues.apache.org/jira/browse/KYLIN-750
> Project: Kylin
> Issue Type: Sub-task
> Components: Job Engine
> Affects Versions: v0.8.1
> Reporter: Shaofeng SHI
> Assignee: Shaofeng SHI
> Fix For: v0.8.1
>
>
> With the new cubing algorithm, there is no intermediate cuboid files
> persisted, so when merge the cube segments, need read the data from HBase
> table directly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
