[
https://issues.apache.org/jira/browse/KYLIN-2565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16205440#comment-16205440
]
SammiChen commented on KYLIN-2565:
----------------------------------
Thanks Cheng for reporting the issue to track the support of Hadoop 3.0 & EC
for Apache Kylin.
Hadoop 3.0-beta1 was recently released. 3.0 GA will be the next milestone and
happen soon. With Hadoop 3.0 new feature HDFS-EC, massive storage space(for
example 50%) can be saved using this new technology. Apache Kylin consumes
large volume of HDFS data and could generate 20% more data onto HDFS after cube
computing in some cases, therefore HDFS EC should have good opportunities to
optimize the storage cost and even performance.
Discussed with Luke, we’d like to collaborate with his team working on this
support. Here is the rough plan:
1) Verify Apahce Kylin stack works with Hadoop 3.0 and EC
Build and run functional tests. The Kylin related issues will be
reported to Kylin community and all Hadoop EC related issues will go to Hadoop
community;
2) Benchmark and report
Given the functional tests passed, we’ll benchmark Kylin over Hadoop 3.0.
Which Kylin workloads to use could be discussed here, and we’d also like to
share the results.
Any comments? Thanks for your thoughts!
> Upgrade Kylin to Hadoop3.0
> --------------------------
>
> Key: KYLIN-2565
> URL: https://issues.apache.org/jira/browse/KYLIN-2565
> Project: Kylin
> Issue Type: New Feature
> Reporter: Wang Cheng
>
> Hadoop3.0-alpha is released, Kylin should also keep compatible with it. Below
> is the Hadoop3.0 components requirements:
> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3.0.0+release
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
