+1 the new Kylin version should target Hadoop 3 Best Regards! ---------------------
Luke Han On Mon, Mar 2, 2020 at 4:41 PM Billy Liu <billy...@apache.org> wrote: > +1. Let's move to Hadoop3 > > With Warm regards > > Billy Liu > > > ShaoFeng Shi <shaofeng...@apache.org> 于2020年2月27日周四 下午10:07写道： > >> Hi Yang, >> >> The main difference between 2.6 and 3.0 is the new real-time OLAP >> feature. Hadoop 2 users can select either of them, depends on whether they >> need the real-time feature. >> >> After 3.0, the next major features would be the Flink cube engine >> (planned in v3.1) and the Parquet storage (early stage, maybe in v4.0). >> >> When the parquet storage is released, as the dependency on HBase can be >> dropped, then we assume the API issue will easier than today. We can >> re-evaluate the possibility to support Hadoop 2. >> >> So I think the impact on today's Hadoop 2 users is acceptable. Not >> mention that they still can manually compile that. >> >> >> Best regards, >> >> Shaofeng Shi 史少锋 >> Apache Kylin PMC >> Email: shaofeng...@apache.org >> >> Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html >> Join Kylin user mail group: user-subscr...@kylin.apache.org >> Join Kylin dev mail group: dev-subscr...@kylin.apache.org >> >> >> >> >> Li Yang <liy...@apache.org> 于2020年2月27日周四 上午7:37写道： >> >>> The proposal means Kylin 3.0 will be the last major version that supports >>> Hadoop 2. >>> >>> What will be recommended version for Hadoop 2 users after this? I feel >>> the >>> latest stable version of 2.6 is better than 3.0. >>> >>> Anyway, I'm fine with moving focus to Hadoop 3. That is the direction. >>> However we shall also think about what it means for Hadoop 2 users. >>> Questions like below shall also be answered. >>> >>> - What is the recommended version/branch for Hadoop 2? (Btw, 3.0 does not >>> sound right here.) >>> - How that version/branch will be maintained? >>> >>> +1 in general >>> >>> Regards >>> -Yang >>> >>> >>> On Wed, Feb 26, 2020 at 5:36 PM Zhou Kang <zhoukan...@outlook.com> >>> wrote: >>> >>> > +1 >>> > >>> > >>> > > 2020年2月26日 下午3:48，ShaoFeng Shi <shaofeng...@apache.org> 写道： >>> > > >>> > > Hello, Kylin users and developers, >>> > > >>> > > As we know Hadoop 3 and HBase 2 have released for some time. Kylin >>> > starts to support Hadoop 3 since v2.5.0 in Sep 2018. As the APIs of >>> HBase >>> > 1 and 2 are incompatible, we need to keep different branches for them. >>> And >>> > in each release, we need to build separate packages and do a round of >>> > testing for them separately. Furthermore, Cloudera's API difference >>> with >>> > the Apache release makes the situation worse; We need to build 4 binary >>> > packages for reach release. That has spent much of our manual effort >>> and >>> > computing resources. >>> > > >>> > > Today, Hadoop 3 + HBase 2 becomes enough mature and stable for >>> > production use; And we see more and more users are starting to use the >>> new >>> > versions. We think it is time for Kylin to totally upgrade to the new >>> > version. So that we can focus more on Kylin itself, instead of >>> environments. >>> > > >>> > > Here is my proposal: >>> > > 1) From Kylin 3.1, Hadoop/HBase version upgrades to 3.1/2.1 (or a >>> close >>> > version); >>> > > 2) Hadoop 2 and HBase 1 users can use Kylin 3.0 and previous >>> releases; >>> > > 3) We will re-evaluate the need for building binary packages for >>> > Cloudera release. (we may raise another discuss) >>> > > >>> > > Please let us know your comments. And please also understand with the >>> > limited resource we couldn't support multiple Hadoop versions... >>> > > >>> > > Thanks! >>> > > >>> > > Best regards, >>> > > >>> > > Shaofeng Shi 史少锋 >>> > > Apache Kylin PMC >>> > > Email: shaofeng...@apache.org >>> > > >>> > > Apache Kylin FAQ: >>> https://kylin.apache.org/docs/gettingstarted/faq.html >>> > > Join Kylin user mail group: user-subscr...@kylin.apache.org >>> > > Join Kylin dev mail group: dev-subscr...@kylin.apache.org >>> > > >>> > > >>> > >>> > >>> >>
