I have a question: Metadata based on MySQL, data in MySQL is KV or relational ?
Thank you! > 2020年7月24日 下午7:23，ShaoFeng Shi <shaofeng...@apache.org> 写道： > > Hello, Kylin users, > > Regarding the Kylin Parquet storage, we hope to update the progress here. At > present, we have completed the main development work[1], design document[2], > and the benchmark. With the new architecture, Kylin is going to be more > efficient and be more cloud-friendly: fully on Spark, less dependency on > Hadoop stack, which made the DevOps easier. > > Here we discuss the future plan, which includes the two aspects. > > 1. The plan for Kylin 4.0 > > In Kylin 3.x, we have released some important functions/features, such as > real-time analysis, Flink building engine, global dictionary with Hive, etc. > In the next phase, we hope to concentrate on the Parquet storage engine and > to release it in Kylin v4.0 within this year. In this period, 3.x will be > keeping maintained for bug fix and security vulnerability, but won't > introduce big change or major features. > > 2. Backward compatibility for HBase storage. > > When we develop the Parquet storage engine, we find it is very difficult to > make the Parquet and HBase engines co-exist. The codebase becomes very > complicated and ugly, inevitably bring big challenges to the maintenance and > release. Besides, as HBase has different APIs (v1.1, v2.0, besides, the CDHs' > are different from the community's'), which makes the testing and release > effort be doubled or tripled in the past years. > > So, we plan to remove the HBase storage engine in Kylin 4.0; The Kylin > metadata will also migrate to MySQL. For existing users, if you want to use > the HBase engine, then keep in the Kylin 3.x; If you want to upgrade to the > Parquet storage, a migration tool can be provided later (another discuss > thread). > > Welcome to tell us your concerns and suggestions! Thank you for your > participation. > > ## Reference > [1] https://github.com/apache/kylin/tree/kylin-on-parquet-v2 > [2] https://cwiki.apache.org/confluence/display/KYLIN/KIP-1%3A+Parquet+storage > > Best regards, > > Shaofeng Shi 史少锋 > Apache Kylin PMC > Email: shaofeng...@apache.org > > Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html > Join Kylin user mail group: user-subscr...@kylin.apache.org > Join Kylin dev mail group: dev-subscr...@kylin.apache.org > >
