Hello Folks, We are actively working towards Apache Kylin 2.0 Release and would like a discussion with the community on what they would like to see in 2.0 release of the product. We have three big rock items we are working towards in 2.0 and lot of additional minor feature enhancements.
Streaming Data Source support. This feature is semi baked in where the source of Kylin Cubes is Kafka Topics. Cube Segment are built on micro batches of messages arriving on Kafka topics. Currently a lot of work is going on to productize this feature. Primary areas of work are Stream Processing Engines/Frameworks to process the micro batches and UI to support out of the box integration of Kafka topics with Kylin Cubes. Spark based Cube building Engine. The initial performance numbers for a Spark based cubing engine did not show substantial improvement over MR based engine, but would like this feature to be baked in for the 2.0 Release. Lot of work underway to stabilize this feature. Amazon EMR Integration We had initial conversations with Amazon EMR to support Apache Kylin on Amazon EMR which was received well. With Kylin 2.0 Apache Kylin will be enabled feature on Amazon EMR. Limited work has gone into this area, but this will be an important milestone for 2.0 We are also working towards creating an area for community driven improvements page similar to Apache Kafka’s KIP https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals. Stay tuned. Regards Seshu Adunuthula
