Streaming cubing is the major difference from Druid. SQL interface and Hadoop family add to that.
On Wed, Jan 28, 2015 at 9:36 AM, Luke Han <[email protected]> wrote: > Thanks Hongbin and Seshu. > > For the Streaming one, there's proposal from eBay Nous team, please refer > to this wiki and discussion for more detail: > > https://github.com/KylinOLAP/Kylin/wiki/%5BProposal%5D-Kylin-Streaming-Cube-Builder > > Previous discussion > https://groups.google.com/forum/#!topic/kylin-olap/xc7EwdHgmBA > > The majority goal of Streaming is to accelerate cube build process to > reduce data time-to-market for end user. > > Thanks. > Luke > > 2015-01-27 23:00 GMT+08:00 Adunuthula, Seshu <[email protected]>: > > > The Roadmap looks great. Some feedback. > > > > 0.73 Release > > - Publish TPC-DS Results. > > > > 0.8x Release > > - Streaming OLAP proposed here, is it different than other offerings like > > Druid? > > - How would Kylin differentiate itself from Druid? > > > > Regards > > Seshu > > > > On 1/27/15, 6:36 AM, "Luke Han" <[email protected]> wrote: > > > > >Hi all, > > > I have updated a new version of this plan at: > > >https://issues.apache.org/jira/browse/KYLIN-577 > > > Changes highlights: > > > 1. We both agreed to keep v1.0 as more high bar to achieve, continue > > >to > > >use 0.6/0.7/0.8/0.9 for different release target > > > 2. Bring DR/Security and others into plan > > > 3. Kylin JIRA Project updated with these versions: > > > > > > https://issues.apache.org/jira/browse/KYLIN/?selectedTab=com.atlassian.jir > > >a.jira-projects-plugin:versions-panel > > > > > > Please kindly help to review and let's know if there's any issue. > > > > > > Thank you very much. > > > > > >Luke > > > > > > > > >==Release Map== > > > > > > https://issues.apache.org/jira/secure/attachment/12694778/Kylin%20Release% > > >20Plan%20-%20H1%202015.png > > > > > >==Text Copy== > > >v0.6.5: MOLAP version > > >Goal: > > >MOLAP capability to serve SQL query with low latency > > >Current version to be released with more stable and bug fixed. So far > > >there's no critical issues. > > >Key features: > > >1. Pre-Calculated Cube > > >2. Incremental refresh > > >3. ANSI SQL Support > > >4. Web Interface > > >5. ODBC Driver > > >Target release date: > > >Jan 2015 > > > > > >v0.7.x: InvertedIndex (HybridOLAP) > > >Goal: > > >Introduce InvertedIndex to optimise queries on raw data and low level > > >aggregation > > >Support multiple storage and Smart Query Router > > >More better user experience > > >Key Features: > > >1. Bitmap InvertedIndex for raw data and low level aggregation > > >2. Logical Model (Data Model) and Physical Model (Storage) > > >3. Smart Query Router (Cost-based query router) > > >4. New Cube Designer > > >5. New Job Manager > > >6. JDBC Driver > > >Target release date > > >Q1 2015 > > >v0.7.3: Capacity and Automation > > >Goal: > > >Capacity management, optimisation and automation > > >Key Features: > > >1. Quota management > > >2. Storage Cleanup and GC > > >3. Sampling and Estimation > > >4. Cube optimization suggestion > > >5. Automate housekeeping jobs > > >Target release date > > >Q2 2015 > > > > > >v0.7.5: DR and Security > > >Goal: > > >Support DR and enhance security > > >Key Features: > > >1. DR support with DR Cluster > > >2. Security enhancement > > >Target release date > > >TBD > > > > > >v0.8.x: Streaming and Real-Time (StreamingOLAP) > > >Goal: > > >Streaming cube building process > > >Hybrid Historical and Real Time analysis > > >Key Features: > > >1. Streaming Cube Builder > > >2. Hybrid Query Router > > >3. Streaming REST API > > >Target release date > > >Q2 2015 > > > > > >v0.9.x: Spark Support > > >Goal: > > >Leverage Spark tech stack like SparkSQL, Spark Cube Builder > > >Key Features: > > >1. Spark Cube Builder > > >2. TBD > > >Target release date > > >TBD > > > > > > > > >v1.0: > > >Goal: > > >Release V1.0 based on community feedback from above release. > > >Key Features: > > >1. Solid APIs > > >2. Solid metadata structure > > >3. Easy of use and development > > >Target release date > > >TBD > > > > > > > > > > > > > > > > > >2015-01-13 22:11 GMT+08:00 Luke Han <[email protected]>: > > > > > >> I already post in Google Group to encourage people subscript this > > >>mailing > > >> list. > > >> Once more and more people coming to ask about user relative questions, > > >>it > > >> will be good time to create user@ list, i think. > > >> > > >> Thanks. > > >> > > >> 2015-01-13 7:04 GMT+08:00 Henry Saputra <[email protected]>: > > >> > > >>> I really hope that people whom would like to participate in the dev > of > > >>> Kylin should subscribe to dev@ list. > > >>> > > >>> Other than that, if a lot of traffic happening about how to use or > > >>> trouble shoot Kylin, maybe we could start adding user@ list. > > >>> > > >>> - Henry > > >>> > > >>> On Mon, Jan 12, 2015 at 2:10 PM, Adunuthula, Seshu > > >>><[email protected]> > > >>> wrote: > > >>> >> I hope Ted's suggestion did not fall to deaf ears. We do not need > > >>>to cc > > >>> >>or > > >>> >> plus anybody for conversations in dev@ list. Julian is already > > >>> subscribed > > >>> >> to this list. > > >>> > > > >>> > > > >>> > Understood! > > >>> > > > >>> > What are your thoughts on including folks into discussions that do > > >>>not > > >>> > fall under > > >>> > The umbrella of dev > > >>> > > > >>> >> And diversity should involve individuals not just from MapR, > > >>>Cloudera, > > >>> or > > >>> >> HW. > > >>> > > > >>> > Couldn’t agree more with this. I know Luke is actively working on > > >>> this... > > >>> > > > >>> > > > >>> > > > >>> > Regards > > >>> > Seshu > > >>> > > > >>> > On 1/12/15, 1:34 PM, "Henry Saputra" <[email protected]> > > wrote: > > >>> > > > >>> >>I hope Ted's suggestion did not fall to deaf ears. We do not need > to > > >>>cc > > >>> or > > >>> >>plus anybody for conversations in dev@ list. Julian is already > > >>> subscribed > > >>> >>to this list. > > >>> >> > > >>> >>And diversity should involve individuals not just from MapR, > > >>>Cloudera, > > >>> or > > >>> >>HW. > > >>> >> > > >>> >>- Henry > > >>> >> > > >>> >>On Monday, January 12, 2015, Manjunatha Rao <[email protected] > > > > >>> wrote: > > >>> >> > > >>> >>> + Julian > > >>> >>> > > >>> >>> On 1/12/15, 12:58 PM, "Adunuthula, Seshu" <[email protected] > > >>> >>> <javascript:;>> wrote: > > >>> >>> > > >>> >>> >On Diversity goal for Kylin would like to explicitly call out > > >>>MapR¹s > > >>> >>> >Involvement and commitment for Kylin and Drill Integration and > > >>> >>> >specifically > > >>> >>> >Ted¹s contribution on guiding Kylin with the Apache Way. > > >>> >>> > > > >>> >>> >Hoping to see similar commitment from Hortonworks and Cloudera. > > >>> >>> > > > >>> >>> >On 1/12/15, 8:07 AM, "Adunuthula, Seshu" <[email protected] > > >>> >>> <javascript:;>> wrote: > > >>> >>> > > > >>> >>> >>Luke, > > >>> >>> >> > > >>> >>> >>Some thoughts before we tag Kylin as V1.0. We should spend some > > >>>time > > >>> >>>on > > >>> >>> >>the fundamentals of Kylin > > >>> >>> >> > > >>> >>> >>1. Availability: Build out the DR capability in Kylin. We have > an > > >>> >>>option > > >>> >>> >>to DR both the offline Cube > > >>> >>> >>Build/Refreshes or only the Cube Store (HBase). The cheaper > > >>>choice > > >>> >>>would > > >>> >>> >>be to DR the Cube Store (HBase) > > >>> >>> >>By creating a Async Replication Instance of HBase on a > different > > >>> Colo. > > >>> >>> >> > > >>> >>> >>2. Security: Currently the Kylin Cube security model inherits > > >>>from > > >>> the > > >>> >>> >>underlying Hive metadata. This > > >>> >>> >>Needs enhancing if we plan to get data from sources other than > > >>>Hive. > > >>> >>> >>Kylin > > >>> >>> >>should have its own security > > >>> >>> >>Model independent of Hive and we should consider integration > with > > >>> the > > >>> >>> >>security solutions out there > > >>> >>> >>Knox/Sentry/Ranger etc. > > >>> >>> >> > > >>> >>> >>3. Performance: We have internal performance numbers, but would > > >>>like > > >>> >>>them > > >>> >>> >>validated with externally > > >>> >>> >>Recognizable benchmarks such as TPC-DS. > > >>> >>> >> > > >>> >>> >>4. Diversity: I would like us to focus on diversity of > committers > > >>> and > > >>> >>> >>deployments as the final criteria > > >>> >>> >>For the V1 release. > > >>> >>> >> > > >>> >>> >>Ron and Manju, > > >>> >>> >> > > >>> >>> >>Would like commitment from Cloudera and Hortonworks towards > > >>>moving > > >>> the > > >>> >>> >>needle on Kylin especially > > >>> >>> >>On Fundamentals such as Security and Performance. > > >>> >>> >> > > >>> >>> >>Regards > > >>> >>> >>Seshu Adunuthula > > >>> >>> >> > > >>> >>> >> > > >>> >>> >>PS: How come Apache DL emails are still in the 90s. Some > > >>>formatting > > >>> in > > >>> >>> >>the > > >>> >>> >>emails would be nice :-) > > >>> >>> >> > > >>> >>> >> > > >>> >>> >>On 1/11/15, 6:52 AM, "Luke Han" <[email protected] > > >>><javascript:;>> > > >>> >>> wrote: > > >>> >>> >> > > >>> >>> >>>After open sourced, there are a lot conversion with customers, > > >>> >>>partners > > >>> >>> >>>and > > >>> >>> >>>also received many feedback from external, community and > others. > > >>> >>>Combine > > >>> >>> >>>all information we have and considering our bandwidth so far, > we > > >>> >>>drafted > > >>> >>> >>>this release plan as below: > > >>> >>> >>> > > >>> >>> >>> > > >>> https://www.dropbox.com/sc/nh90bniy7szyvta/AACuv2O3pO1Fog6RwQFfr6YIa > > >>> >>> >>> > > >>> >>> >>>Here are summary: > > >>> >>> >>> > > >>> >>> >>> - v1.0 (MOLAP), current version to be released with more > > >>>stable > > >>> >>>and > > >>> >>> >>>bug > > >>> >>> >>> fixed. So far there's no majority issue. Target release > date: > > >>> >>> >>>2015-01-15 > > >>> >>> >>> - v1.1, release via Apache process, target release date: > > >>> >>>2015-01-31 > > >>> >>> >>> - v1.x, bug fix and enhance, soon be merged to HOLAP > > >>> >>> >>> - v2.0 (HOLAP), introduced InvertedIndex, since there are > > >>> totally > > >>> >>> >>> different design and concept, we name this release as 2.0, > > >>> target > > >>> >>> >>>release > > >>> >>> >>> date: 2015-03-31 > > >>> >>> >>> - v2.3, Capacity management and Automation > > >>> >>> >>> - v2.5 (Streaming Cube Builder), introduce Streaming Cube > > >>> Builder > > >>> >>> >>> capability in this release > > >>> >>> >>> - v2.x, Spark & SparkSQL support, target release date: > > >>> 2015-06-30 > > >>> >>> >>> > > >>> >>> >>> > > >>> >>> >>>Please let's know if there any problem and reply here to > > >>>discuss. > > >>> >>> >>> > > >>> >>> >>>Thank you very much > > >>> >>> >>> > > >>> >>> >>>Luke > > >>> >>> >> > > >>> >>> > > > >>> >>> > > >>> >>> > > >>> > > > >>> > > >> > > >> > > > > > > > -- > > Best Regards! > --------------------- > > Luke Han >
