This will be one of the most important checkins for Kylin. Looking to the community for a wide array of Storage and Data Source Plugins. We should add a document for developing, deploying and using these plugins.
Regards Seshu Adunuthula On 7/23/15, 7:43 PM, "Li Yang" <[email protected]> wrote: >Have finished first round of refactor. The work is on 0.8. > >The modules now look like: > >- core-common >- core-metadata >- core-dictionary >- core-cube >- core-job >- core-storage > >- engine-mr >- source-hive >- storage-hbase > >- storage (legacy) >- job (legacy) > >- query >- server > >The old "job" and "storage" remain to hold legacy classes, and should be >dropped in the end. Qianhao and Mahone will continue to refactor streaming >and storage code to fit in this new architecture. > >Cheers >Yang > > >On Wed, Jul 15, 2015 at 8:56 PM, Luke Han <[email protected]> wrote: > >> Would love to see the first version to work with SparkSQL;-) >> >> >> Best Regards! >> --------------------- >> >> Luke Han >> >> On Wed, Jul 15, 2015 at 6:04 PM, Li Yang <[email protected]> wrote: >> >> > I'm working on an effort to make Build Engine / Table Source / Storage >> more >> > plugin-able. The benefit is, for example, Spark can be an alternative >> build >> > engine in addition to MR, or use Spark RDD as table source, or >>implement >> > Cassandra as storage with minimal effort. >> > >> > https://issues.apache.org/jira/browse/KYLIN-875 >> > >> > The current source modules will be refactored. The future might look >>like >> > >> > - "core" module, merged from common, cube, dictionary, metadata, job >> > - "engine-mr" module, the default MapReduce build engine >> > - "source-hive" module, Hive table source >> > - "storage-hbase" module, HBase as default storage >> > - "query" & "server", as is >> > >> > Later alternative engine / source / storage can be added. >> > >> > - "engine-spark" >> > - "source-sparksql" >> > - "storage-cassandra" >> > >> > Any thoughts? The change will happen on 0.8 branch gradually. >> > >> > >> > Cheers >> > Yang >> > >>
