Have finished first round of refactor. The work is on 0.8. The modules now look like:
- core-common - core-metadata - core-dictionary - core-cube - core-job - core-storage - engine-mr - source-hive - storage-hbase - storage (legacy) - job (legacy) - query - server The old "job" and "storage" remain to hold legacy classes, and should be dropped in the end. Qianhao and Mahone will continue to refactor streaming and storage code to fit in this new architecture. Cheers Yang On Wed, Jul 15, 2015 at 8:56 PM, Luke Han <[email protected]> wrote: > Would love to see the first version to work with SparkSQL;-) > > > Best Regards! > --------------------- > > Luke Han > > On Wed, Jul 15, 2015 at 6:04 PM, Li Yang <[email protected]> wrote: > > > I'm working on an effort to make Build Engine / Table Source / Storage > more > > plugin-able. The benefit is, for example, Spark can be an alternative > build > > engine in addition to MR, or use Spark RDD as table source, or implement > > Cassandra as storage with minimal effort. > > > > https://issues.apache.org/jira/browse/KYLIN-875 > > > > The current source modules will be refactored. The future might look like > > > > - "core" module, merged from common, cube, dictionary, metadata, job > > - "engine-mr" module, the default MapReduce build engine > > - "source-hive" module, Hive table source > > - "storage-hbase" module, HBase as default storage > > - "query" & "server", as is > > > > Later alternative engine / source / storage can be added. > > > > - "engine-spark" > > - "source-sparksql" > > - "storage-cassandra" > > > > Any thoughts? The change will happen on 0.8 branch gradually. > > > > > > Cheers > > Yang > > >
