Hi Ze, from what you mentioned, it seems there is no dictionary encoding and pre-calculation (cube build), is it? How about the query performance and storage space on ES from your experiments? Thanks for the info.
2016-01-28 10:03 GMT+08:00 zeLiu <[email protected]>: > hi,shaofeng > I am trying to review the code of 2.x,and Compared with 1.x, the code of > 2.x > change is very big. > this version is just implemented functional,There are a lot of place needs > to be optimized in the performance and architecture. > > > Because I do not know how to modify the webapp code,So if the description > of > Cube contains "ES", then it is a ES task > build job has four step:"Create Intermediate Flat Hive Table"---->"Bulk > Index and Load Data to ES"---->"Update Cube Info"---->"Garbage Collection" > "Bulk Index and Load Data to ES" is a new mapreduce job which use to > mapping > and bulk index . > the central class is "org.apache.kylin.job.hadoop.cube.BulkESMapper" . > > The query is based on SQLDigest,and return a ITupleIterator. > when the description of Cube contains "ES" ,the "StorageEngineFactory" > will > return > "org.apache.kylin.storage.elasticsearch.ElasticSearchStorageEngine". > if there are no aggregate,excute > > "org.apache.kylin.storage.elasticsearch.SerializedElasticSearchTupleIterator" > ,Otherwise, excute > > "org.apache.kylin.storage.elasticsearch.SerializedElasticAggregationTupleIterator" > > the count() function is blocked before the SQLDigest,so it does not support > count() now. > > thanks! > > -- > View this message in context: > http://apache-kylin.74782.x6.nabble.com/elastic-search-as-kylin-storage-engine-tp3429p3470.html > Sent from the Apache Kylin mailing list archive at Nabble.com. > -- Best regards, Shaofeng Shi
