Hi Tokayer,
Yes it works; I just verified that works, the only additional work is
copying one property from /etc/hbase/conf.dist/hbase-site.xml to
KYLIN_HOME/conf/kylin_job_conf.xml:
<property>
<name>hbase.zookeeper.quorum</name>
<value>ip-10-0-0-222.us-west-2.compute.internal</value>
</property>
Otherwise the "Convert to HFile" step will got zk connection error.
After built, the data will be loaded to S3 for HBase, e.g:
hbase(main):002:0> describe 'KYLIN_ROXUG026FC'
Table KYLIN_ROXUG026FC is ENABLED
KYLIN_ROXUG026FC, {TABLE_ATTRIBUTES => {coprocessor$1 =>
'hdfs://ip-10-0-0-222.us-west-2.compute.internal:8020/kylin/kylin_metadata/coprocessor/
kylin-coprocessor-2.0.0-0.jar|org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.CubeVisitService|1001|',
METADATA => {'CREATION_TIME'
=> '1496192721720', 'GIT_COMMIT' =>
'375fd807c281d8c5deff0620747c806be2019782;', 'KYLIN_HOST' =>
'kylin_metadata', 'OWNER' => '[email protected]
he.org', 'SEGMENT' => 'kylin_sales_cube[20120101000000_20170501000000]',
'SPLIT_POLICY' => 'org.apache.hadoop.hbase.regionserver.DisabledRegionS
plitPolicy'}}
[hadoop@ip-10-0-0-222 apache-kylin-2.0.0-bin]$ hadoop fs -ls
s3://kap-test-hbase-buket1/data/default/KYLIN_ROXUG026FC
Found 6 items
drwxrwxrwx - hadoop hadoop 0 1970-01-01 00:00
s3://kap-test-hbase-buket1/data/default/KYLIN_ROXUG026FC/.tabledesc
drwxrwxrwx - hadoop hadoop 0 1970-01-01 00:00
s3://kap-test-hbase-buket1/data/default/KYLIN_ROXUG026FC/12b468c028a2be8f6c781d6acf28d36b
drwxrwxrwx - hadoop hadoop 0 1970-01-01 00:00
s3://kap-test-hbase-buket1/data/default/KYLIN_ROXUG026FC/4fd1f27663a7216f8de833224d099ad4
drwxrwxrwx - hadoop hadoop 0 1970-01-01 00:00
s3://kap-test-hbase-buket1/data/default/KYLIN_ROXUG026FC/b655b7e35df6908d204bb3854e9eef0b
drwxrwxrwx - hadoop hadoop 0 1970-01-01 00:00
s3://kap-test-hbase-buket1/data/default/KYLIN_ROXUG026FC/e6ab10cd14cb5830eb80cda78dff7a61
drwxrwxrwx - hadoop hadoop 0 1970-01-01 00:00
s3://kap-test-hbase-buket1/data/default/KYLIN_ROXUG026FC/f31928db11f79dbd8961880d099f6e2b
2017-05-30 21:37 GMT+08:00 Tokayer, Jason M. <[email protected]>:
> Does Kylin support storing data in S3 rather than HDFS?
>
> ________________________________________________________
>
> The information contained in this e-mail is confidential and/or
> proprietary to Capital One and/or its affiliates and may only be used
> solely in performance of work or services for Capital One. The information
> transmitted herewith is intended only for use by the individual or entity
> to which it is addressed. If the reader of this message is not the intended
> recipient, you are hereby notified that any review, retransmission,
> dissemination, distribution, copying or other use of, or taking of any
> action in reliance upon this information is strictly prohibited. If you
> have received this communication in error, please contact the sender and
> delete the material from your computer.
>
--
Best regards,
Shaofeng Shi 史少锋
