[
https://issues.apache.org/jira/browse/KYLIN-3626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI reassigned KYLIN-3626:
-----------------------------------
Assignee: Chao Long
Component/s: Metadata
> Allow customization for Parquet path
> ------------------------------------
>
> Key: KYLIN-3626
> URL: https://issues.apache.org/jira/browse/KYLIN-3626
> Project: Kylin
> Issue Type: Sub-task
> Components: Metadata
> Reporter: Shaofeng SHI
> Assignee: Chao Long
> Priority: Major
> Fix For: v2.6.0
>
>
> Due to the difference of HDFS and object store like S3/blob store, Kylin
> needs to allow different path patterns for the files when putting to the
> underlying storage.
>
> For example, to HDFS, this pattern is good:
> /kylin/metadata_name/cube_name/segment_name/file1.parquet
>
> But, the above pattern is not good for S3, since S3 uses the first couple
> bytes of the path to do the hashing, so we'd better add salt ahead of the
> original pathname:
> /salt/kylin/metadata_name/cube_name/segment_name/file1.parquet
> or do a reverse order:
> /segment_name/cube_name/metadata_name/kylin/file1.parquet
>
> To get the flexibility, we should make this extensible.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
