[
https://issues.apache.org/jira/browse/TAJO-1614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14549767#comment-14549767
]
Hyunsik Choi commented on TAJO-1614:
------------------------------------
Hi [~jihoonson],
> Is this able to be difined for storage and file types?
Yes, it can. If a user implements their own storage and file type, they can
customize this file too. I'll embed some default config file to Tajo source and
also allows users to specific their own file, which overrides the default
config.
> do you have any plans to provide customizable properties or only predefined
> ones?
Except for some common properties like timezone, I'll move most predefined
properties to custom properties.
> Configuration format proposal for generic storage support
> ---------------------------------------------------------
>
> Key: TAJO-1614
> URL: https://issues.apache.org/jira/browse/TAJO-1614
> Project: Tajo
> Issue Type: Sub-task
> Components: storage
> Reporter: Hyunsik Choi
> Assignee: Hyunsik Choi
> Fix For: 0.11.0
>
>
> This is an example of the first draft of new storage configuration file for
> generic storage. It allows users to specify:
> * table spaces
> * space handlers
> * data formats
> * relationships among them
> More detailed explanation is as follows:
> * {{/storages}} specifies all storage types.
> * {{/storages/*}} specifies which handler deals with which uri scheme.
> * {{/formats}} can specify each data format or file format
> * {{/formats/*}} specifies which format is available in which storage
> types.
> * {{/formats/*/handler}} specifies a handler class to allows clients to
> get scanner or appender.
> {code}
> {
> "spaces": {
> "default": {
> "uri": "hdfs://localhost:8020/tajo/warehouse",
> "configs": [
> {"x": "y"},
> {"x": "y"}
> ]
> },
> "ssd1": {
> "uri": "hdfs://localhost:8020/....",
> "configs": [
> {"x": "y"},
> {"x": "y"}
> ]
> },
> "mysql": {
> "uri": "jdbc://....",
> "configs": [
> {"x": "y"},
> {"x": "y"}
> ]
> }
> },
>
> "storages": {
> "hdfs": {
> "handler": "org.apache.tajo.storage.HdfsTablespace",
> "default-format": "text"
> },
> "file": {
> "handler": "org.apache.tajo.storage.FileTablespace",
> "default-format": "text"
> },
> "hbase": {
> "handler": "org.apache.tajo.storage.hbase.HBaseTablespace",
> "default-format": "hbase"
> },
> "s3": {
> "handler": "org.apache.tajo.storage.S3Tablespace",
> "default-format": "text"
> },
> "jdbc": {
> "handler": "org.apache.tajo.storage.JdbcTablespace",
> "default-format": "rowstore"
> }
> },
>
> "formats": {
>
> "avro": {
> "storage-support": ["hdfs", "file", "s3"],
> "handler": "org.apache.tajo.storage.AvroHandler"
> },
> "csv": {
> "storage-support": ["hdfs", "file", "s3"],
> "handler": "org.apache.tajo.storage.CSVHandler"
> },
> "text": {
> "storage-support": ["hdfs", "file", "s3"],
> "handler": "org.apache.tajo.storage.TextHandler"
> },
> "parquet": {
> "storage-support": ["hdfs", "file", "s3"],
> "handler": "org.apache.tajo.storage.ParquetHandler"
> },
> "raw": {
> "storage-support": ["hdfs", "file", "s3"],
> "handler": "org.apache.tajo.storage.RawHandler"
> },
> "rcfile": {
> "storage-support": ["hdfs", "file", "s3"],
> "handler": "org.apache.tajo.storage.RCFileHandler"
> },
> "row": {
> "storage-support": ["hdfs", "file", "s3"],
> "handler": "org.apache.tajo.storage.RowHandler"
> },
> "sequencefile": {
> "storage-support": ["hdfs", "file", "s3"],
> "handler": "org.apache.tajo.storage.SeqFileHandler"
> },
>
>
> "hbase": {
> "storage-support": ["hbase"],
> "handler": "org.apache.tajo.storage.HbaseHandler"
> },
> "kyro": {
> "storage-support": ["hbase"],
> "handler": "org.apache.tajo.storage.KyroHandler"
> },
>
> "rowstore": {
> "storage-support": ["jdbc"],
> "handler": "org.apache.tajo.storage.RowStoreHandler"
> }
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
