Hyunsik Choi created TAJO-1614:
----------------------------------
Summary: Configuration format proposal for generic storage support
Key: TAJO-1614
URL: https://issues.apache.org/jira/browse/TAJO-1614
Project: Tajo
Issue Type: Sub-task
Components: storage
Reporter: Hyunsik Choi
Assignee: Hyunsik Choi
Fix For: 0.11.0
This is an example of the first draft of new storage configuration file for
generic storage. It allows users to specify:
* table spaces
* space handlers
* data formats
* relationships among them
More detailed explanation is as follows:
* {{/storages}} specifies all storage types.
* {{/storages/*}} specifies which handler deals with which uri scheme.
* {{/formats}} can specify each data format or file format
* {{/formats/*}} specifies which format is available in which storage types.
* {{/formats/*/handler}} specifies a handler class to allows clients to get
scanner or appender.
{code}
{
"spaces": {
"default": {
"uri": "hdfs://localhost:8020/tajo/warehouse",
"configs": [
{"x": "y"},
{"x": "y"}
]
},
"ssd1": {
"uri": "hdfs://localhost:8020/....",
"configs": [
{"x": "y"},
{"x": "y"}
]
},
"mysql": {
"uri": "jdbc://....",
"configs": [
{"x": "y"},
{"x": "y"}
]
}
},
"storages": {
"hdfs": {
"handler": "org.apache.tajo.storage.HdfsTablespace",
"default-format": "text"
},
"file": {
"handler": "org.apache.tajo.storage.FileTablespace",
"default-format": "text"
},
"hbase": {
"handler": "org.apache.tajo.storage.hbase.HBaseTablespace",
"default-format": "hbase"
},
"s3": {
"handler": "org.apache.tajo.storage.S3Tablespace",
"default-format": "text"
},
"jdbc": {
"handler": "org.apache.tajo.storage.JdbcTablespace",
"default-format": "rowstore"
}
},
"formats": {
"avro": {
"storage-support": ["hdfs", "file", "s3"],
"handler": "org.apache.tajo.storage.AvroHandler"
},
"csv": {
"storage-support": ["hdfs", "file", "s3"],
"handler": "org.apache.tajo.storage.CSVHandler"
},
"text": {
"storage-support": ["hdfs", "file", "s3"],
"handler": "org.apache.tajo.storage.TextHandler"
},
"parquet": {
"storage-support": ["hdfs", "file", "s3"],
"handler": "org.apache.tajo.storage.ParquetHandler"
},
"raw": {
"storage-support": ["hdfs", "file", "s3"],
"handler": "org.apache.tajo.storage.RawHandler"
},
"rcfile": {
"storage-support": ["hdfs", "file", "s3"],
"handler": "org.apache.tajo.storage.RCFileHandler"
},
"row": {
"storage-support": ["hdfs", "file", "s3"],
"handler": "org.apache.tajo.storage.RowHandler"
},
"sequencefile": {
"storage-support": ["hdfs", "file", "s3"],
"handler": "org.apache.tajo.storage.SeqFileHandler"
},
"hbase": {
"storage-support": ["hbase"],
"handler": "org.apache.tajo.storage.HbaseHandler"
},
"kyro": {
"storage-support": ["hbase"],
"handler": "org.apache.tajo.storage.KyroHandler"
},
"rowstore": {
"storage-support": ["jdbc"],
"handler": "org.apache.tajo.storage.RowStoreHandler"
}
}
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
