[
https://issues.apache.org/jira/browse/TAJO-1388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14359532#comment-14359532
]
YeonSu Han edited comment on TAJO-1388 at 3/12/15 10:21 PM:
------------------------------------------------------------
Hi Jakob,
Thanks for your comments.
- I suppose that kafka message is only text format like delimited text line and
json.
- I'm newbie about kafka. So i take a simple approach for log compaction.
I do not use the compaction option about 'log.cleanup.policy'. Instead I use
delete opteion(log.cleanup.policy=delete). Or create unique key of message.
Samza is very interesting project to me.
was (Author: hys9958):
Hi Jakob,
Thanks for your comments.
- I suppose that kafka message is only text format like delimited text line and
json.
- I'm newbie about kafka. So i take a simple approach for log compaction.
I do not use the compaction option about 'log.cleanup.policy'. Or create unique
key of message.
Samza is very interesting project to me.
> [Umbrella] Kafka Storage Integration.
> -------------------------------------
>
> Key: TAJO-1388
> URL: https://issues.apache.org/jira/browse/TAJO-1388
> Project: Tajo
> Issue Type: New Feature
> Components: storage
> Reporter: YeonSu Han
> Assignee: YeonSu Han
> Labels: kafka_storage
> Attachments: Kafka _Storage_Ingegration_draft.pdf
>
>
> Apache Kafka is one of the widely used message queueing system. If we can use
> the Kafka as Tajo storage, analysis area of Tajo user is be broaden. For
> example, as realtime analysis.
> For this, I propose 'Kafka storage'. Please review my proposal and give your
> opinion.
> * Table Creation
> {code:sql}
> CREATE [EXTERNAL] TABLE [IF NOT EXISTS] <table_name> [(<column_name>
> <data_type>, ... )]
> using kafka with
> (‘kafka.topic’=’<kafka_topic_name>’,‘kafka.zk’=’<kafka_zookeeper_info>’,[other
> options])
> {code}
> ** Use “kafka” keyword in “using” clause for creating kafka table in Tajo.
> ** kafka table name is mapped to a Tajo table name with , 'kafka.topic'
> property.
> * Column mapping of kafka message
> ** Delimited line mapping (default)
> ** json mapping
> ** ...
> * Concept
> ** The topic of kafka correspond to table.
> ** The partition of kafka correspond to file.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
