[DOC] Add kafka example in StreamSQL doc This closes #2769
Project: http://git-wip-us.apache.org/repos/asf/carbondata/repo Commit: http://git-wip-us.apache.org/repos/asf/carbondata/commit/629d6254 Tree: http://git-wip-us.apache.org/repos/asf/carbondata/tree/629d6254 Diff: http://git-wip-us.apache.org/repos/asf/carbondata/diff/629d6254 Branch: refs/heads/branch-1.5 Commit: 629d62549860e037c2816702564713ce696419d5 Parents: 54bcf49 Author: Jacky Li <[email protected]> Authored: Wed Sep 26 21:56:52 2018 +0800 Committer: QiangCai <[email protected]> Committed: Fri Sep 28 11:52:41 2018 +0800 ---------------------------------------------------------------------- docs/streaming-guide.md | 33 +++++++++++++++++++++++++++------ 1 file changed, 27 insertions(+), 6 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/carbondata/blob/629d6254/docs/streaming-guide.md ---------------------------------------------------------------------- diff --git a/docs/streaming-guide.md b/docs/streaming-guide.md index 3b71662..56e400e 100644 --- a/docs/streaming-guide.md +++ b/docs/streaming-guide.md @@ -304,8 +304,9 @@ Following example shows how to start a streaming ingest job | register TIMESTAMP, | updated TIMESTAMP |) - |STORED BY carbondata + |STORED AS carbondata |TBLPROPERTIES ( + | 'streaming'='source', | 'format'='csv', | 'path'='$csvDataDir' |) @@ -324,7 +325,7 @@ Following example shows how to start a streaming ingest job | register TIMESTAMP, | updated TIMESTAMP |) - |STORED BY carbondata + |STORED AS carbondata |TBLPROPERTIES ( | 'streaming'='true' |) @@ -378,11 +379,14 @@ When this is issued, carbon will start a structured streaming job to do the stre name STRING, age INT ) - STORED BY carbondata + STORED AS carbondata TBLPROPERTIES( - 'format'='socket', - 'host'='localhost', - 'port'='8888' + 'streaming'='source', + 'format'='socket', + 'host'='localhost', + 'port'='8888', + 'record_format'='csv', // can be csv or json, default is csv + 'delimiter'='|' ) ``` @@ -394,6 +398,7 @@ When this is issued, carbon will start a structured streaming job to do the stre .format("socket") .option("host", "localhost") .option("port", "8888") + .option("delimiter", "|") ``` @@ -402,6 +407,22 @@ When this is issued, carbon will start a structured streaming job to do the stre - In the given STMPROPERTIES, user must specify `'trigger'`, its value must be `ProcessingTime` (In future, other value will be supported). User should also specify interval value for the streaming job. - If the schema specifid in sink table is different from CTAS, the streaming job will fail +For Kafka data source, create the source table by: + ```SQL + CREATE TABLE source( + name STRING, + age INT + ) + STORED AS carbondata + TBLPROPERTIES( + 'streaming'='source', + 'format'='kafka', + 'kafka.bootstrap.servers'='kafkaserver:9092', + 'subscribe'='test' + 'record_format'='csv', // can be csv or json, default is csv + 'delimiter'='|' + ) + ``` ##### STOP STREAM
