Repository: sqoop Updated Branches: refs/heads/sqoop2 06f79093f -> 7d9f6c840
SQOOP-2287: Sqoop2: Create TO direction documentation (Abraham Elmahrek via Jarek Jarcec Cecho) Project: http://git-wip-us.apache.org/repos/asf/sqoop/repo Commit: http://git-wip-us.apache.org/repos/asf/sqoop/commit/7d9f6c84 Tree: http://git-wip-us.apache.org/repos/asf/sqoop/tree/7d9f6c84 Diff: http://git-wip-us.apache.org/repos/asf/sqoop/diff/7d9f6c84 Branch: refs/heads/sqoop2 Commit: 7d9f6c840e8077c303ad3c4d1d6f79eea46dd7b2 Parents: 06f7909 Author: Jarek Jarcec Cecho <[email protected]> Authored: Wed Apr 8 18:10:16 2015 -0700 Committer: Jarek Jarcec Cecho <[email protected]> Committed: Wed Apr 8 18:10:16 2015 -0700 ---------------------------------------------------------------------- docs/src/site/sphinx/Connectors.rst | 46 ++++++++++++++++++++++++++++++++ 1 file changed, 46 insertions(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/sqoop/blob/7d9f6c84/docs/src/site/sphinx/Connectors.rst ---------------------------------------------------------------------- diff --git a/docs/src/site/sphinx/Connectors.rst b/docs/src/site/sphinx/Connectors.rst index cc71ff5..26f3492 100644 --- a/docs/src/site/sphinx/Connectors.rst +++ b/docs/src/site/sphinx/Connectors.rst @@ -340,3 +340,49 @@ Destroyers ---------- The HDFS TO destroyer moves all created files to the proper output directory. + + ++++++++++++++++ +Kafka Connector ++++++++++++++++ + +Currently, only the TO direction is supported. + +----- +Usage +----- + +To use the Kafka Connector, create a link for the connector and a job that uses the link. + +**Link Configuration** +++++++++++++++++++++++ + +Inputs associated with the link configuration include: + ++----------------------+---------+-----------------------------------------------------------+-------------------------------------+ +| Input | Type | Description | Example | ++======================+=========+===========================================================+=====================================+ +| Broker list | String | Comma separated list of kafka brokers. | example.com:10000,example.com:11000 | +| | | *Required*. | | ++----------------------+---------+-----------------------------------------------------------+-------------------------------------+ +| Zookeeper connection | String | Comma separated list of zookeeper servers in your quorum. | /etc/conf/hadoop | +| | | *Required*. | | ++----------------------+---------+-----------------------------------------------------------+-------------------------------------+ + +**TO Job Configuration** +++++++++++++++++++++++++ + +Inputs associated with the Job configuration for the FROM direction include: + ++-------+---------+---------------------------------+----------+ +| Input | Type | Description | Example | ++=======+=========+=================================+==========+ +| topic | String | The Kafka topic to transfer to. | my topic | +| | | *Required*. | | ++-------+---------+---------------------------------+----------+ + +------ +Loader +------ + +During the *loading* phase, Kafka is written to directly from each loader. The order in which data is loaded into Kafka is not guaranteed.
