mcvsubbu commented on a change in pull request #3563: Re-org documentation URL: https://github.com/apache/incubator-pinot/pull/3563#discussion_r237264403
########## File path: docs/pinot_hadoop.rst ########## @@ -45,28 +49,109 @@ Create a job properties configuration file, such as one below: push.to.port=8888 -Index file creation -------------------- +Executing the job +^^^^^^^^^^^^^^^^^ The Pinot Hadoop module contains a job that you can incorporate into your -workflow to generate Pinot indices. Note that this will only create data for you. -In order to have this data on your cluster, you want to also run the SegmentTarPush -job, details below. To run SegmentCreation through the command line: +workflow to generate Pinot segments. -:: +.. code-block:: none mvn clean install -DskipTests -Pbuild-shaded-jar hadoop jar pinot-hadoop-0.016-shaded.jar SegmentCreation job.properties +You can then use the SegmentTarPush job to push segments via the controller REST API. -Index file push ---------------- - -This job takes generated Pinot index files from an input directory and pushes -them to a Pinot controller node. - -:: +.. code-block:: none - mvn clean install -DskipTests -Pbuild-shaded-jar hadoop jar pinot-hadoop-0.016-shaded.jar SegmentTarPush job.properties + +Creating Pinot segments outside of Hadoop +----------------------------------------- + +This document describes steps required for creating Pinot segments from standard formats like CSV/JSON. Review comment: perhaps so, we will get to it ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
