Will Berkeley has posted comments on this change. ( http://gerrit.cloudera.org:8080/11788 )
Change subject: [examples] Add basic Spark example (scala) ...................................................................... Patch Set 11: (9 comments) http://gerrit.cloudera.org:8080/#/c/11788/11/examples/scala/spark-example/README.adoc File examples/scala/spark-example/README.adoc: http://gerrit.cloudera.org:8080/#/c/11788/11/examples/scala/spark-example/README.adoc@32 PS11, Line 32: compile and remove http://gerrit.cloudera.org:8080/#/c/11788/11/examples/scala/spark-example/README.adoc@33 PS11, Line 33: spark-example/ `spark-example` instead. http://gerrit.cloudera.org:8080/#/c/11788/11/examples/scala/spark-example/README.adoc@33 PS11, Line 33: a new : java executable a Spark application jar http://gerrit.cloudera.org:8080/#/c/11788/11/examples/scala/spark-example/README.adoc@34 PS11, Line 34: target/ `target` instead http://gerrit.cloudera.org:8080/#/c/11788/11/examples/scala/spark-example/README.adoc@34 PS11, Line 34: within in http://gerrit.cloudera.org:8080/#/c/11788/11/examples/scala/spark-example/README.adoc@34 PS11, Line 34: spark-example/ remove http://gerrit.cloudera.org:8080/#/c/11788/11/examples/scala/spark-example/README.adoc@41 PS11, Line 41: There are a few Java system properties defined in SparkExample.scala: : : - kuduMasters: A comma-separated list of Kudu master addresses. Defaults to : 'localhost:7051'. : - tableName: The name of the table you wish to create on the Kudu cluster. : Defaults to 'spark_test'. : : To run this as a spark2 'spark-submit' job, use the spark-submit command : as follows from the spark-example directory - it requires that you have built : and compiled the package using the 'mvn package' command above. You will also : need a Spark on YARN cluster and a Kudu cluster, both of which should be : resolvable and accessible from the host executing the command: I think we should redo this section a little bit. How do you like: To configure the kudu-spark example, there are two Java system properties available: - kuduMasters: a comma-separated list of Kudu master addresses. Default: localhost:7051. - tableName: the name of the table to use for the example program. This table should not exist in Kudu. Defaults to 'spark_test'. The application can be run using `spark-submit`. For example, to run the example against a Spark cluster running on YARN, use a command like the following: (long command i ain't typin) You will need the Kudu cluster to be up and running and Spark correctly configured for the example to work. http://gerrit.cloudera.org:8080/#/c/11788/11/examples/scala/spark-example/README.adoc@57 PS11, Line 57: <preferred deploy mode> What's the default? We want client mode so people will see the log messages showing the app running vs Kudu. If the default is client, just remove this param. http://gerrit.cloudera.org:8080/#/c/11788/11/examples/scala/spark-example/src/main/scala/org/apache/kudu/spark/examples/SparkExample.scala File examples/scala/spark-example/src/main/scala/org/apache/kudu/spark/examples/SparkExample.scala: http://gerrit.cloudera.org:8080/#/c/11788/11/examples/scala/spark-example/src/main/scala/org/apache/kudu/spark/examples/SparkExample.scala@65 PS11, Line 65: defaul default -- To view, visit http://gerrit.cloudera.org:8080/11788 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9ba09f0118c054a07b951e241c31d66245c57d3f Gerrit-Change-Number: 11788 Gerrit-PatchSet: 11 Gerrit-Owner: Mitch Barnett <mbarn...@cloudera.com> Gerrit-Reviewer: Adar Dembo <a...@cloudera.com> Gerrit-Reviewer: Attila Bukor <abu...@apache.org> Gerrit-Reviewer: Grant Henke <granthe...@apache.org> Gerrit-Reviewer: Greg Solovyev <gsolov...@cloudera.com> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Mitch Barnett <mbarn...@cloudera.com> Gerrit-Reviewer: Will Berkeley <wdberke...@gmail.com> Gerrit-Comment-Date: Fri, 02 Nov 2018 19:05:55 +0000 Gerrit-HasComments: Yes
