On 16 July 2016 at 03:23, ckadner <[email protected]> wrote:
> Github user ckadner commented on the issue: > > https://github.com/apache/bahir/pull/10 > > **How to test these changes:** > > (1) Test that the example `mqtt_wordcount.py` is running fine: > ```console > export SPARK_HOME="~/Runtimes/spark/spark-2.0.0-preview-bin-hadoop2.7" > export PYTHONPATH="${BAHIR_HOME}/streaming-mqtt/python" > > cd ~/Projects/bahir > mvn clean install -pl streaming-mqtt > > ${SPARK_HOME}/bin/spark-submit \ > --packages > org.apache.bahir:spark-streaming-mqtt_2.11:2.0.1-SNAPSHOT \ > > streaming-mqtt/examples/src/main/python/streaming/mqtt_wordcount.py \ > tcp://localhost:1883 foo > ``` > ``` > ... > 16/07/15 19:16:47 WARN AbstractHandler: No Server set for > org.spark_project.jetty.server.handler.ErrorHandler@1e8d73e4 > > /Users/ckadner/Runtimes/spark/spark-2.0.0-preview-bin-hadoop2.7/python/lib/pyspark.zip/pyspark/shuffle.py:58: > UserWarning: Please install psutil to have better support with spilling > ------------------------------------------- > Time: 2016-07-15 19:16:49 > ------------------------------------------- > > ------------------------------------------- > Time: 2016-07-15 19:16:50 > ------------------------------------------- > > ------------------------------------------- > Time: 2016-07-15 19:16:51 > ------------------------------------------- > ... > ``` > > (2) Test the `bin/run-example` script: > ```console > export SPARK_HOME="~/Runtimes/spark/spark-2.0.0-preview-bin-hadoop2.7" > > cd ~/Projects/bahir > # mvn clean install -pl streaming-mqtt > > bin/run-example \ > > streaming-mqtt/examples/src/main/python/streaming/mqtt_wordcount.py \ > tcp://localhost:1883 foo > ``` > ``` > ... > 16/07/15 19:21:19 WARN AbstractHandler: No Server set for > org.spark_project.jetty.server.handler.ErrorHandler@64bc10e5 > > /Users/ckadner/Runtimes/spark/spark-2.0.0-preview-bin-hadoop2.7/python/lib/pyspark.zip/pyspark/shuffle.py:58: > UserWarning: Please install psutil to have better support with spilling > ------------------------------------------- > Time: 2016-07-15 19:21:21 > ------------------------------------------- > > ------------------------------------------- > Time: 2016-07-15 19:21:22 > ------------------------------------------- > > ------------------------------------------- > Time: 2016-07-15 19:21:23 > ------------------------------------------- > > ^Z > [1]+ Stopped bin/run-example > streaming-mqtt/examples/src/main/python/streaming/mqtt_wordcount.py > tcp://localhost:1883 foo > ... > ``` > > > (3) Test the error message contains the correct Spark version and > Scala version when Bahir MQTT package is missing or was not resolved: > ```console > BAHIR_HOME="~/Projects/bahir" > export SPARK_HOME="~/Runtimes/spark/spark-2.0.0-preview-bin-hadoop2.7" > export PYTHONPATH="${BAHIR_HOME}/streaming-mqtt/python" > > ${SPARK_HOME}/bin/spark-submit \ > > ${BAHIR_HOME}/streaming-mqtt/examples/src/main/python/streaming/mqtt_wordcount.py > \ > tcp://localhost:1883 foo > ``` > ``` > > ________________________________________________________________________________________________ > > Spark Streaming's MQTT libraries not found in class path. Try one of > the following. > > 1. Include the MQTT library and its dependencies with in the > spark-submit command as > > ${SPARK_HOME}/bin/spark-submit --packages > org.apache.bahir:spark-streaming-mqtt_2.11:2.0.0 ... > > 2. Download the JAR of the artifact from Maven Central > http://search.maven.org/, > Group Id = org.apache.bahir, Artifact Id = spark-streaming-mqtt, > Version = 2.0.0. > Then, include the jar in the spark-submit command as > > ${SPARK_HOME}/bin/spark-submit --jars <spark-streaming-mqtt.jar> > ... > > ________________________________________________________________________________________________ > ``` > > > --- > If your project is set up for it, you can reply to this email and have your > reply appear on GitHub as well. If your project does not have this feature > enabled and wishes so, or if the feature is enabled but not working, please > contact infrastructure at [email protected] or file a JIRA ticket > with INFRA. > --- >
