Ed,
Thanks for your interest!
You're definitely right - ClientConfig should be "config". I have updated the
code on the blog post. Thanks for pointing that out.
To include the Receiver in your application, you'll need the following
dependency in your application:
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-spark-receiver</artifactId>
<version>0.1.0-incubating</version>
</dependency>
Then, you should create an "uber jar" that you can deploy (in Maven it's
usually done with something like the following snippet):
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
You should then be able to use spark-deploy or whatever mechanism you usually
use to deploy your applications to Spark.
Thanks
-Mark
----------------------------------------
> From: [email protected]
> To: [email protected]
> Subject: NiFi and Spark Streaming
> Date: Tue, 26 May 2015 15:04:36 +0000
>
> Hi,
>
> I have been reading your blog:
> https://blogs.apache.org/nifi/entry/stream_processing_nifi_and_spark.
>
> This has been a useful resource for learning a new technology. So thanks for
> putting it together.
> Firstly I believe there is a typo. The code when creating the receiver passes
> in an object called "ClientConfig". However from the code above I believe it
> is meant to be simply "config".
>
> Secondly when trying to run your example on my own machine I am getting
> issues when submitting the job to Spark. It moans about not finding the
> "NiFiReceiver" class. How do I add this external dependency to spark?
>
> Thanks in advance for any help you can offer,
> Edd
>
>
> EDWARD SAVAGE | Software Engineer, Cyber | NORTHROP GRUMMAN
>
> Northrop Grumman
> Cyber Division
> 95, Promenade
> Cheltenham,
> GL50 1HH
>
> Email: [email protected]<mailto:[email protected]>
>
