Kyle Can you try connecting to that nifi port using telnet and see if you are able?
Use the same host and port as you are in your spark job. Thanks Joe On Feb 20, 2016 4:55 PM, "Kyle Burke" <[email protected]> wrote: > All, > I’m attempting to connect Spark to Nifi but I’m getting a “connect > timed out” error when spark tries to pull records from the input port. I > don’t understand why I”m getting the issue because nifi and spark are both > running on my local laptop. Any suggestions about how to get around the > issue? > > *It appears that nifi is listening on the port because I see the following > when running the lsof command:* > > java 31455 kyle.burke 1054u IPv4 0x1024ddd67a640091 0t0 TCP > *:9099 (LISTEN) > > > *I’ve been following the instructions give in these two articles:* > https://blogs.apache.org/nifi/entry/stream_processing_nifi_and_spark > > https://community.hortonworks.com/articles/12708/nifi-feeding-data-to-spark-streaming.html > > *Here is how I have my nifi.properties setting:* > > # Site to Site properties > > nifi.remote.input.socket.host= > > nifi.remote.input.socket.port=9099 > > nifi.remote.input.secure=false > > > *Below is the full error stack:* > > 16/02/20 16:34:45 ERROR Executor: Exception in task 0.0 in stage 0.0 (TID > 0) > > java.net.SocketTimeoutException: connect timed out > > at java.net.PlainSocketImpl.socketConnect(Native Method) > > at > java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) > > at > java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) > > at > java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) > > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) > > at java.net.Socket.connect(Socket.java:589) > > at sun.net.NetworkClient.doConnect(NetworkClient.java:175) > > at sun.net.www.http.HttpClient.openServer(HttpClient.java:432) > > at sun.net.www.http.HttpClient.openServer(HttpClient.java:527) > > at sun.net.www.http.HttpClient.<init>(HttpClient.java:211) > > at sun.net.www.http.HttpClient.New(HttpClient.java:308) > > at sun.net.www.http.HttpClient.New(HttpClient.java:326) > > at > sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1168) > > at > sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1104) > > at > sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:998) > > at > sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:932) > > at org.apache.spark.util.Utils$.doFetchFile(Utils.scala:555) > > at org.apache.spark.util.Utils$.fetchFile(Utils.scala:369) > > at > org.apache.spark.executor.Executor$$anonfun$org$apache$spark$executor$Executor$$updateDependencies$5.apply(Executor.scala:405) > > at > org.apache.spark.executor.Executor$$anonfun$org$apache$spark$executor$Executor$$updateDependencies$5.apply(Executor.scala:397) > > at > scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scala:772) > > at > scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:98) > > at > scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:98) > > at > scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:226) > > at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:39) > > at scala.collection.mutable.HashMap.foreach(HashMap.scala:98) > > at > scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:771) > > at org.apache.spark.executor.Executor.org > $apache$spark$executor$Executor$$updateDependencies(Executor.scala:397) > > at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:193) > > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > > at java.lang.Thread.run(Thread.java:745) > > > Respectfully, > > *Kyle Burke *| Data Science Engineer > *IgnitionOne - *Marketing Technology. Simplified. >
