It could be that the hostname that Spark uses to identify the node is different from the one you are providing. Are you using the Spark standalone mode? In that case, you can check out the hostnames that Spark is seeing and use that name.
Let me know if that works out. TD On Mon, Feb 17, 2014 at 12:36 PM, anoldbrain <anoldbr...@gmail.com> wrote: > FlumeInputDStream extends NetworkInputDStream, which runs on worker nodes. > This brings up another question. 'getLocationPreference' doesn't seem to > work like I expected it to be. It got called on driver noder, and yet the > returned 'host' value doesn't get honored. > > The behavior I observed is: > > 1. FlumeEventCount /master/ /workerA-ip/ 4141: bind failed, because the > actual worker node assigned the listen for FlumeInputDStream is not > workerA. > However, the designated worker node, say workerB, is always the same run > after run. > 2. FlumeEventCount /master/ localhost 4141: at each round, different worker > node listens on 4141 > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/How-to-use-FlumeInputDStream-in-spark-cluster-tp1604p1643.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. >
