Thanks, Aaron.
Your advice really works.

Does this mean that the collect() method pulls all related data from slave 
nodes to master node?




    Sincerely

Yang, Qiang

发件人: Aaron Davidson
发送时间: 2013年11月19日(星期二) 下午12:38
收件人: user; yangqiang
主题: Re: Can not get the expected output when running the BroadcastTestexample 
program.
Assuming your cluster is actually working (e.g., other examples like SparkPi 
work), then the problem is probably that println() doesn't actually write 
output back to the driver; instead, it may just be outputting locally to each 
slave. You can test this by replacing lines 43 through 45 with:


  sc.parallelize(1 to 10, slices).map {
    i => barr1.value.size
  }.collect().foreach(i => println(i))


which should gather the exact same data but ensure that the printlns actually 
occur on the driver.

Reply via email to