Thanks, Aaron.
Your advice really works.
Does this mean that the collect() method pulls all related data from slave
nodes to master node?
Sincerely
Yang, Qiang
发件人: Aaron Davidson
发送时间: 2013年11月19日(星期二) 下午12:38
收件人: user; yangqiang
主题: Re: Can not get the expected output when running the BroadcastTestexample
program.
Assuming your cluster is actually working (e.g., other examples like SparkPi
work), then the problem is probably that println() doesn't actually write
output back to the driver; instead, it may just be outputting locally to each
slave. You can test this by replacing lines 43 through 45 with:
sc.parallelize(1 to 10, slices).map {
i => barr1.value.size
}.collect().foreach(i => println(i))
which should gather the exact same data but ensure that the printlns actually
occur on the driver.