Stdout will not be sent back to driver, no matter you use Scala or Java.
You must do something wrongly that makes you think it is an expected
behavior.
On Mon, Dec 28, 2015 at 5:33 PM, David John <david_john_2...@outlook.com>
wrote:
> I have used Spark *1.4* for 6 months. Thanks all the members of this
> community for your great work.
> I have a question about the logging issue. I hope this question can be
> solved.
>
> The program is running under this configurations:
> YARN Cluster,
> YARN-client mode.
>
> In *Scala*,
> writing a code like:
> *rdd.map( a => println(a) ); *
> will get the output about the value of a in our console.
>
> However,
> in *Java (1.7)*,
> writing
> rdd.map( new Function<Integer,Integer>(){
> @Override
> public Integer call(Integer a) throws Exception {
> *System.out.println(a);*
> }
> });
> won't get the output in our console.
>
> The configuration is the same.
>
> I have try this code but not work either:
> rdd.map( new Function<Integer,Integer>(){
> @Override
> public Integer call(Integer a) throws Exception {
> org.apache.log4j.Logger log =
> Logger.getLogger(this.getClass());
> log.info(a);
> log.warn(a);
> log.error(a);
> log.fatal(a);
> }
> });
>
> No output either:
> final org.apache.log4j.Logger log = Logger.getLogger(this.getClass());
> rdd.map( new Function<Integer,Integer>(){
> @Override
> public Integer call(Integer a) throws Exception {
> log.info(a);
> log.warn(a);
> log.error(a);
> log.fatal(a);
> }
> });
>
> It seems that the output of stdout in worker doesn't send the output back
> to our driver.
> I am wonder why it works in scala but not in java.
> Is there a simple way to make java work like scala?
>
> Thanks.
>
