Hi Andy, That's a feature -- you'll have to print out the return value from collect() if you want the contents to show up on stdout.
Probably something like this: for(Iterator<String> iter = rdd.pipe(pwd + "/src/main/bin/RDDPipe.sh").collect().iterator(); iter.hasNext();) System.out.println(iter.next()); Hope that helps, -Jey On Fri, Sep 19, 2014 at 11:21 AM, Andy Davidson <a...@santacruzintegration.com> wrote: > Hi > > I am wrote a little java job to try and figure out how RDD pipe works. > Bellow is my test shell script. If in the script I turn on debugging I get > output. In my console. If debugging is turned off in the shell script, I do > not see anything in my console. Is this a bug or feature? > > I am running the job locally on a Mac > > Thanks > > Andy > > > Here is my Java > > rdd.pipe(pwd + "/src/main/bin/RDDPipe.sh").collect(); > > > > #!/bin/sh > > > # > > # Use this shell script to figure out how spark RDD pipe() works > > # > > > set -x # turns shell debugging on > > #set +x # turns shell debugging off > > > while read x ; > > do > > echo RDDPipe.sh $x ; > > Done > > > > Here is the output if debugging is turned on > > $ !grep > > grep RDDPipe run.sh.out > > + echo RDDPipe.sh 0 > > + echo RDDPipe.sh 0 > > + echo RDDPipe.sh 2 > > + echo RDDPipe.sh 0 > > + echo RDDPipe.sh 3 > > + echo RDDPipe.sh 0 > > + echo RDDPipe.sh 0 > > $ --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
