Re: Running a task once on each executor

2014-08-11 Thread RodrigoB
from a user's perspective. Tnks, Rod -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Running-a-task-once-on-each-executor-tp3203p11908.html Sent from the Apache Spark User List mailing list archive at Nabbl

Re: Running a task once on each executor

2014-03-28 Thread dmpour23
/Running-a-task-once-on-each-executor-tp3203p3427.html Sent from the Apache Spark User List mailing list archive at Nabble.com.

Re: Running a task once on each executor

2014-03-27 Thread Christopher Nguyen
t; long as the RDD had atleast one partition on each executor > > Deenar > > > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Running-a-task-once-on-each-executor-tp3203p3393.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. >

Re: Running a task once on each executor

2014-03-27 Thread deenar.toraskar
rn, my function would be called only once per executor as long as the RDD had atleast one partition on each executor Deenar -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Running-a-task-once-on-each-executor-tp3203p3393.html Sent from the Apache Spark Use

Re: Running a task once on each executor

2014-03-27 Thread Christopher Nguyen
lues().mapPartitions(new > FlatMapFunction, String>() { > @Override > public Iterable call(Iterator arg0) throws Exception { >System.out.println("Usage should call my jar once: " + arg0); >return Lists.newArrayList();} > }); > > > > -- > View this messa

Re: Running a task once on each executor

2014-03-27 Thread dmpour23
st.1001560.n3.nabble.com/Running-a-task-once-on-each-executor-tp3203p3353.html Sent from the Apache Spark User List mailing list archive at Nabble.com.

Re: Running a task once on each executor

2014-03-27 Thread deenar.toraskar
ecutor rdd.mapPartitions would return multiple results. Deenar -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Running-a-task-once-on-each-executor-tp3203p3337.html Sent from the Apache Spark User List mailing list archive at Nabble.com.

Re: Running a task once on each executor

2014-03-25 Thread Christopher Nguyen
RDD to > ensure that there is one element per executor. > > Deenar > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Running-a-task-once-on-each-executor-tp3203p3208.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. >

Re: Running a task once on each executor

2014-03-25 Thread deenar.toraskar
context: http://apache-spark-user-list.1001560.n3.nabble.com/Running-a-task-once-on-each-executor-tp3203p3208.html Sent from the Apache Spark User List mailing list archive at Nabble.com.

Re: Running a task once on each executor

2014-03-25 Thread Christopher Nguyen
mething similar, I just run a > map on a large RDD that is hash partitioned, this does not guarantee that > the job would run just once. > > Deenar > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Running-a-task-once-on-each-executor-tp3203.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. >

Running a task once on each executor

2014-03-25 Thread deenar.toraskar
context: http://apache-spark-user-list.1001560.n3.nabble.com/Running-a-task-once-on-each-executor-tp3203.html Sent from the Apache Spark User List mailing list archive at Nabble.com.