sage in context: http://apache-spark-
> developers-list.1001551.n3.nabble.com/Launching-multiple-
> spark-jobs-within-a-main-spark-job-tp20311p20327.html
> Sent from the Apache Spark Developers List mailing list archive at
> Nabble.com.
>
> -
> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>
>
.nabble.com/Launching-multiple-spark-jobs-within-a-main-spark-job-tp20311p20327.html
Sent from the Apache Spark Developers List mailing list archive at Nabble.com.
-
To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
...@spark.apache.org
Subject: Launching multiple spark jobs within a main spark job.
Hi Team,
Is it ok to spawn multiple spark jobs within a main spark job, my main spark
job's driver which was launched on yarn cluster, will do some preprocessing and
based on it, it needs to launch multilple spark jobs
;
> *From:* Naveen [mailto:hadoopst...@gmail.com]
> *Sent:* Wednesday, December 21, 2016 9:18 AM
> *To:* dev@spark.apache.org; u...@spark.apache.org
> *Subject:* Launching multiple spark jobs within a main spark job.
>
>
>
> Hi Team,
>
>
>
> Is it ok to spawn multip
gt; Similarly, future2 to futureN.
>>>
>>> future1.onComplete{...}
>>> }
>>>
>>> }// end of mainsparkjob
>>> --
>>>
>>>
>>> [image: Inline image 1]
>>>
>>> On Wed, Dec 21, 2016 at 3:1
yway, If you run spark applicaction you would have multiple jobs, which
>> makes sense that it is not a problem.
>>
>>
>>
>> Thanks David.
>>
>>
>>
>> *From:* Naveen [mailto:hadoopst...@gmail.com]
>> *Sent:* Wednesday, December 21, 2016 9:18
/
--
View this message in context:
http://apache-spark-developers-list.1001551.n3.nabble.com/Launching-multiple-spark-jobs-within-a-main-spark-job-tp20311p20315.html
Sent from the Apache Spark Developers List mailing list archive at Nabble.com
rg; u...@spark.apache.org
> *Subject:* Launching multiple spark jobs within a main spark job.
>
>
>
> Hi Team,
>
>
>
> Is it ok to spawn multiple spark jobs within a main spark job, my main
> spark job's driver which was launched on yarn cluster, will do some
> preproc
this message in context:
http://apache-spark-developers-list.1001551.n3.nabble.com/Launching-multiple-spark-jobs-within-a-main-spark-job-tp20311p20312.html
Sent from the Apache Spark Developers List mailing list archive at Nabble.com
Hi Team,
Is it ok to spawn multiple spark jobs within a main spark job, my main
spark job's driver which was launched on yarn cluster, will do some
preprocessing and based on it, it needs to launch multilple spark jobs on
yarn cluster. Not sure if this right pattern.
Please share your thoughts.
10 matches
Mail list logo