Already tried setting spark.driver.allowMultipleContexts to true. But it
not successful. I the problem is we have different test suites which of
course run in parallel. How do we stop sparkContext after each test suite
and start it in the next test suite or is there any way to share
sparkContext ac
Priya,
If you're trying to get unit tests running local spark contexts, you can
just set up your spark context with 'spark.driver.allowMultipleContexts'
set to true.
Example:
def create(seconds : Int, appName : String): StreamingContext = {
val master = "local[*]"
val conf = new SparkConf().
Are you trying to speed up tests where each test suite uses single SparkContext
?
You may want to read:
https://issues.apache.org/jira/browse/SPARK-2243
Cheers
On Wed, Nov 4, 2015 at 4:59 AM, Priya Ch
wrote:
> Hello All,
>
> How to use multiple Spark Context in executing multiple test suite
Hello All,
How to use multiple Spark Context in executing multiple test suite of
spark code ???
Can some one throw light on this ?