[
https://issues.apache.org/jira/browse/TOREE-260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15149516#comment-15149516
]
Kapil Malik commented on TOREE-260:
-----------------------------------
Here's my full kernel.json for reference (I am using a Spark EMR cluster).
{
"display_name": "Spark",
"language_info": { "name": "scala" },
"argv": [
"/home/hadoop/incubator-toree/dist/toree/bin/run.sh",
"--stdin-port",
"35153",
"--control-port",
"43870",
"--heartbeat-port",
"33136",
"--shell-port",
"48798",
"--iopub-port",
"45141",
"--profile",
"{connection_file}"
],
"codemirror_mode": "scala",
"env": {
"SPARK_OPTS": "--master=yarn-client --conf
spark.hadoop.fs.s3n.impl=org.apache.hadoop.fs.s3native.NativeS3FileSystem
--conf spark.hadoop.fs.s3.impl=org.apache.hadoop.fs.s3.S3FileSystem
--driver-java-options=-Xms1024M --driver-java-options=-Xmx2048M
--driver-java-options=-Dlog4j.logLevel=info --conf spark.cores.max=3 --conf
spark.driver.memory=1g --conf spark.executor.memory=2g",
"MAX_INTERPRETER_THREADS": "4",
"CAPTURE_STANDARD_OUT": "true",
"CAPTURE_STANDARD_ERR": "true",
"SEND_EMPTY_OUTPUT": "false",
"SPARK_HOME": "/usr/lib/spark",
"PYTHONPATH":
"/usr/lib/spark/python:/usr/lib/spark/python/lib/py4j-0.9-src.zip"
}
}
> Using same spark context for multiple notebooks
> -----------------------------------------------
>
> Key: TOREE-260
> URL: https://issues.apache.org/jira/browse/TOREE-260
> Project: TOREE
> Issue Type: Wish
> Reporter: Kapil Malik
>
> Hi,
> We are using toree with Jupyter and have a pressing requirement to use same
> spark context for multiple notebooks.
> I know this has been referred in
> https://issues.apache.org/jira/browse/TOREE-211 as well, but I am not clear
> whether this has been resolved, and if yes, how to go about achieving this.
> I am ready to spend active development time on this, but would need some
> shepherding around where to look.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
