Hi,
I've just seen your post, and I have the same error.
Did you find a solution?
I can see that the error is :
<console>:14: error: not found: value sqlContext
Maybe it's a Livy problem.
Thanks.
On Monday, November 28, 2016 at 7:27:24 AM UTC+1, Vivek Suvarna wrote:
>
> Hi,
>
> I have Jupyter installed on my windows machine.
>
> I have Livy server 0.2 installed on the remote hadoop cluster where spark
> is also running.
> I can successfully connect to the cluster via Livy and execute a snippet
> of code on the cluster.
>
> I now want to connect via the notebook.
>
> I start livy on the remote cluster
> *./bin/livy-server*
>
> I start the notebook on my windows machine as follows
> *jupyter.exe notebook*
>
> This launches the jupyter browser and I select python 2.
>
>
>
>
> Once I select python 2 on the notebook I run the following
> *%load_ext sparkmagic.magics*
>
> then I run
> *%manage_spark*
>
> Here I enter the endpoint which is the same as the remote server where
> Livy is running
> *<remote host>:8998*
>
> this adds the endpoint. I then create a session.
>
> *I give the below properties*
> *{"conf": {"spark.master": "yarn-cluster", "spark.submit.deployMode":
> "cluster"}, "proxyUser": "<username>@<proxy>:8080"}*
>
> On clicking Create session
> I get the below message on the notebook
> *Starting Spark application*
>
> I can see that the Livy server responds with the creation on the session ID
>
> But on the notebook I get the following error
>
> *Starting Spark application *
> *ID**YARN Application ID**Kind**State**Spark UI**Driver log**Current
> session?*
> *1* *None* *pyspark* *idle* *✔*
>
>
> *---------------------------------------------------------------------------
> SqlContextNotFoundException Traceback (most recent call last)
> c:\fast\python\2.7.12\lib\site-packages\hdijupyterutils\ipywidgetfactory.pyc
> in submit_clicked(self, button)
> 63
> 64 def submit_clicked(self, button):
> ---> 65 self.parent_widget.run()
>
> c:\fast\python\2.7.12\lib\site-packages\sparkmagic\controllerwidget\createsessionwidget.pyc
> in run(self)
> 56
> 57 try:
> ---> 58 self.spark_controller.add_session(alias, endpoint, skip,
> properties)
> 59 except ValueError as e:
> 60 self.ipython_display.send_error("""Could not add session
> with
>
> c:\fast\python\2.7.12\lib\site-packages\sparkmagic\livyclientlib\sparkcontroller.pyc
> in add_session(self, name, endpoint, skip_if_exists, properties)
> 79 session = self._livy_session(http_client, properties,
> self.ipython_display)
> 80 self.session_manager.add_session(name, session)
> ---> 81 session.start()
> 82
> 83 def get_session_id_for_client(self, name):
>
> c:\fast\python\2.7.12\lib\site-packages\sparkmagic\livyclientlib\livysession.pyc
> in start(self)
> 154 self.sql_context_variable_name = "sqlContext"
> 155 else:
> --> 156 raise SqlContextNotFoundException(u"Neither
> SparkSession nor HiveContext/SqlContext is available.")
> 157 except Exception as e:
> 158
> self._spark_events.emit_session_creation_end_event(self.guid, self.kind,
> self.id <http://self.id>, self.status,
>
> SqlContextNotFoundException: Neither SparkSession nor HiveContext/SqlContext
> is available.
> *
>
>
>
> Any help would be appreciated!
>
> regards
> Vivek
>
--
You received this message because you are subscribed to the Google Groups
"Project Jupyter" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/jupyter/e3973675-ab05-4547-90a1-af49300e747e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
