holdenk created SPARK-19094:
-------------------------------
Summary: Plumb through logging/error messages from the JVM to
Jupyter PySpark
Key: SPARK-19094
URL: https://issues.apache.org/jira/browse/SPARK-19094
Project: Spark
Issue Type: Improvement
Components: PySpark
Reporter: holdenk
Priority: Trivial
Jupyter/IPython notebooks works by overriding sys.stdout & sys.stderr, as such
the error messages that show up in IJupyter/IPython are often missing the
related logs - which is often more useful than the exception its self.
This could make it easier for Python developers getting started with Spark on
their local laptops to debug their applications, since otherwise they need to
remember to keep going to the terminal where they launched the notebook from.
One counterpoint to this is that Spark's logging is fairly verbose, but since
we provide the ability for the user to tune the log messages from within the
notebook that should be OK.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
