[
https://issues.apache.org/jira/browse/SPARK-19094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15803086#comment-15803086
]
holdenk commented on SPARK-19094:
---------------------------------
I've got something basic working for this, but thinking it might make sense to
put this behind a configuration flag as well should it have any unexpected side
effects. (Also need to test on a windows machine when I get some time with one).
> Plumb through logging/error messages from the JVM to Jupyter PySpark
> --------------------------------------------------------------------
>
> Key: SPARK-19094
> URL: https://issues.apache.org/jira/browse/SPARK-19094
> Project: Spark
> Issue Type: Improvement
> Components: PySpark
> Reporter: holdenk
> Priority: Trivial
>
> Jupyter/IPython notebooks works by overriding sys.stdout & sys.stderr, as
> such the error messages that show up in IJupyter/IPython are often missing
> the related logs - which is often more useful than the exception its self.
> This could make it easier for Python developers getting started with Spark on
> their local laptops to debug their applications, since otherwise they need to
> remember to keep going to the terminal where they launched the notebook from.
> One counterpoint to this is that Spark's logging is fairly verbose, but since
> we provide the ability for the user to tune the log messages from within the
> notebook that should be OK.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
