[
https://issues.apache.org/jira/browse/SPARK-61110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon updated SPARK-61110:
---------------------------------
Fix Version/s: connect-rust-4.3.0
> Add DataFrame._repr_html_, the HTML table of eager evaluation in notebooks,
> in the Python client as PySpark does
> ----------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-61110
> URL: https://issues.apache.org/jira/browse/SPARK-61110
> Project: Spark
> Issue Type: Bug
> Components: Connect
> Affects Versions: connect-rust-4.3.0
> Reporter: Hyukjin Kwon
> Priority: Major
> Fix For: connect-rust-4.3.0
>
>
> In PySpark Connect, {{DataFrame._repr_html_}}, which Jupyter calls to display
> a DataFrame, works as follows:
> * It reads {{spark.sql.repl.eagerEval.enabled}}, {{maxNumRows}} and
> {{truncate}}.
> * If eager evaluation is enabled, it returns the HTML table of the DataFrame.
> The server formats the table, of an {{HtmlString}} relation.
> * Otherwise it returns None.
> * Once it has been called, {{__repr__}} of the DataFrame is its
> {{DataFrame[...]}} string, so a notebook does not show the table twice.
> The Python client lacked {{_repr_html_}}. Attribute access took it as a
> column, so {{df._repr_html_()}} raised a {{TypeError}} ("'Column' object is
> not callable"). A notebook with eager evaluation showed the text table of
> {{__repr__}} instead of the HTML table.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
