[ https://issues.apache.org/jira/browse/TOREE-460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Luciano Resende resolved TOREE-460. ----------------------------------- Resolution: Cannot Reproduce Assignee: Luciano Resende Fix Version/s: 0.2.0 I have tried Toree master with both Spark 2.1.0 and 2.2.0 and both have the same behavior where: ``` val a="abc" ``` will produce ``` a = abc Out[5]: abc ``` > Difference in output between Spark 2.1.0 and Spark 2.1.2+ > --------------------------------------------------------- > > Key: TOREE-460 > URL: https://issues.apache.org/jira/browse/TOREE-460 > Project: TOREE > Issue Type: Bug > Components: Kernel > Affects Versions: 0.2.0 > Reporter: Vinayak Joshi > Assignee: Luciano Resende > Fix For: 0.2.0 > > > Using the current version of Toree (0.2.0) & Scala 2.11 when Scala code such > as this is executed: > ``` > var v = "abc" > ``` > against Spark 2.1.0, there is no output generated on the notebook. However, > against Spark 2.1.2 and above, the notebook displays the following lines as > output for the same code: > ``` > v = abc > Out[2]: abc > ``` > When used via a kernel-gateway, I could trace the websocket messages and > observed that a couple of extra messages are received by the notebook when > the code is executed against Spark 2.1.2+ : > ``` > { > "parent_header": { > ... > }, > "msg_type": "display_data", > "msg_id": "7ee0fe40-3283-45dc-8632-a9f013ae7052", > "content": { > "source": "user", > "data": { > "text/plain": "v = abc" > }, > "metadata": {} > }, > "header": { > ... > }, > "channel": "iopub", > "buffers": [], > "metadata": { > "timestamp": "1513154854626" > } > } > ``` > and > ``` > { > "parent_header": { > ... > }, > "msg_type": "execute_result", > "msg_id": "98dae777-c5cf-4ae7-9903-306e1fdf7bc1", > "content": { > "execution_count": 2, > "data": { > "text/plain": "abc" > }, > "metadata": {} > }, > "header": { > ... > }, > "channel": "iopub", > "buffers": [], > "metadata": { > "timestamp": "1513154854754" > } > } > ``` > These messages appear to be getting rendered on the notebook ui. > This is breaking consistency/compatibility for expected outputs when Spark > version changes for notebooks. Also there are users that use some kind of > code cell hiding to prevent credentials etc being displayed in a notebook - > this use case now breaks since the cell output now displays any credential > values that are assigned to variables. > So looks like something's changed between Spark 2.1.0 and later versions that > causes Toree to produce different replies to the same code being executed? Is > it something Toree needs to adapt to? -- This message was sent by Atlassian JIRA (v6.4.14#64029)