Hyukjin Kwon created SPARK-61478:
------------------------------------

             Summary: Raise the done error of a progress handler over a decode 
error of the results, as PySpark does
                 Key: SPARK-61478
                 URL: https://issues.apache.org/jira/browse/SPARK-61478
             Project: Spark
          Issue Type: Bug
          Components: Connect
    Affects Versions: connect-rust-4.3.0
            Reporter: Hyukjin Kwon


Follow-up of SPARK-61398. When a result batch fails to decode, collect and 
toLocalIterator return the decode error without finishing the progress, so the 
done error of a progress handler goes to sys.unraisablehook later, named as a 
LocalRowIterator also for collect. PySpark raises the done error with the 
decode error as its context. A stream whose schema changes between batches can 
also panic in the scan for invalid UTF-8.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to