Yicong-Huang commented on PR #57800: URL: https://github.com/apache/spark/pull/57800#issuecomment-5198449468
> Should we update the migration guide? To explain mapInPandas UDF returning a plain list of DataFrames (e.g. lambda it: [pdf for pdf in it]) previously succeeded and now raises UDF_RETURN_TYPE with "but is list" I am on the fence here. The public doc declares `Callable[[Iterator[DataFrameLike]]` and accepting `list` which is an iterable not iterator has always been a silently allowed side effect. This PR tightens the allowed return type to its declared type, so not sure how to mention it in migration guide. Maybe @HyukjinKwon @cloud-fan can suggest action here? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
