HyukjinKwon commented on issue #23810: [SPARK-26901][SQL][R] Avoid to prune columns for vectorized gapply() URL: https://github.com/apache/spark/pull/23810#issuecomment-464364408 cc @cloud-fan and @viirya . I have been separately taking a look for this. I was thinking I might need a Python UDF like extraction and projection rules but looks I don't need it if I am not mistaken. FYI, regular `gapply` passes since it's always wrapped by `SerializeFromObject`. In case of Pandas UDFs, I think they are guided by Python UDF extraction + projection. Can you take a look and see if it makes sense to you?
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
