Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/7564#issuecomment-123199696
Looks like thereâs a problem with using ScalaUDFs in the unsafe
GeneratedAggregate path. It looks like thereâs a case where we end up using
a BoundReference expression to extract fields before passing them to the UDF.
This ends up calling the generic Row.apply(), which returns data of the wrong
type when called on an UnsafeRow (specifically, the failing test tries to get a
Long column but ends up getting a byte array back).
One way to fix this would be to implement code generation for ScalaUDF so
that the field access expressions are code generated.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
