andygrove commented on pull request #7967: URL: https://github.com/apache/arrow/pull/7967#issuecomment-683301870
Thanks @alamb for that write-up of options. I am short on time this weekend due to work commitments but here are my thoughts so far: Option 1 seems to add complexity in the internal implementation. Option 2 is the closest to what Spark and other query engines do and is the simplest to implement. As a side note, when you register a Scala UDF in Spark, you do not specify a return type. It is always `Any`, which does not seem like a good design to me. Option 3 seems like the most elegant way to achieve Jorge's requirement and is a great example of the difference between a logical plan and a physical plan. At first, I had a concern about the logical and physical plans ending up with different schemas, but I suppose users would only have this schema difference if they choose to register these types of dynamic functions in the first place, so I think that alleviates my concerns here. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
