MrPowers commented on pull request #31000: URL: https://github.com/apache/spark/pull/31000#issuecomment-754163014
@MaxGekk - thanks for the review and for your excellent work on this project. I really like the idea of exposing the `make_interval()` function. `make_interval()` is way more powerful than just `add_hours`. Lots of Spark users are struggling with datetime addition (and then drowning in UDF complexity), so I know this'll be a high value function for a lot of users. I created [a JIRA](https://issues.apache.org/jira/browse/SPARK-33995) where we can discuss in more detail. My concern with exposing the `make_interval()` function with the current implementation is that it takes a lot of arguments. `make_interval(hours = 2)` would rely on having default arguments, which I haven't seen in other `org.apach.spark.sql.functions` functions. I'm not sure the named parameter work around to avoid listing all 7 args would translate well to the Java, Python, and R APIs. Datetime objects in most languages are instantiated with 6 args (not including weeks), so that might be another confusing part of `make_interval()` for end users. The [JIRA](https://issues.apache.org/jira/browse/SPARK-33995) outlines different ways we can expose this functionality. Look forward to working with you to decide on the best user interface and will then be happy to implement it. These functions will make users really happy :) ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
