ion-elgreco commented on issue #876: URL: https://github.com/apache/datafusion-python/issues/876#issuecomment-2363581681
> My primary concerns about doing this are: > > * API churn rate. If we keep changing our interface too rapidly between versions it will turn people off the project. So if we did this I'd probably support defining them on the expr but also having the functions available - one to call the other, almost as an alias. I think this can work but with the goal to deprecate the functions after X releases/months. One thing that is problematic for example with PySpark codebases is that there is no consistency due to the aliases. I don't think constant change is an issue, unless you don't document well how to change. > * Familiarity. Like it or not, many if not most, of the people who will be coming to the project will be coming from a pyspark/pandas/polars background. The layout we currently have will be more familiar _in some places_ which I think will lead to greater adoption. In some way this might speak more to PySpark users the current API, but arguably and I think many will agree if you come from polars and pandas the current API isn't close to their familiarity. > * Diverging too much from the datafusion core approach. I think some level of difference is expected since rust and python do operate very differently. However if we diverge too much, it makes maintenance harder and harder to onboard new contributors. This is a smaller concern. > > That being said, I have found that some of the functions I expected to be on `Expr` were in `functions` and I do find the interface to be somewhat non-intuitive. My proposal would be to handle these on a case by case basis of what should be where. I can make a draft of mapping Functions -> Expr, so we can get a full picture on how this will look like -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
