zhengruifeng commented on PR #43011: URL: https://github.com/apache/spark/pull/43011#issuecomment-1732777865
> > > > > What is the purpose of "lambda function"? All others are type-specific or "functionality"-specific. But lambda is "technology". > > > > > > > > > > > > lambda functions were already exposed to end users (e.g. https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.functions.array_sort.html#pyspark.sql.functions.array_sort). > > > > I think if we document other functions here, it is better to add lambda functions as well. > > > > > What is the user journey that would drive one to browse lambda functions? > > > > > > > > > > > > I think this could be an example: when a user try to sort array of structs by a specific order, he may refer to the document of `array_sort` > > > > > > > > > If I try to find a function that sorts arrays I will try to find that function under collection functions. Just like substr() is not a math function, even though most of its arguments are integers. Substr operates on strings... array_sort operates on arrays. > > > > > > Got it. I think we can rename the group name in `.md` files, but need to be different from others. What about `Advanced Collection Functions`? @srielau > > How about having our cake and eat it to? Can a function be in more than one group? probably we can. I will try to map `lambda` to `collection` just in the doc build. I think making a function in more than one group would be much complex. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
