[GitHub] [spark] zhengruifeng commented on pull request #43011: [SPARK-45232][SQL][DOCS] Add missing function groups to SQL references
zhengruifeng commented on PR #43011: URL: https://github.com/apache/spark/pull/43011#issuecomment-1734891541 thanks @srielau @allisonwang-db @HyukjinKwon merged to master -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] zhengruifeng commented on pull request #43011: [SPARK-45232][SQL][DOCS] Add missing function groups to SQL references
zhengruifeng commented on PR #43011: URL: https://github.com/apache/spark/pull/43011#issuecomment-1734633746 @srielau I have put `lambda functions` in `collection functions`, I think this PR is ready to merge? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] zhengruifeng commented on pull request #43011: [SPARK-45232][SQL][DOCS] Add missing function groups to SQL references
zhengruifeng commented on PR #43011: URL: https://github.com/apache/spark/pull/43011#issuecomment-1732950643   now we put `lambda functions` into `collection functions` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] zhengruifeng commented on pull request #43011: [SPARK-45232][SQL][DOCS] Add missing function groups to SQL references
zhengruifeng commented on PR #43011: URL: https://github.com/apache/spark/pull/43011#issuecomment-1732777865 > > > > > What is the purpose of "lambda function"? All others are type-specific or "functionality"-specific. But lambda is "technology". > > > > > > > > > > > > lambda functions were already exposed to end users (e.g. https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.functions.array_sort.html#pyspark.sql.functions.array_sort). > > > > I think if we document other functions here, it is better to add lambda functions as well. > > > > > What is the user journey that would drive one to browse lambda functions? > > > > > > > > > > > > I think this could be an example: when a user try to sort array of structs by a specific order, he may refer to the document of `array_sort` > > > > > > > > > If I try to find a function that sorts arrays I will try to find that function under collection functions. Just like substr() is not a math function, even though most of its arguments are integers. Substr operates on strings... array_sort operates on arrays. > > > > > > Got it. I think we can rename the group name in `.md` files, but need to be different from others. What about `Advanced Collection Functions`? @srielau > > How about having our cake and eat it to? Can a function be in more than one group? probably we can. I will try to map `lambda` to `collection` just in the doc build. I think making a function in more than one group would be much complex. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] zhengruifeng commented on pull request #43011: [SPARK-45232][SQL][DOCS] Add missing function groups to SQL references
zhengruifeng commented on PR #43011: URL: https://github.com/apache/spark/pull/43011#issuecomment-1730830602 > > > What is the purpose of "lambda function"? All others are type-specific or "functionality"-specific. But lambda is "technology". > > > > > > lambda functions were already exposed to end users (e.g. https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.functions.array_sort.html#pyspark.sql.functions.array_sort). > > I think if we document other functions here, it is better to add lambda functions as well. > > > What is the user journey that would drive one to browse lambda functions? > > > > > > I think this could be an example: when a user try to sort array of structs by a specific order, he may refer to the document of `array_sort` > > If I try to find a function that sorts arrays I will try to find that function under collection functions. Just like substr() is not a math function, even though most of its arguments are integers. Substr operates on strings... array_sort operates on arrays. Got it. I think we can rename the group name in `.md` files, but need to be different from others. What about `Advanced Collection Functions`? @srielau -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] zhengruifeng commented on pull request #43011: [SPARK-45232][SQL][DOCS] Add missing function groups to SQL references
zhengruifeng commented on PR #43011: URL: https://github.com/apache/spark/pull/43011#issuecomment-1730787450 @HyukjinKwon this page is not built from `functions.scala`, but from the groups specified in expression definitions, like https://github.com/apache/spark/blob/6b747ab8cef44ea46102b2685b9254f7c4828ea8/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/mathExpressions.scala#L1313 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] zhengruifeng commented on pull request #43011: [SPARK-45232][SQL][DOCS] Add missing function groups to SQL references
zhengruifeng commented on PR #43011: URL: https://github.com/apache/spark/pull/43011#issuecomment-1730672720 > What is the purpose of "lambda function"? All others are type-specific or "functionality"-specific. But lambda is "technology". lambda functions were already exposed to end users (e.g. https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.functions.array_sort.html#pyspark.sql.functions.array_sort). I think if we document other functions here, it is better to add lambda functions as well. > What is the user journey that would drive one to browse lambda functions? I think this could be an example: when a user try to sort array of structs by a specific order, he may refer to the document of `array_sort` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
