HyukjinKwon edited a comment on issue #28170: [SPARK-31369][SQL][DOCS] Documentation for JSON Functions URL: https://github.com/apache/spark/pull/28170#issuecomment-611917191 I actually discussed this with @gatorsmile and @cloud-fan offline. I think we should basically reuse `ExpressionDescription` and generate the function documentation there . We can add some more tags or fields into `ExpressionDescription`, and generate the documentation just like we do for SQL configuration (see also https://github.com/apache/spark/pull/27459, #24259). I am sure we can generate similar pages by grouping, updating, etc. in the Python script (e.g., [gen-sql-api-docs.py](https://github.com/apache/spark/blob/master/sql/gen-sql-api-docs.py) and [gen-sql-config-docs.py](https://github.com/apache/spark/blob/master/sql/gen-sql-config-docs.py)). Also, we can remove the documentation when it's wrapped `ExpressionDescription`. For instance, I think we can remove all Scaladoc in the case like this (and merge the doc into `ExpressionDescription`): https://github.com/apache/spark/blob/6704103499d2003b1879ff0b4b8e29141e401b9f/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/mathExpressions.scala#L310-L325 Also, the examples in `ExpressionDescription` are tested in the PR builders as of https://github.com/apache/spark/pull/25942 so we won't have to update the examples.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
