[
https://issues.apache.org/jira/browse/SPARK-21914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16157542#comment-16157542
]
Reynold Xin commented on SPARK-21914:
-------------------------------------
Thanks for pinging me. Given the test coverage for this is going to be super
low, I'm not sure how useful it is ...
> Running examples as tests in SQL builtin function documentation
> ---------------------------------------------------------------
>
> Key: SPARK-21914
> URL: https://issues.apache.org/jira/browse/SPARK-21914
> Project: Spark
> Issue Type: Test
> Components: SQL
> Affects Versions: 2.3.0
> Reporter: Hyukjin Kwon
>
> It looks we have added many examples in {{ExpressionDescription}} for builtin
> functions.
> Actually, if I have seen correctly, we have fixed many examples so far in
> some minor PRs and sometimes require to add the examples as tests sql and
> golden files.
> As we have formatted examples in {{ExpressionDescription.examples}} -
> https://github.com/apache/spark/blob/ba327ee54c32b11107793604895bd38559804858/sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/ExpressionDescription.java#L44-L50,
> and we have `SQLQueryTestSuite`, I think we could run the examples as tests
> like Python's doctests.
> Rough way I am thinking:
> 1. Loads the example in {{ExpressionDescription}}.
> 2. identify queries by {{>}}.
> 3. identify the rest of them as the results.
> 4. run the examples by reusing {{SQLQueryTestSuite}} if possible.
> 5. compare the output by reusing {{SQLQueryTestSuite}} if possible.
> Advantages of doing this I could think for now:
> - Reduce the number of PRs to fix the examples
> - De-duplicate the test cases that should be added into sql and golden files.
> - Correct documentation with correct examples.
> - Reduce reviewing costs for documentation fix PRs.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
