I filed SPARK-30510
<https://issues.apache.org/jira/browse/SPARK-30510> thinking
that we had forgotten to document an option, but it turns out that there's
a whole bunch of stuff under SQLConf.scala
<https://github.com/apache/spark/blob/0f3d744c3f19750ab03eeae3606e122dcffae5df/sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala>
that has no public documentation under http://spark.apache.org/docs.

Would it be appropriate to somehow automatically generate a documentation
page from SQLConf.scala, as Hyukjin suggested on that ticket?

Another thought that comes to mind is moving the config definitions out of
Scala and into a data format like YAML or JSON, and then sourcing that both
for SQLConf as well as for whatever documentation page we want to generate.
What do you think of that idea?

Nick

Reply via email to