Github user vanzin commented on the pull request:
https://github.com/apache/spark/pull/10205#issuecomment-189390014
> I assume the intention of the doc is to eventually generate the web pages
from this string?
That can be a future use but right now it's so Spark SQL can use this code
also. Spark SQL has a command (I think) where they print all available options
to the output, and that uses these strings.
> What exactly does optional mean? Is it just that there is no default
It's a config with no default. Basically any config you'd retrieve using
`SparkConf.getOption`.
I'm not sure I fully understand the rest of your suggestions.
Having the type and default value be separate parameters complicates the
builder class a little. You'd have to specify the type up front, when
instantiating the builder, or have a bunch of different builder sub-classes,
otherwise it becomes tricky to maintain the type information as you call
different methods.
Not sure what you mean by validator either. There's already code to parse
the strings into the proper types, is that what you mean? Or you mean make sure
types conform to a certain spec (like an enum, which already exists)?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
