Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/6747#issuecomment-111337220
Thanks for working on this much needed cleanup! A few comments (some of
which were preexisting):
- The schema for set commands is wrong (we need some column name)
```
scala> sql("SET -v")
res4: org.apache.spark.sql.DataFrame = [: string]
```
- Related to the above, I think we should consider breaking out the parts
of commands like `SET` and `SET -v` instead of just returning a string.
- When the option has a default, I think we should show that instead of
`<undefined>`
```
scala> sql("SET spark.sql.autoBroadcastJoinThreshold").collect()
res1: Array[org.apache.spark.sql.Row] =
Array([spark.sql.autoBroadcastJoinThreshold=<undefined>])
```
- It seems a bunch of boilerplate could be removed and we could get some
more typesafety by making the conf entries inter classes of SQLConf with
methods like `.value` and `.set(...)`. This would avoid having to have an
entry and a `def` for each config option. Perhaps I'm missing something though.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
