Hi all,

I happened to review some PRs and I noticed that some configurations don't
have some information
necessary.

To be explicit, I would like to make sure we document the direct
relationship between other configurations
in the documentation. For example,
`spark.sql.adaptive.shuffle.reducePostShufflePartitions.enabled`
can be only enabled when `spark.sql.adaptive.enabled` is enabled. That's
clearly documented.
We're good in general given that we document them in general in Apache
Spark.
See 'spark.task.reaper.enabled', 'spark.dynamicAllocation.enabled',
'spark.sql.parquet.filterPushdown', etc.

However, I noticed such a pattern that such information is missing in some
components in general, for example,
`spark.history.fs.cleaner.*`, `spark.history.kerberos.*` and
`spark.history.ui.acls.* `

I hope we all start to document such information. Logically users can't
know the relationship and I myself
had to read the codes to confirm when I review.
I don't plan to document this officially yet because to me it looks a
pretty logical request to me; however,
let me know if you guys have some different opinions.

Thanks.

Reply via email to