Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/19973
Well, the default for fallback configs is the current value of the parent
conf - it needs context. The code you reference in the a previous comment has
that context (the SQL conf map), so the value of the parent conf might not be
the default.
If you look at `ConfigReader.getOrDefault` it has some (kinda nasty) code
to do this.
If I understand correctly, the problem is really here (the other
`getConfString` method):
```
def getConfString(key: String): String = {
Option(settings.get(key)).
orElse {
// Try to use the default value
Option(sqlConfEntries.get(key)).map(_.defaultValueString)
}.
getOrElse(throw new NoSuchElementException(key))
}
```
That will be broken for fallback configs with the existing code, but it
will also return the wrong thing with your updated code, in the case where the
parent conf is set.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
