jwang0306 commented on PR #41302:
URL: https://github.com/apache/spark/pull/41302#issuecomment-1564666698
@dongjoon-hyun thanks for the review,
1. This config statically overrides the log level, but that doesn't prevent
users from dynamically setting new overrides with `setLogLevel`. If you have a
strong preference, we can change it to a name without the override word in it,
what would you suggest?
2. Sounds good, but let's decide whether to add it or not after (3).
3. That's a very good point. To solve this inconsistency, I think we have
multiple alternatives:
- That line of log happens when `isInterpreter` is true and `silent` =
false, however, these are not fields in the Logging trait. So we can't read
them in `SparkContext` to warn users about the override under the exact same
condition. Maybe we can made the `isInterpreter` and `silent` fields in the
Logging trait, or add a new filed to tell whether or not the REPL warning is
printed. If we don't care about the original condition (i.e., we don't care
about the `silent` mode), then we can print the warning log unconditionally
like you suggested.
- Alternatively, the Logging trait itself might be a better place to
implement the override, but `SparkConf` is not available when the trait is
initializing. So if we choose to configure the log level override there, we'll
have to change the conf to use something else that's already available at that
point, like environment variable or Java System Properties. WDYT? cc.
@grundprinzip @HyukjinKwon @cloud-fan @LuciferYang
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
