JoshRosen commented on code in PR #36747:
URL: https://github.com/apache/spark/pull/36747#discussion_r887637758
##########
docs/configuration.md:
##########
@@ -449,7 +449,7 @@ of the most common options to set are:
</tr>
<tr>
<td><code>spark.driver.log.layout</code></td>
- <td>%d{yy/MM/dd HH:mm:ss.SSS} %t %p %c{1}: %m%n</td>
+ <td>%d{yy/MM/dd HH:mm:ss.SSS} %t %p %c{1}: %m%n%ex</td>
Review Comment:
If we want to mention something about this in the docs, I think it's
probably better to mention it further down in the section that discusses log
configuration more generally:
https://github.com/apache/spark/blob/7f7768773add6a2129fd1ba18654c5d811b18aa5/docs/configuration.md#configuring-logging
. The specific line that I changed here is a configuration which is used only
if the `spark.driver.log.persistToDfs.enabled` feature is enabled, which I
think is too specific of a place to mention this more general caveat.
I think the best place to mention this is `log4j.properties.template`, since
that's probably where most users will look when seeking to customize their
logging configuration. In
https://github.com/apache/spark/pull/36747/commits/44664b7559b402548400408e478581d2ad6ddb31
I added a note to that template file; take a look and let me know if that
looks okay.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
