Github user Sherry302 commented on a diff in the pull request:
    --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
    @@ -2420,6 +2420,44 @@ private[spark] object Utils extends Logging {
    +private[spark] class CallerContext(
    +   appName: Option[String] = None,
    +   appID: Option[String] = None,
    +   appAttemptID: Option[String] = None,
    +   jobID: Option[Int] = None,
    +   stageID: Option[Int] = None,
    +   stageAttemptId: Option[Int] = None,
    +   taskId: Option[Long] = None,
    +   taskAttemptNumber: Option[Int] = None) extends Logging {
    +   val AppName = if (appName.isDefined) s"_AppName_${appName.get}" else ""
    --- End diff --
    I have updated the PR to remove appName, and replace appName with something 
to differentiate the context from ApplicationMaster vs Yarn Client vs Task. But 
for AppID, I think it is better to keep it since in hdfs-audit.log, there is no 
info about application. For example, the record below was produced when Task 
did a read/write operation to HDFS, except `callerContext`, there is no other 
info about application:
    2016-09-14 22:29:06,526 INFO FSNamesystem.audit: allowed=true   ugi=wyang 
(auth:SIMPLE) ip=/   cmd=open    src=/lr_big.txt dst=null    perm=null 

If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at or file a JIRA ticket
with INFRA.

To unsubscribe, e-mail:
For additional commands, e-mail:

Reply via email to