bbenzikry commented on pull request #10023: URL: https://github.com/apache/airflow/pull/10023#issuecomment-675086606
> Is this log available in another application? Maybe a better solution would be to add a link to another console? https://airflow.readthedocs.io/en/latest/howto/write-logs.html#external-links Not necessarily, as it requires either A) deploying a spark history server in addition to the operator ( we do have that deployed in our cluster, but it's not applicable for any SparkApplication ) B) setting up log forwarding to ES/equivalent from within the spark context C) forwarding logs for kubernetes pods to an external system For a user with airflow access alone, we want the ability to quickly inspect driver logs while inspecting the task without delegating to external systems ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
