Harish Jaiprakash created TEZ-4028:
--------------------------------------
Summary: Events not visible from proto history logging for s3a
filesystem until dag completes.
Key: TEZ-4028
URL: https://issues.apache.org/jira/browse/TEZ-4028
Project: Apache Tez
Issue Type: Bug
Reporter: Harish Jaiprakash
Assignee: Harish Jaiprakash
The events are not visible in the files because s3 filesystem
* flush writes to local disk and only upload/commit to s3 on close.
* does not support append
As an initial fix we log the dag submitted, initialized and started events into
a file and these can be read to get the dag plan, config from the AM. The
counters are anyways not available until the dag completes.
The in-progress information cannot be read, this can be obtained from the AM
once we have the above events.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
