[ https://issues.apache.org/jira/browse/YARN-6875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16109148#comment-16109148 ]
Xuan Gong commented on YARN-6875: --------------------------------- Thanks for the suggestion. [~leftnoteasy] The approach looks fine. But it would introduce extra complexity if we enable the compression for the log aggregation. Instead of appending UUID + block_id for every fixed N bytes, we would append them for every aggregated log which might be easier given that we do the compression for every log type separately. > New aggregated log file format for YARN log aggregation. > -------------------------------------------------------- > > Key: YARN-6875 > URL: https://issues.apache.org/jira/browse/YARN-6875 > Project: Hadoop YARN > Issue Type: New Feature > Reporter: Xuan Gong > Assignee: Xuan Gong > Attachments: YARN-6875-NewLogAggregationFormat-design-doc.pdf > > > T-file is the underlying log format for the aggregated logs in YARN. We have > seen several performance issues, especially for very large log files. > We will introduce a new log format which have better performance for large > log files. -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org