[
https://issues.apache.org/jira/browse/YARN-6509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15991305#comment-15991305
]
Junping Du commented on YARN-6509:
----------------------------------
bq. Minor point of correction, 2^63 -1 is something like 8.3 million TiB.
My bad. You are right, it is ~8 EB which should be large enough.
bq. I'd like it if we could find some operational users to weigh in on wether
10GiB is enough for a default. could we ask on the yarn user list?
I doubt we can have a uniformed value for different users that operated on
different scale of cluster. My personal feeling is over 10GB log file is hard
to analysis but it happen often for large job on some large clusters. Or we can
keep above size as default which almost means no limit by default and let each
user/admin set this value case by case if they want to?
> Add a size threshold beyond which yarn logs will require a force option
> -----------------------------------------------------------------------
>
> Key: YARN-6509
> URL: https://issues.apache.org/jira/browse/YARN-6509
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Siddharth Seth
> Assignee: Xuan Gong
> Fix For: 2.9.0
>
> Attachments: YARN-6509.1.patch
>
>
> An accidental fetch for a long running application can lead to scenario which
> the large size of log can fill up a disk.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
