[
https://issues.apache.org/jira/browse/NUTCH-2212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15936741#comment-15936741
]
Markus Jelsma commented on NUTCH-2212:
--------------------------------------
Yes i agree. In local mode it is usually not a problem. And we cannot force
mapreduce.map.java.opts to contain lower stack size. I opened this issue when i
had a brain fart when tuning Yarn. Let's close this.
> Decrease memory consumption by tuning stack size
> ------------------------------------------------
>
> Key: NUTCH-2212
> URL: https://issues.apache.org/jira/browse/NUTCH-2212
> Project: Nutch
> Issue Type: Task
> Affects Versions: 1.11
> Reporter: Markus Jelsma
> Assignee: Markus Jelsma
> Priority: Minor
> Fix For: 1.13
>
>
> In today's environments it is common to see a default stack size (-Xss) of 1
> MB. This is ridiculous for a fetcher running many fetcher.threads.fetch. The
> actual number of threads is much higher due to parsing and running it on
> YARN.
> We can decrease stack usage by 75 %, 1 MB to a safe 256 kB. YARN will run out
> of stack size if we set it to 128 kB.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
