[
https://issues.apache.org/jira/browse/NUTCH-2212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15936581#comment-15936581
]
Sebastian Nagel commented on NUTCH-2212:
----------------------------------------
Hi Markus, is this really a problem of Nutch? The stack size can be set as Java
option {{-Xss}} by adding it to the environment variable NUTCH_OPTS which is
read by bin/nutch. In Hadoop distributed mode there is
{{mapreduce.map.java.opts}}. The platform-specific default it on the safe side,
users who know when they can safely tune it, are already able to do this.
> Decrease memory consumption by tuning stack size
> ------------------------------------------------
>
> Key: NUTCH-2212
> URL: https://issues.apache.org/jira/browse/NUTCH-2212
> Project: Nutch
> Issue Type: Task
> Affects Versions: 1.11
> Reporter: Markus Jelsma
> Assignee: Markus Jelsma
> Priority: Minor
> Fix For: 1.13
>
>
> In today's environments it is common to see a default stack size (-Xss) of 1
> MB. This is ridiculous for a fetcher running many fetcher.threads.fetch. The
> actual number of threads is much higher due to parsing and running it on
> YARN.
> We can decrease stack usage by 75 %, 1 MB to a safe 256 kB. YARN will run out
> of stack size if we set it to 128 kB.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
