[
https://issues.apache.org/jira/browse/HDFS-12711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16222841#comment-16222841
]
Allen Wittenauer commented on HDFS-12711:
-----------------------------------------
Adding some better metrics gathering for YETUS-561 so we have some raw numbers
to look at. Also working on some automatic child killing drones in YETUS-570.
In the process of this, learned that while running trunk HDFS unit tests,
thread count spikes up to ~4000 threads on their own. I think some tests or
maybe even some core code need some major tuning. It's pretty easy to see that
if those JVMs were to stall, the thread count is going to climb to ridiculous
heights.
In other news, all Yetus-based Hadoop jobs on Jenkins are now running this code
with the process limit set to 5k. This is likely too few for mapreduce, given
it's proclivity to bringing up multiple clusters in their entirety on a regular
basis.
> deadly hdfs test
> ----------------
>
> Key: HDFS-12711
> URL: https://issues.apache.org/jira/browse/HDFS-12711
> Project: Hadoop HDFS
> Issue Type: Test
> Affects Versions: 2.9.0, 2.8.2
> Reporter: Allen Wittenauer
> Priority: Critical
> Attachments: HDFS-12711.branch-2.00.patch, MAPREDUCE-4980.016.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
