amahussein commented on pull request #2537:
URL: https://github.com/apache/hadoop/pull/2537#issuecomment-747057646
> All OOMs are "unable to create new native thread" indicating ulimit or
resource shortage to create LWP. The first OOM is in TestJvmMetrics in
hadoop-common. If ISA-L is related, the cause should be in the code path of
ErasureCodeNative#loadLibrary. I don't have clear insight yet. I think we have
been familiar with test failures by "unable to create new native thread" for a
long time..
@iwasakims , I cannot fully confident that `ErasureCodeNative#loadLibrary`
is a strong indication that ISLA-L does not contribute to the OOM.
ISA-L is a native library; therefore loading this library means different
memory allocations and possibly some background threads.
For sure, we do not want to blame those pre-existing failures to ISA-L.
However, adding ISA-L could increase failures because of the hadoop code, or
the native code.
I think there are two approaches:
1. Profile the memory. Then compare the two profiles with and without ISA-L.
If there is no Yetus hookup to do that, then it will have to be done on a local
machine for a sample of unit tests.
2. Add another commit that ignores the failures frequently reported in QBT
report. In addition I suggest adding "ignore" to
`TestDistributredShell#testDistributedShellWithResourcesWithLargeContainers`
and `TestDistributredShell#testDistributedShellWithResources`. Those two tests
leave two ApplicationMaster processes running in the background. After ignoring
the "every-day" failures, we can look at the remaining failures as possible
consequences of loading ISA-L.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
