> In most cases, the OOM occurs when closing MiniDFSCluster.
@Akira Ajisaka , I see some usage of parallelStream()
in RouterRPCServer.
If we consider pool executors in the code and parallelStream(), could it be
possible that there
is a large number of threads created in the MiniCluster and
Hi Akira,
Will reducing the number of threads not increase the build time? I guess it
takes in general 2.5-3.5 hrs in the present scenario. Moreover the thread count
hasn’t been increased recently, Would that be the root of all evil?
Can we not ask for more resources?
Anyway, I think RBF
Sorry, now I think the above comment is wrong. Please ignore.
In hadoop-common, hadoop-hdfs, and hadoop-hdfs-rbf, the unit tests are
executed in parallel. I'd like to reduce the number of tests running
at the same time to avoid OOM. Filed
https://issues.apache.org/jira/browse/HDFS-15731
On Thu,
In most cases, the OOM occurs when closing MiniDFSCluster.
Added a detailed comment in
https://issues.apache.org/jira/browse/HDFS-13579 and created a PR:
https://github.com/apache/hadoop/pull/2555
-Akira
On Fri, Dec 4, 2020 at 12:43 AM Ahmed Hussein wrote:
>
> I remember this error was there
I remember this error was there for more than 6 months. It significantly
slows down the progress of collaboration.
Then, eventually, the community will develop another habit of ignoring the
prebuilds (out of despair).
I am willing to help to get this fixed.
Anyone knows who owns and has
This is still happening.
Latest build:
https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/343/#showFailuresLink
Looks like we are running out of threads in the containers where the unit
tests run. Anyone know where this is setup?
On Wed, Oct 21, 2020 at 5:51 PM Ahmed Hussein
