Istvan Toth created HBASE-25081:
-----------------------------------
Summary: Up the container nproc uplimit to 30000
Key: HBASE-25081
URL: https://issues.apache.org/jira/browse/HBASE-25081
Project: HBase
Issue Type: Bug
Components: test
Reporter: Istvan Toth
We (Apache Phoenix team) have recently switched our precommit tests to
Dockerized Yetus (mostly adopted from the solution in Hbase)
We see
java.lang.OutOfMemoryError: unable to create new native thread
errors , while Yetus shows
|Max. process+thread count|6833 (vs. ulimit of 12500)|
While I couldn't determine what was the job that we shared the Agent with at
the time, statistically it was very likely HBase, and an HBase job probably
failed with a similar error.
Some research has thrown up the official Docker docs:
[https://docs.docker.com/engine/reference/commandline/run/#set-ulimits-in-container-ulimit]
According to which it is not possible to set container level nprocs ulimit with
Docker.
All settings apply to the docker Daemon user instead, and the limit is shared
between all containers.
Based on this, I think that it makes no sense to set a container (really docker
user) nprocs ulimit any lower than the current hard limit of 30000.
I have already set PROC_LIMIT=30000 in the Phoenix Yetus personality, but it is
only a half solution until some Docker users set lower values, as the later
setting will apply as soon as the container is started.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
