Reylak opened a new issue, #5397: URL: https://github.com/apache/openwhisk/issues/5397
## Environment details: (all that matters is that I run on ArchLinux) * host: ArchLinux (2023-04-20), Linux 6.2.11 * Gradlew: ``` ------------------------------------------------------------ Gradle 6.9.1 ------------------------------------------------------------ Build time: 2021-08-20 11:15:18 UTC Revision: f0ddb54aaae0e44f0a7209c3c0274d506ea742a0 Kotlin: 1.4.20 Groovy: 2.5.12 Ant: Apache Ant(TM) version 1.10.9 compiled on September 27 2020 JVM: 1.8.0_362 (Oracle Corporation 25.362-b09) OS: Linux 6.2.11-arch1-1 amd64 ``` ## Steps to reproduce the issue: 1. clone OpenWhisk source repository 2. `./gradlew distDocker` ## Provide the expected results and outputs: OpenWhisk should build in reasonable time (a matter of minutes, less than 3min on my computer). ## Provide the actual results and outputs: OpenWhisk is very slow to build. The very slow phase is building ow-utils, in particular installing Python 2 native modules which also compiles them (>=30min and counting, I stopped after this time). ## Proposed fix Move to Python 3-only base Docker images. Workaround: set a `ulimit` directive on Docker build commands in "gradle/docker.gradle" to lower the limit on the number of open files (`nofile`). For example: ```gradle dockerBuildArg = ['build', '--ulimit', 'nofile=4096:4096'] ``` ## Additional information you deem important: The underlying problem has been identified in an issue about another topic in [openwhisk-deploy-kube](https://github.com/apache/openwhisk-deploy-kube/issues/673#issuecomment-869067074): Python 2 does not handle well high numbers of possibly open FDs when asked to close them when calling `subprocess.Popen` (see [Python bug report](https://bugs.python.org/issue1663329)). What this has to do with building OpenWhisk, is that the Docker image for ow-utils (possibly others) uses Python 2 and installs Python packages with native modules; the latter must be compiled on-the-fly, incurring a lot of calls to `subprocesss.Popen` (just a guess here) that trigger this behavior. It is very visible on ArchLinux because the default ulimit on the number of open files in a Docker container is very high in this distribution (not sure why not in others, I guess they set a lower default value system-wide). Honestly, I think effort should be made to definitely migrate from Python 2 in every OpenWhisk component. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
