lhotari opened a new pull request #11414:
URL: https://github.com/apache/pulsar/pull/11414


   ### Motivation
   
   There are several integration test jobs failing where the docker container 
run by Testcontainers gets terminated with error code 137 (maps to out of 
memory error).
   
   The failing jobs are:
   CI - Integration - Sql - 
https://github.com/apache/pulsar/actions/workflows/ci-integration-sql.yaml 
(most fail)
   CI - Integration - Process - 
https://github.com/apache/pulsar/actions/workflows/ci-integration-process.yaml 
(some succeed)
   CI - Integration - Messaging - 
https://github.com/apache/pulsar/actions/workflows/ci-integration-messaging.yaml
 (some succeed)
   CI - Integration - Function & IO - 
https://github.com/apache/pulsar/actions/workflows/ci-integration-function.yaml 
(some succeed)
   
   This started happening yesterday for most PR builds.
   
   For example:
   
https://github.com/apache/pulsar/runs/3111868662?check_suite_focus=true#step:14:1024
   
   ```
   Error:  Tests run: 22, Failures: 1, Errors: 0, Skipped: 21, Time elapsed: 
292.035 s <<< FAILURE! - in TestSuite
   Error:  
testPythonWordCountFunction(org.apache.pulsar.tests.integration.functions.PulsarStateTest)
  Time elapsed: 43.416 s  <<< FAILURE!
   org.apache.pulsar.tests.integration.docker.ContainerExecException: 
/pulsar/bin/pulsar-admin functions querystate --tenant public --namespace 
default --name test-wordcount-py-fn-tfhycxsf --key message-1 failed on 
705ecb067214d1cc42cd16358df6fa6d7a8cacc6c5ddd0cdde84a73b3e2e1f76 with error 
code 137
   at 
org.apache.pulsar.tests.integration.utils.DockerUtils$2.onComplete(DockerUtils.java:248)
   at 
org.testcontainers.shaded.com.github.dockerjava.core.exec.AbstrAsyncDockerCmdExec$1.onComplete(AbstrAsyncDockerCmdExec.java:51)
   at 
org.testcontainers.shaded.com.github.dockerjava.core.DefaultInvocationBuilder.lambda$executeAndStream$1(DefaultInvocationBuilder.java:276)
   at java.base/java.lang.Thread.run(Thread.java:829)
   ```
   
   It seems that GitHub Actions Runner VM has increased memory consumption 
after[ the most recent 
updates](https://github.com/actions/virtual-environments/blob/releases/ubuntu20/20210718/images/linux/Ubuntu2004-README.md).
   
   ### Modifications
   
   - Reduce Presto maximum heap size in integration tests
   - Reduce standalone container memory usage
   - Stop some system services to save RAM
   - Redcue memory usage of the integration tests surefire JVM


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


Reply via email to