ctubbsii opened a new issue #2463:
URL: https://github.com/apache/accumulo/issues/2463
**Test name(s)**
-
org.apache.accumulo.test.compaction.ExternalCompaction_2_IT.testDeleteTableCancelsExternalCompaction
-
org.apache.accumulo.test.compaction.ExternalCompaction_3_IT.testCoordinatorRestartsDuringCompaction
**Describe the failure observed**
These tests frequently time out, and never finish, even when given an
excessive amount of time to finish. I have also seem them frequently take down
my testing environment because they eventually cause Jenkins to be killed by
the oom-killer. Even if they survive the oom-killer, when the test eventually
times out, sometimes they leave lots of processes around, which causes further
instability in subsequent tests.
**Testing Environment:**
```
Executing Maven: -B -f
/home/jenkins/jenkins-agent/workspace/Accumulo/main/pom.xml -V -e -ntp clean
verify -Dstyle.color=always -Dfailsafe.skipAfterFailureCount=10
-Dfailsafe.rerunFailingTestsCount=1 -Dsurefire.forkCount=2 -Dtimeout.factor=3
Apache Maven 3.8.4 (9b656c72d54e5bacbed989b64718c159fe39b537)
Maven home: /usr/local/asfpackages/maven/apache-maven-3.8.4
Java version: 11.0.12, vendor: Oracle Corporation, runtime:
/usr/local/asfpackages/java/oraclejdk-11.0.12
Default locale: en_US, platform encoding: ISO-8859-1
OS name: "linux", version: "4.15.0-162-generic", arch: "amd64", family:
"unix"
```
**What have you tried already?**
I've already fixed #2455 and #2454 which have been observed with these
tests. I looked at the code and saw that there's at least one issue with the
Compactors trying indefinitely in an infinite retry loop to connect with the
coordinator. If the coordinator dies, that could cause the Compactor to hang
indefinitely, and the test to hang because of it. If the coordinator just isn't
up yet, the compactor's constant retrying might cause a problem with CPU load,
preventing things from working well (although that's probably a stretch). I
looked at the code in these tests and saw that they were doing something unique
for starting/stopping processes that's different from how other services are
managed by MiniAccumuloCluster. So, there could be a problem there. At the very
least, the new server types should be known to Mini's ServerType enum.
**Additional context**
Most recently, I saw these fail on ASF's Jenkins:
https://ci-builds.apache.org/job/Accumulo/job/main/267/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
