milleruntime commented on issue #1789:
URL: https://github.com/apache/accumulo/issues/1789#issuecomment-785226596
After some experimenting with the ZERO_TIME value of the test and printing
different values coming out of the `LargestFirstMemoryManager`, I think I have
figured out how the test occasionally fails. As to exactly how all the numbers
at play come to cause it to fail, is another story. Here are the strange
circumstances I created to lead to the failure.
I modified the ZERO_TIME of the test to be _exactly_ 4 minutes less than the
`LargestFirstMemoryManager`:
<pre>
public class LargestFirstMemoryManagerTest {
private static final long ZERO = LargestFirstMemoryManager.ZERO_TIME - (4
* 60 * 1000);
</pre>
This will cause the test to always fail on my machine at the same spot we
have seen above and with the exact same AssertionError message:
<pre>
[ERROR] Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
1.008 s <<< FAILURE! - in
org.apache.accumulo.tserver.memory.LargestFirstMemoryManagerTest
[ERROR]
org.apache.accumulo.tserver.memory.LargestFirstMemoryManagerTest.test Time
elapsed: 0.93 s <<< FAILURE!
java.lang.AssertionError: expected:<1;f<> but was:<1;e<>
at
org.apache.accumulo.tserver.memory.LargestFirstMemoryManagerTest.test(LargestFirstMemoryManagerTest.java:118)
</pre>
The weird timing difference combined with the following oddities will lead
to this failure:
- The math used in the `LargestFirstMemoryManager` to calculate
timeMemoryLoad, which is used as a key in the `LargestMap` largestMemTablets
- The fact that the `LargestMap` inner class uses an `ArrayList` to store
multiple tablets keyed at the the same Long
- The final loop in the `LargestFirstMemoryManager` will end as soon as it
hits the desired number of tablets
This can be seen in these log statements:
<pre>
2021-02-24T11:15:57,223 [memory.LargestFirstMemoryManager] INFO :
largestMemTablets.put(timeMemoryLoad 446267447, 1;a<)
2021-02-24T11:15:57,224 [memory.LargestFirstMemoryManager] INFO :
largestMemTablets.put(timeMemoryLoad 892534895, 1;b<)
2021-02-24T11:15:57,224 [memory.LargestFirstMemoryManager] INFO :
largestMemTablets.put(timeMemoryLoad 892534896, 1;c<)
2021-02-24T11:15:57,224 [memory.LargestFirstMemoryManager] INFO :
largestMemTablets.put(timeMemoryLoad 892534897, 1;d<)
2021-02-24T11:15:57,225 [memory.LargestFirstMemoryManager] INFO :
<b>largestMemTablets.put(timeMemoryLoad 892534898, 1;e<)</b>
2021-02-24T11:15:57,225 [memory.LargestFirstMemoryManager] INFO :
<b>largestMemTablets.put(timeMemoryLoad 892534898, 1;f<)</b>
2021-02-24T11:15:57,225 [memory.LargestFirstMemoryManager] INFO : <b>adding
1;f< to list for 892534898</b>
2021-02-24T11:15:57,226 [memory.LargestFirstMemoryManager] INFO :
largestMemTablets.put(timeMemoryLoad 892534899, 1;g<)
2021-02-24T11:15:57,226 [memory.LargestFirstMemoryManager] INFO : Last entry
892534899 has list of size = 1
2021-02-24T11:15:57,226 [memory.LargestFirstMemoryManager] INFO : COMPACTING
1;g< total = 6,979,321,877 ingestMemory = 6,979,321,877
2021-02-24T11:15:57,227 [memory.LargestFirstMemoryManager] INFO : chosenMem
= 1,073,741,830 chosenIT = -240.00 load 892,534,899
2021-02-24T11:15:57,227 [memory.LargestFirstMemoryManager] INFO : <b>Last
entry 892534898 has list of size = 2</b>
2021-02-24T11:15:57,227 [memory.LargestFirstMemoryManager] INFO : COMPACTING
1;e< total = 6,979,321,877 ingestMemory = 6,979,321,877
2021-02-24T11:15:57,228 [memory.LargestFirstMemoryManager] INFO : chosenMem
= 1,073,741,828 chosenIT = -240.00 load 892,534,898
2021-02-24T11:15:57,228 [memory.LargestFirstMemoryManager] INFO : returning
tabletsToMinorCompact with 2 tablets
</pre>
During a failure, the final loop in `LargestFirstMemoryManager` will take
the next tablet in the list of the lastEntry, which may or may not be the next
largest tablet since the backing `ArrayList` of `LargestMap` is not sorted.
During a success, each entry has an `ArrayList` of size one, stored at distinct
Long keys.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
