Scott Selvia created DIRECTMEMORY-139:
-----------------------------------------
Summary: IllegalArgumentException Error when Loading a Map using
JDK 8
Key: DIRECTMEMORY-139
URL: https://issues.apache.org/jira/browse/DIRECTMEMORY-139
Project: Apache DirectMemory
Issue Type: Bug
Components: core
Affects Versions: 0.2
Environment: Windows 7 x64 32 GB of RAM. Running app from Netbeans 8
using JDK 8
Reporter: Scott Selvia
Fix For: 0.2
I’m using the following code to initialize my map, I’m also trying to get a
handle the amount of memory to allocate. Using the code below the memory
starts off right around 3 GB as I would expect. However, it keeps increasing
up to 7 GB. I’m trying to load a map of unique addresses with a count of the
number of hits for each. I’m starting small and using only 5 million of the
270 million records. I’ve performed a dump from the database of all of the
unique values and in CSV form it is only 2.5 GB, hence the 3 GB starting point
for the DirectMemory cache.
AllocationPolicy allocationPolicy = new RoundRobinAllocationPolicy();
MemoryManagerService<Integer> memoryManager = new MemoryManagerServiceImpl<>(
allocationPolicy, true );
CacheService<String, Integer> cache = new DirectMemory<String,
Integer>().setMap(new ConcurrentHashMap<String,
Pointer<Integer>>()).setMemoryManager(memoryManager).setNumberOfBuffers(3).setSize(Ram.Mb(1024)).setInitialCapacity(Ram.Mb(1024)).setConcurrencyLevel(4).newCacheService();
As I read an load the file, I get about 400K in and I get the following
stacktrace, but the process continues and finally around 1.2 Million records
processed I get the second stacktrace. I’m using the following code to
retrieve the current count value and put the count back into the map. I’ve
changed the code to only process 500K records, when I dump the map contents I
have a Number of Entries equal to NULL. That part I don’t understand, since
I’m setting either a 1 or a count + 1 value.
*** NOTE *** Using JDK 7 Update 51 I do not get the stacktraces!!!
Integer count = cache.retrieve(temp);
if (count != null)
{
cache.put(temp, count + 1);
}
else
{
cache.put(temp, 1);
}
100000
200000
300000
400000
Exception in thread "Timer-0" java.lang.IllegalArgumentException: The buffer
org.apache.directmemory.memory.allocator.MergingByteBufferAllocator$MergingNioMemoryBuffer@25015757
seems not to belong to this allocator
at
org.apache.directmemory.memory.allocator.MergingByteBufferAllocator$MergingNioMemoryBuffer.free(MergingByteBufferAllocator.java:396)
at
org.apache.directmemory.memory.allocator.MergingByteBufferAllocator.free(MergingByteBufferAllocator.java:157)
at
org.apache.directmemory.memory.MemoryManagerServiceImpl.free(MemoryManagerServiceImpl.java:185)
at
org.apache.directmemory.memory.AbstractMemoryManager.free(AbstractMemoryManager.java:149)
at
org.apache.directmemory.memory.AbstractMemoryManager.collectLFU(AbstractMemoryManager.java:139)
at
org.apache.directmemory.cache.CacheServiceImpl.collectLFU(CacheServiceImpl.java:260)
at
org.apache.directmemory.cache.CacheServiceImpl$1.run(CacheServiceImpl.java:85)
at java.util.TimerThread.mainLoop(Timer.java:555)
at java.util.TimerThread.run(Timer.java:505)
500000
600000
700000
800000
900000
1000000
1100000
1200000
java.lang.IllegalArgumentException: The buffer
org.apache.directmemory.memory.allocator.MergingByteBufferAllocator$MergingNioMemoryBuffer@39e2ab21
seems not to belong to this allocator
at
org.apache.directmemory.memory.allocator.MergingByteBufferAllocator$MergingNioMemoryBuffer.free(MergingByteBufferAllocator.java:396)
at
org.apache.directmemory.memory.allocator.MergingByteBufferAllocator.free(MergingByteBufferAllocator.java:157)
at
org.apache.directmemory.memory.MemoryManagerServiceImpl.free(MemoryManagerServiceImpl.java:185)
at
org.apache.directmemory.cache.CacheServiceImpl.store(CacheServiceImpl.java:148)
at
org.apache.directmemory.cache.CacheServiceImpl.put(CacheServiceImpl.java:118)
at
org.apache.directmemory.cache.CacheServiceImpl.put(CacheServiceImpl.java:109)
at
com.datamentors.CreateVoltDBLoadFileAndMaps.<init>(CreateMaps.java:150)
at
com.datamentors.CreateVoltDBLoadFileAndMaps.main(CreateMaps.java:290)
--
This message was sent by Atlassian JIRA
(v6.2#6252)
