Hi,
Thanks a lot for the great test case! I think I found the problem, but
couldn't fix it yet. It is a concurrency problem. By default, the MVStore
now stores changes in a background thread. This background writing thread
conflicts with the main thread. Concurrency problems are always hard to
test... I'm thinking about how to ensure things like this can't happen.
Anyway, for your use case, the solution is quite easy: disable the
background thread. You don't need it, as you call store yourself. To do
that, use a write delay of -1:
MVStore mvstore = new MVStore.Builder().
writeBufferSize(25).
writeDelay(-1).
fileName(fileName).
cacheSize(100).
open();
Regards,
Thomas
On Thu, Oct 10, 2013 at 3:16 AM, Brian Bray <[email protected]> wrote:
> I was able to boil my stuff down quite a bit by focusing on the
> troublemaker portion. See attached unit test. Using h2-latest.jar, I seem
> to get the IllegalStateException ("negative position exception") after 2-5
> minutes of letting this run. Using h2-1.3.173-release, after 5-6 minutes I
> see the classic signs of a memory leak in jvisualvm (I didn't wait long
> enough for a OutOfMemoryError).
>
> I've tinkered with a bunch of different aspects of this:
> - writeBufferSize()
> - w/ and wo/ mvstore.store() calls every 25k records.
> - various cache sizes
> - store-per-map instead of storing all maps in 1 store/file.
> - Tried different data types in mrrels MVMap (string[], List<String>,
> Map<String,String>)
>
> I realize this test may look a little strange, but there is a lot more
> going on in my real tool and this is representative how it works and the
> strange data I'm working with.
>
> It seems like my issue is in the second loop, where I'm interspersing lots
> of .get()'s from 1 map and .put()'s to another.
>
> Thanks for your help,
> Brian
>
>
> On Wednesday, October 9, 2013 5:05:38 PM UTC-6, Thomas Mueller wrote:
>
>> Hi,
>>
>> I'm sorry I can't say what the problem is. It kind of looks like it's
>> trying to read from a chunk while the chunk is not yet written. But I would
>> need to know what your are doing exactly.
>>
>> - Did you start with a new file?
>> - How large is the file when the problem occurs?
>> - How do you open the MVStore?
>> - How to you process the data (concurrently, when do you call commit, do
>> you call save yourself and if yes when,...)?
>> - How large is a typical record?
>> - What data types do you use?
>>
>> Regards,
>> Thomas
>>
>>
>>
>> On Wed, Oct 9, 2013 at 11:41 PM, Brian Bray <[email protected]> wrote:
>>
>>> Thanks Thomas! So I switched to h2-latest.jar and it fails sooner with
>>> an IllegalStateException:
>>>
>>> Caused by: java.lang.**IllegalStateException: Negative position
>>> -9223372036854646992 [1.3.173/6]
>>> at org.h2.mvstore.DataUtils.**newIllegalStateException(**
>>> DataUtils.java:709)
>>> at org.h2.mvstore.MVStore.**readPage(MVStore.java:1439)
>>> at org.h2.mvstore.MVMap.readPage(**MVMap.java:759)
>>> at org.h2.mvstore.Page.**getChildPage(Page.java:207)
>>> at org.h2.mvstore.MVMap.**binarySearch(MVMap.java:449)
>>> at org.h2.mvstore.MVMap.**binarySearch(MVMap.java:450)
>>> at org.h2.mvstore.MVMap.get(**MVMap.java:431)
>>>
>>> Here is whats odd though, I basically have 3 large CSV files I'm parsing
>>> right now (#1=500MB/4M rows, #2=2.2GB/26M rows, #3=1.3GB/15M rows), it
>>> makes it through the first 2 fine, then I get the above exception about
>>> 100k rows into file #3.
>>>
>>> File #3 was also the only file failing with the previously mentioned
>>> BufferOverflowException in 1.3.173-release (it also seemed to have a slow
>>> memory leak), So I'm wondering if I have some weird data in file #3 that is
>>> causing issues? Again, it might be difficult to extract a unit test out of
>>> this, but focusing on replicating file #3 might make it a bit easier.
>>>
>>> Brian
>>>
>>> On Wednesday, October 9, 2013 1:00:32 PM UTC-6, Thomas Mueller wrote:
>>>
>>>> Hi,
>>>>
>>>> We fixed quite a number of bugs in the MVStore recently. I suggest to
>>>> try again with the nightly build at http://h2database.com/html/**bui**
>>>> ld.html#automated <http://h2database.com/html/build.html#automated> -
>>>> direct link
>>>> http://www.h2database.**com**/automated/h2-latest.jar<http://www.h2database.com/automated/h2-latest.jar>-
>>>> or of course you could build it yourself.
>>>>
>>>> > I'm currently using MVStore as a large, temporary disk cache
>>>>
>>>> Noel and me recently implemented an off-heap storage for the MVStore,
>>>> so it's using memory (not disk) outside the normal Java heap. This might be
>>>> interesting for you. The documentation is not online yet (as it's not
>>>> released yet), it is only available in source form yet at
>>>> https://h2database.**googleco**de.com/svn/trunk/h2/**src/**
>>>> docsrc/html/mvstore.html<https://h2database.googlecode.com/svn/trunk/h2/src/docsrc/html/mvstore.html>(see
>>>> 'off-heap'). To use it, call:
>>>>
>>>> OffHeapStore offHeap = new OffHeapStore();
>>>> MVStore s = new MVStore.Builder().
>>>> fileStore(offHeap).open();
>>>>
>>>> I'm also thinking about combining the LIRS cache with the MVStore, so
>>>> that you could build an off-heap LIRS cache. It shouldn't be complicated to
>>>> implement (the cache would simply needs a map factory).
>>>> Regards,
>>>> Thomas
>>>>
>>>>
>>>>
>>>> On Wed, Oct 9, 2013 at 7:52 PM, Brian Bray <[email protected]> wrote:
>>>>
>>>>> Thomas,
>>>>>
>>>>> Here is a more elusive issue that I'm wondering if its a bug in
>>>>> MVMap/MVStore.
>>>>>
>>>>> I'm currently using MVStore as a large, temporary disk cache, I have a
>>>>> java program that scans about 10GB of raw CSV-ish files, and for each
>>>>> file,
>>>>> plucks out a few fields I care about and stores it in a MVMap. At the end,
>>>>> I merge the data from several MVMaps into a small JSON document which
>>>>> represents all the data buried in the CSV files for a particular entity.
>>>>> I
>>>>> then store that JSON document in H2 for my app to use.
>>>>>
>>>>> Its been working fairly well (and fast!), but there is one weird issue
>>>>> I'm encountering, that throws the exception below after 6-8M rows have
>>>>> been
>>>>> processed. Its going to be tough to extrapolate a test case, I was
>>>>> wondering if you had any insight into this? It seems to go away when I
>>>>> shorten some key sizes, but I don't know if I'm just delaying the problem
>>>>> and eventually this would still happen?
>>>>>
>>>>> I'm using 1.3.173, basically every 50k records or so I call
>>>>> MVstore.store() to flush to disk and eventually its throwing this:
>>>>>
>>>>> Caused by: java.nio.**BufferOverflowExcepti**on
>>>>> at java.nio.HeapByteBuffer.put(**He**apByteBuffer.java:183)
>>>>> at java.nio.ByteBuffer.put(**ByteBu**ffer.java:832)
>>>>> at org.h2.mvstore.type.**ObjectData**Type$**
>>>>> SerializedObjectType.**write(**ObjectDataTpe.java:1515)
>>>>> at org.h2.mvstore.type.**ObjectData**Type.write(**
>>>>> ObjectDataType.**java:113)
>>>>> at org.h2.mvstore.Page.write(**Page**.java:799)
>>>>> at org.h2.mvstore.Page.**writeUnsav**edRecursive(Page.**
>>>>> java:860)
>>>>> at org.h2.mvstore.Page.**writeUnsav**edRecursive(Page.**
>>>>> java:855)
>>>>> at org.h2.mvstore.Page.**writeUnsav**edRecursive(Page.**
>>>>> java:855)
>>>>> at org.h2.mvstore.Page.**writeUnsav**edRecursive(Page.**
>>>>> java:855)
>>>>> at org.h2.mvstore.MVStore.store(**M**VStore.java:921)
>>>>> at org.h2.mvstore.MVStore.store(**M**VStore.java:813)
>>>>>
>>>>> Thanks,
>>>>> Brian
>>>>>
>>>>> --
>>>>> You received this message because you are subscribed to the Google
>>>>> Groups "H2 Database" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>>> an email to h2-database...@**googlegroups.**com.
>>>>> To post to this group, send email to [email protected].
>>>>>
>>>>> Visit this group at
>>>>> http://groups.google.com/**group**/h2-database<http://groups.google.com/group/h2-database>
>>>>> .
>>>>> For more options, visit
>>>>> https://groups.google.com/**grou**ps/opt_out<https://groups.google.com/groups/opt_out>
>>>>> .
>>>>>
>>>>
>>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "H2 Database" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to h2-database...@**googlegroups.com.
>>> To post to this group, send email to [email protected].
>>> Visit this group at
>>> http://groups.google.com/**group/h2-database<http://groups.google.com/group/h2-database>
>>> .
>>> For more options, visit
>>> https://groups.google.com/**groups/opt_out<https://groups.google.com/groups/opt_out>
>>> .
>>>
>>
>> --
> You received this message because you are subscribed to the Google Groups
> "H2 Database" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> To post to this group, send email to [email protected].
> Visit this group at http://groups.google.com/group/h2-database.
> For more options, visit https://groups.google.com/groups/opt_out.
>
--
You received this message because you are subscribed to the Google Groups "H2
Database" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/h2-database.
For more options, visit https://groups.google.com/groups/opt_out.
