Hi,
I'm pretty sure that this method is the one introducing the extra
full mapping of the repository:
FileStoreHelper.checkFileStoreVersionOrFail
We should probably run this check with memory mapping disabled anyway.
Nothing to gain here but probably this would fix the double mapping and
sideline OAK-4274, which effectively is
http://bugs.java.com/view_bug.do?bug_id=4724038.
Michael
On 23.03.17 16:13, Alex Parvulescu wrote:
Hi,
To add what I have found so far. This seems related to OAK-4274, but
I think there might be a twist in there somewhere. I'm pretty sure
that this method is the one introducing the extra full mapping of the
repository: FileStoreHelper.checkFileStoreVersionOrFail [0].
Disabling this method takes the 2x mapping away completely.
The reason I'm saying it is relate to OAK-4274 is because I looked at
a heap dump to verify what is keeping the references to the readonly
store and there are no live ones, the refs should be GC'ed, but for
some reason they are not.
I'm still poking around, did not create an oak issue yet. Still
pending is to verify if this affects other areas than oak-run.
Feedback is more than welcome!
best, alex
[0]
https://github.com/apache/jackrabbit-oak/blob/1.6/oak-run/src/main/java/org/apache/jackrabbit/oak/plugins/segment/FileStoreHelper.java#L209
On Thu, Mar 23, 2017 at 12:10 PM, Ian Boston <i...@tfd.co.uk> wrote:
Hi, Based on the page fault behaviour, I think the areas mapped and
reported by pmap are being actively accessed by the JVM. The number
of page faults for Oak 1.4.11 is well over 2x the number of page
faults for Oak 1.0.29 on the same VM, with the same DB when
performing an oak-run offline compaction. This is on the same VM
with the same repository in the same state. The Tar files are not
the same, but 1 copy of the tar files is 32GB in both instances,
1.4.11 maps 64GB as mentioned before.
I dont know if its the behaviour seen in OAK-4274. I have seen
similar in the past. I was not confident that a GC cycle did unmap,
but it would be logical. Best Regards Ian
On 23 March 2017 at 09:07, Francesco Mari
<mari.france...@gmail.com> wrote:
You might be hitting OAK-4274, which I discovered quite some time
ago. I'm not aware of a way to resolve this issue at the moment.
2017-03-22 16:47 GMT+01:00 Alex Parvulescu
<alex.parvule...@gmail.com>:
Hi,
To give more background this came about during an investigation
into a
slow
offline compaction but it may affect any running FileStore as
well (to
be
verified). I don't think it's related to oak-run itself, but
more with the way we
map
files, and so far it looks like a bug (there is no reasonable
explanation
for mapping each tar file twice).
Took a quick look at the TarReader but there are not many
changes in
this
area 1.0 vs. 1.4 branches. If no one has better ideas, I'll
create an oak issue and investigate
this a
bit further.
thanks, alex
On Wed, Mar 22, 2017 at 4:28 PM, Ian Boston <i...@tfd.co.uk>
wrote:
Hi, I am looking at Oak-run and I see 2x the mapped memory
between 1.0.29
and
1.4.10. It looks like in 1.0.29 each segment file is mapped
into
memory
once, but in 1.4.10 its mapped into memory 2x.
Is this expected ?
Its not great for page faults. Best Regards Ian
