[jira] [Updated] (OAK-1844) Verify resilience goals
[ https://issues.apache.org/jira/browse/OAK-1844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Dürig updated OAK-1844: --- Fix Version/s: (was: 1.2) 1.4 > Verify resilience goals > --- > > Key: OAK-1844 > URL: https://issues.apache.org/jira/browse/OAK-1844 > Project: Jackrabbit Oak > Issue Type: Task >Reporter: Michael Dürig >Assignee: Michael Dürig > Labels: resilience > Fix For: 1.4 > > > This is a container issue for verifying the resilience goals set out for Oak. > See https://wiki.apache.org/jackrabbit/Resilience for the work in progress of > such goals. Once we have an agreement on that, subtasks of this issue could > be used to track the verification process of each of the individual goals. > Discussion here: http://jackrabbit.markmail.org/thread/5cndir5sjrc5dtla -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-1368) Only one Observer per session
[
https://issues.apache.org/jira/browse/OAK-1368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig updated OAK-1368:
---
Fix Version/s: (was: 1.2)
1.4
> Only one Observer per session
> -
>
> Key: OAK-1368
> URL: https://issues.apache.org/jira/browse/OAK-1368
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: jcr
>Reporter: Jukka Zitting
>Assignee: Michael Dürig
> Labels: observation
> Fix For: 1.4
>
>
> As mentioned in OAK-1332, a case where a single session registers multiple
> observation listeners can be troublesome if events are delivered concurrently
> to all of those listeners, since in such a case the {{NamePathMapper}} and
> other session internals will likely suffer from lock contention.
> A good way to avoid this would be to have all the listeners registered within
> a single session be tied to a single {{Observer}} and thus processed
> sequentially.
> Doing so would also improve performance as the listeners could leverage the
> same content diff. As the listeners come from a single session and thus
> presumably from a single client, there's no need to worry about one client
> blocking the work of another.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-2556) do intermediate commit during async indexing
[ https://issues.apache.org/jira/browse/OAK-2556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Egli updated OAK-2556: - Fix Version/s: 1.4 > do intermediate commit during async indexing > > > Key: OAK-2556 > URL: https://issues.apache.org/jira/browse/OAK-2556 > Project: Jackrabbit Oak > Issue Type: Bug > Components: oak-lucene >Affects Versions: 1.0.11 >Reporter: Stefan Egli > Fix For: 1.4 > > > A recent issue found at a customer unveils a potential issue with the async > indexer. Reading the AsyncIndexUpdate.updateIndex it looks like it is doing > the entire update of the async indexer *in one go*, ie in one commit. > When there is - for some reason - however, a huge diff that the async indexer > has to process, the 'one big commit' can become gigantic. There is no limit > to the size of the commit in fact. > So the suggestion is to do intermediate commits while the async indexer is > going on. The reason this is acceptable is the fact that by doing async > indexing, that index is anyway not 100% up-to-date - so it would not make > much of a difference if it would commit after every 100 or 1000 changes > either. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-1327) Cleanup NodeStore and MK implementations
[ https://issues.apache.org/jira/browse/OAK-1327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angela updated OAK-1327: Fix Version/s: (was: 1.2) 1.4 > Cleanup NodeStore and MK implementations > > > Key: OAK-1327 > URL: https://issues.apache.org/jira/browse/OAK-1327 > Project: Jackrabbit Oak > Issue Type: Wish > Components: core, mk, segmentmk >Reporter: angela > Fix For: 1.4 > > Attachments: OAK-1327.patch > > > as discussed during the oak-call today, i would like to cleanup the code base > before we officially release OAK. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2382) Move NodeStore implementations to separate modules
[ https://issues.apache.org/jira/browse/OAK-2382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angela updated OAK-2382: Fix Version/s: (was: 1.2) 1.4 > Move NodeStore implementations to separate modules > -- > > Key: OAK-2382 > URL: https://issues.apache.org/jira/browse/OAK-2382 > Project: Jackrabbit Oak > Issue Type: Sub-task > Components: core, mk, segmentmk >Reporter: angela > Fix For: 1.4 > > > as discussed in the oak-call yesterday, i think we should take another look > at the modularization of the oak-core module. > some time ago i proposed to move the NodeStore implementations into separate > modules. > to begin with i just tried 2 separate modules: > - oak-ns-document: > everything below oak.plugins.document > - oak-ns-segment: > everything below oak.plugins.segment > segment specific > parts of oak.plugins.backup > i found the following issues: > - org.apache.jackrabbit.oak.plugins.cache is not part of the exported > packages - oak.plugins.backup contains both public API and implementations > without separation - the following test-classes have a hard dependency on one > or more ns implementations: > KernelNodeStoreCacheTest > > ClusterPermissionsTest > NodeStoreFixture to fix those we could need to be > able to run the tests with the individual nodestore modules and move those > tests that are just intended to work with a particular impl. > such a move would not only prevent us from introducing unintended package > dependencies but would also reduce the number of dependencies present with > oak-core. > as discussed yesterday we may want to pick this up again this year. > see also http://markmail.org/message/6cpbyuthub4jxase for the whole > discussion. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-1649) NamespaceException: OakNamespace0005 on save, after replica crash
[
https://issues.apache.org/jira/browse/OAK-1649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Egli updated OAK-1649:
-
Fix Version/s: (was: 1.2)
1.4
moving this to 1.2 - this is a rather old issue which would require being
reproduced on a replica set first
> NamespaceException: OakNamespace0005 on save, after replica crash
> -
>
> Key: OAK-1649
> URL: https://issues.apache.org/jira/browse/OAK-1649
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: core, mongomk
>Affects Versions: 0.19
> Environment: 0.20-SNAPSHOT as of March 31
>Reporter: Stefan Egli
> Fix For: 1.4
>
> Attachments: OverwritePropertyTest.java
>
>
> After running a test that produces couple thousands of nodes, and overwrites
> the same properties couple thousand times, then crashing the replica primary,
> the exception below occurs.
> The exception can be reproduced on the db and with the test case I'll attach
> in a minute
> {code}javax.jcr.NamespaceException: OakNamespace0005: Namespace modification
> not allowed: rep:nsdata
> at
> org.apache.jackrabbit.oak.api.CommitFailedException.asRepositoryException(CommitFailedException.java:227)
> at
> org.apache.jackrabbit.oak.api.CommitFailedException.asRepositoryException(CommitFailedException.java:212)
> at
> org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.newRepositoryException(SessionDelegate.java:679)
> at
> org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.save(SessionDelegate.java:553)
> at
> org.apache.jackrabbit.oak.jcr.session.SessionImpl$8.perform(SessionImpl.java:417)
> at
> org.apache.jackrabbit.oak.jcr.session.SessionImpl$8.perform(SessionImpl.java:1)
> at
> org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.perform(SessionDelegate.java:308)
> at
> org.apache.jackrabbit.oak.jcr.session.SessionImpl.perform(SessionImpl.java:127)
> at
> org.apache.jackrabbit.oak.jcr.session.SessionImpl.save(SessionImpl.java:414)
> at
> org.apache.jackrabbit.oak.run.OverwritePropertyTest.testReplicaCrashResilience(OverwritePropertyTest.java:74)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
> at
> org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
> at
> org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
> Caused by: org.apache.jackrabbit.oak.api.CommitFailedException:
> OakNamespace0005: Namespace modification not allowed: rep:nsdata
> at
> org.apache.jackrabbit.oak.plugins.name.NamespaceEditor.modificationNotAllowed(NamespaceEditor.java:122)
> at
> org.apache.jackrabbit.oak.plugins.name.NamespaceEditor.childNodeChanged(NamespaceEditor.java:140)
> at
> org.apache.jackrabbit.oak.spi.commit.CompositeEditor.childNodeChanged(CompositeEditor.java:122)
> at
> org.apache.jackrabbit.oak.spi.commit.Editor
[jira] [Updated] (OAK-2500) checkDeepHistory/fixDeepHistory/prepareDeepHistory for oak-mongo.js
[ https://issues.apache.org/jira/browse/OAK-2500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Egli updated OAK-2500: - Fix Version/s: 1.0.13 > checkDeepHistory/fixDeepHistory/prepareDeepHistory for oak-mongo.js > --- > > Key: OAK-2500 > URL: https://issues.apache.org/jira/browse/OAK-2500 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: run >Affects Versions: 1.0.8 >Reporter: Stefan Egli > Fix For: 1.0.13 > > Attachments: oak-mongo-mod.js > > > The oak-mongo.js currently contains checkHistory/fixHistory which are > cleaning up 'dangling revisions/split-documents' on a particular path. > Now it would be good to have a command that goes through the entire > repository and checks/fixes these dangling revisions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-2500) checkDeepHistory/fixDeepHistory/prepareDeepHistory for oak-mongo.js
[ https://issues.apache.org/jira/browse/OAK-2500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14355194#comment-14355194 ] Stefan Egli commented on OAK-2500: -- [~chetanm], would it make sense to include that in 1.0 branch? (as it affects 1.0.8 if I remember correctly) > checkDeepHistory/fixDeepHistory/prepareDeepHistory for oak-mongo.js > --- > > Key: OAK-2500 > URL: https://issues.apache.org/jira/browse/OAK-2500 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: run >Affects Versions: 1.0.8 >Reporter: Stefan Egli > Fix For: 1.0.13 > > Attachments: oak-mongo-mod.js > > > The oak-mongo.js currently contains checkHistory/fixHistory which are > cleaning up 'dangling revisions/split-documents' on a particular path. > Now it would be good to have a command that goes through the entire > repository and checks/fixes these dangling revisions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2500) checkDeepHistory/fixDeepHistory/prepareDeepHistory for oak-mongo.js
[ https://issues.apache.org/jira/browse/OAK-2500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Egli updated OAK-2500: - Affects Version/s: 1.0.8 > checkDeepHistory/fixDeepHistory/prepareDeepHistory for oak-mongo.js > --- > > Key: OAK-2500 > URL: https://issues.apache.org/jira/browse/OAK-2500 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: run >Affects Versions: 1.0.8 >Reporter: Stefan Egli > Fix For: 1.0.13 > > Attachments: oak-mongo-mod.js > > > The oak-mongo.js currently contains checkHistory/fixHistory which are > cleaning up 'dangling revisions/split-documents' on a particular path. > Now it would be good to have a command that goes through the entire > repository and checks/fixes these dangling revisions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-2480) Incremental (FileStore)Backup copies the entire source instead of just the delta
[ https://issues.apache.org/jira/browse/OAK-2480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14355201#comment-14355201 ] Stefan Egli commented on OAK-2480: -- [~alex.parvulescu], what's your take on this one, something for 1.2 or 1.4 ? thx. > Incremental (FileStore)Backup copies the entire source instead of just the > delta > > > Key: OAK-2480 > URL: https://issues.apache.org/jira/browse/OAK-2480 > Project: Jackrabbit Oak > Issue Type: Bug > Components: run >Affects Versions: 1.1.5 >Reporter: Stefan Egli > Attachments: IncrementalBackupTest.java, > oak-2480.incremental.partial.patch > > > Running the FileStoreBackup (in oak-run) sequentially should correspond to an > incremental backup. This implies the expectation, that the incremental backup > is very resource-friendly, ie that it only adds the delta/diff that changed > since the last backup. Instead what can be een at the moment, is that it > copies the entire source-store again on each 'incremental' backup. > Tested with the latest trunk snapshot. > Suspecting the problem to be as follows: on the first backup the > FileStoreBackup stores a checkpoint created in the source-store and adds it > as a property "checkpoint" to the backup root node, besides the actual backup > which is stored in '/root'. > On subsequent incremental runs, the backup tries to retrieve said property > "checkpoint" from the backup and uses that in the compactor to do the diff > based upon. > Now the problem seems to be that in Compactor.compact it goes to call > process(), which does a writer.writeNode(before) (where before is the > checkpoint in the origin store but writer is a writer of the backup store). > And in this SegmentWriter.writeNode() it fails to find the 'before' segment, > and thus traverses the entire tree and copies it from the origin to the > backup. > So the problem looks to be in the area where it assumes to find this > 'checkpoint-before' in the backup but that's not the case. > So a solution would have been to not do the diff between the checkpoint and > the current origin-head, but between the backup-head and the origin-head > instead. Now apparently this was not the intention though, as that would mean > to read through the entire backup for doing the diffing - and that would be > inefficient... -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-2479) FileStoreBackup throws SegmentNotFoundException
[
https://issues.apache.org/jira/browse/OAK-2479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14355205#comment-14355205
]
Stefan Egli commented on OAK-2479:
--
[~mduerig] what's your take on this one, 1.2 or 1.4 ? thx
> FileStoreBackup throws SegmentNotFoundException
> ---
>
> Key: OAK-2479
> URL: https://issues.apache.org/jira/browse/OAK-2479
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: run
>Affects Versions: 1.1.5, 1.0.11
>Reporter: Stefan Egli
> Attachments: BackupTest.java, oak-2479.cloneBinaries.partial.patch
>
>
> Running the FileStoreBackup (in oak-run) results in a
> SegmentNotFoundException to be thrown.
> Narrowed this down to a regression introduced with
> https://github.com/apache/jackrabbit-oak/commit/6129da4251d36e3f2e1ac4b72ebf3602d0073d47
> The issue seems to be related to the fact that it creates a Compactor with
> the parameter cloneBinaries set to false. This results in blobs not being
> copied by value but rather by reference. Which results in segments not being
> found in the backup (since they only exist in the origin-store).
> Creating the Compactor with cloneBinaries set to true fixes this (not saying
> that this is the correct fix - as I understood at least in online-compaction
> case you want to have cloneBinaries set to false for example).
> Attaching a test case later which reproduces the problem and currently
> results in the following exception (in trunk):
> {code}
> org.apache.jackrabbit.oak.plugins.segment.SegmentNotFoundException: Segment
> 7433ef00-fac5-48d5-b91d-923c517c4a5b not found
> at
> org.apache.jackrabbit.oak.plugins.segment.file.FileStore.readSegment(FileStore.java:711)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentTracker.getSegment(SegmentTracker.java:122)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentId.getSegment(SegmentId.java:108)
> at
> org.apache.jackrabbit.oak.plugins.segment.Record.getSegment(Record.java:82)
> at
> org.apache.jackrabbit.oak.plugins.segment.BlockRecord.read(BlockRecord.java:55)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentStream.read(SegmentStream.java:171)
> at com.google.common.io.ByteStreams.read(ByteStreams.java:828)
> at com.google.common.io.ByteSource.contentEquals(ByteSource.java:303)
> at com.google.common.io.ByteStreams.equal(ByteStreams.java:661)
> at
> org.apache.jackrabbit.oak.plugins.memory.AbstractBlob.equal(AbstractBlob.java:58)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentBlob.equals(SegmentBlob.java:211)
> at
> org.apache.jackrabbit.oak.plugins.segment.Compactor.compact(Compactor.java:229)
> at
> org.apache.jackrabbit.oak.plugins.segment.Compactor.compact(Compactor.java:185)
> at
> org.apache.jackrabbit.oak.plugins.segment.Compactor.access$0(Compactor.java:181)
> at
> org.apache.jackrabbit.oak.plugins.segment.Compactor$CompactDiff.propertyAdded(Compactor.java:115)
> at
> org.apache.jackrabbit.oak.plugins.memory.EmptyNodeState.compareAgainstEmptyState(EmptyNodeState.java:155)
> at
> org.apache.jackrabbit.oak.plugins.segment.Compactor$CompactDiff.childNodeAdded(Compactor.java:137)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentNodeState.compareAgainstBaseState(SegmentNodeState.java:488)
> at
> org.apache.jackrabbit.oak.plugins.segment.Compactor.process(Compactor.java:92)
> at
> org.apache.jackrabbit.oak.plugins.segment.Compactor.compact(Compactor.java:97)
> at
> org.apache.jackrabbit.oak.plugins.backup.FileStoreBackup.backup(FileStoreBackup.java:81)
> at
> org.apache.jackrabbit.oak.run.BackupTest.testInMemoryBackup(BackupTest.java:55)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
> at org.junit.runners.
[jira] [Created] (OAK-2605) Support for additional encodings needed in ReversedLinesFileReader
Leandro Reis created OAK-2605: - Summary: Support for additional encodings needed in ReversedLinesFileReader Key: OAK-2605 URL: https://issues.apache.org/jira/browse/OAK-2605 Project: Jackrabbit Oak Issue Type: Bug Affects Versions: 1.1.7 Environment: Windows 2012 R2 Japanese Windows 2012 R2 Korean Windows 2012 R2 Simplified Chinese Windows 2012 R2 Traditional Chinese Reporter: Leandro Reis Priority: Critical I¹m working on a product that uses Commons IO via Jackrabbit Oak. In the process of testing the launch of such product on Japanese Windows 2012 Server R2, I came across the following exception: "(java.io.UnsupportedEncodingException: Encoding windows-31j is not supported yet (feel free to submit a patch))" windows-31j is the IANA name for Windows code page 932 (Japanese), and is returned by Charset.defaultCharset(), used in org.apache.commons.io.input.ReversedLinesFileReader [0]. This issue can be resolved by adding a check for 'windows-31j' to ReversedLinesFileReader. A patch for this issue was provided in https://issues.apache.org/jira/browse/IO-471 . It also includes changes needed to support Chinese Simplified, Chinese Traditional and Korean. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2605) Support for additional encodings needed in ReversedLinesFileReader
[ https://issues.apache.org/jira/browse/OAK-2605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leandro Reis updated OAK-2605: -- Description: I'm working on a product that uses Commons IO via Jackrabbit Oak. In the process of testing the launch of such product on Japanese Windows 2012 Server R2, I came across the following exception: "(java.io.UnsupportedEncodingException: Encoding windows-31j is not supported yet (feel free to submit a patch))" windows-31j is the IANA name for Windows code page 932 (Japanese), and is returned by Charset.defaultCharset(), used in org.apache.commons.io.input.ReversedLinesFileReader [0]. A patch for this issue was provided in https://issues.apache.org/jira/browse/IO-471 . It also includes changes needed to support Chinese Simplified, Chinese Traditional and Korean. was: I¹m working on a product that uses Commons IO via Jackrabbit Oak. In the process of testing the launch of such product on Japanese Windows 2012 Server R2, I came across the following exception: "(java.io.UnsupportedEncodingException: Encoding windows-31j is not supported yet (feel free to submit a patch))" windows-31j is the IANA name for Windows code page 932 (Japanese), and is returned by Charset.defaultCharset(), used in org.apache.commons.io.input.ReversedLinesFileReader [0]. This issue can be resolved by adding a check for 'windows-31j' to ReversedLinesFileReader. A patch for this issue was provided in https://issues.apache.org/jira/browse/IO-471 . It also includes changes needed to support Chinese Simplified, Chinese Traditional and Korean. > Support for additional encodings needed in ReversedLinesFileReader > -- > > Key: OAK-2605 > URL: https://issues.apache.org/jira/browse/OAK-2605 > Project: Jackrabbit Oak > Issue Type: Bug >Affects Versions: 1.1.7 > Environment: Windows 2012 R2 Japanese > Windows 2012 R2 Korean > Windows 2012 R2 Simplified Chinese > Windows 2012 R2 Traditional Chinese >Reporter: Leandro Reis >Priority: Critical > > I'm working on a product that uses Commons IO via Jackrabbit Oak. In the > process of testing the launch of such product on Japanese Windows 2012 > Server R2, I came across the following exception: > "(java.io.UnsupportedEncodingException: Encoding windows-31j is not supported > yet (feel free to submit a patch))" > windows-31j is the IANA name for Windows code page 932 (Japanese), and is > returned by Charset.defaultCharset(), used in > org.apache.commons.io.input.ReversedLinesFileReader [0]. > A patch for this issue was provided in > https://issues.apache.org/jira/browse/IO-471 . > It also includes changes needed to support Chinese Simplified, Chinese > Traditional and Korean. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2605) Support for additional encodings needed in ReversedLinesFileReader
[ https://issues.apache.org/jira/browse/OAK-2605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leandro Reis updated OAK-2605: -- Priority: Major (was: Critical) > Support for additional encodings needed in ReversedLinesFileReader > -- > > Key: OAK-2605 > URL: https://issues.apache.org/jira/browse/OAK-2605 > Project: Jackrabbit Oak > Issue Type: Bug >Affects Versions: 1.1.7 > Environment: Windows 2012 R2 Japanese > Windows 2012 R2 Korean > Windows 2012 R2 Simplified Chinese > Windows 2012 R2 Traditional Chinese >Reporter: Leandro Reis > > I'm working on a product that uses Commons IO via Jackrabbit Oak. In the > process of testing the launch of such product on Japanese Windows 2012 > Server R2, I came across the following exception: > "(java.io.UnsupportedEncodingException: Encoding windows-31j is not supported > yet (feel free to submit a patch))" > windows-31j is the IANA name for Windows code page 932 (Japanese), and is > returned by Charset.defaultCharset(), used in > org.apache.commons.io.input.ReversedLinesFileReader [0]. > A patch for this issue was provided in > https://issues.apache.org/jira/browse/IO-471 . > It also includes changes needed to support Chinese Simplified, Chinese > Traditional and Korean. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2582) RDB: improve memory cache handling
[ https://issues.apache.org/jira/browse/OAK-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Reschke updated OAK-2582: Fix Version/s: 1.0.13 > RDB: improve memory cache handling > -- > > Key: OAK-2582 > URL: https://issues.apache.org/jira/browse/OAK-2582 > Project: Jackrabbit Oak > Issue Type: Bug > Components: rdbmk >Affects Versions: 1.0.11, 1.1.6 >Reporter: Julian Reschke >Assignee: Julian Reschke > Fix For: 1.1.8, 1.0.13 > > > Improve memory cache handling: > - invalidateCache should just mark cache entries as to be revalidated > - to-be revalidated cache entries can be used for conditional retrieval from > DB -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-2557) VersionGC uses way too much memory if there is a large pile of garbage
[
https://issues.apache.org/jira/browse/OAK-2557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14355322#comment-14355322
]
Stefan Egli commented on OAK-2557:
--
{{}}
we could make the VersionGC play nicely to existing load on the system: it
could progress slower if the load is higher and vice-verca. One simple measure
could be: if the observation queue is small (eg below 10) then the load is low
and it could progress full-speed. Otherwise it could add some artificial
sleeping in between.
{{}}
> VersionGC uses way too much memory if there is a large pile of garbage
> --
>
> Key: OAK-2557
> URL: https://issues.apache.org/jira/browse/OAK-2557
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: core, mongomk
>Affects Versions: 1.0.11
>Reporter: Stefan Egli
>Assignee: Chetan Mehrotra
>Priority: Blocker
> Fix For: 1.0.13, 1.2
>
> Attachments: OAK-2557.patch
>
>
> It has been noticed that on a system where revision-gc
> (VersionGarbageCollector of mongomk) did not run for a few days (due to not
> interfering with some tests/large bulk operations) that there was such a
> large pile of garbage accumulating, that the following code
> {code}
> VersionGarbageCollector.collectDeletedDocuments
> {code}
> in the for loop, creates such a large list of NodeDocuments to delete
> (docIdsToDelete) that it uses up too much memory, causing the JVM's GC to
> constantly spin in Full-GCs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-1453) MongoMK failover support for replica sets (esp. shards)
[ https://issues.apache.org/jira/browse/OAK-1453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Marth updated OAK-1453: --- Fix Version/s: (was: 1.2) 1.4 > MongoMK failover support for replica sets (esp. shards) > --- > > Key: OAK-1453 > URL: https://issues.apache.org/jira/browse/OAK-1453 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: mongomk >Reporter: Michael Marth > Labels: production, resilience > Fix For: 1.4 > > > With OAK-759 we have introduced replica support in MongoMK. I think we still > need to address the resilience for failover from primary to secoandary: > Consider a case where Oak writes to the primary. Replication to secondary is > ongoing. During that period the primary goes down and the secondary becomes > primary. There could be some "half-replicated" MVCC revisions, which need to > be either discarded or be ignored after the failover. > This might not be an issue if there is only one shard, as the commit root is > written last (and replicated last) > But with 2 shards the the replication state of these 2 shards could be > inconsistent. Oak needs to handle such a situation without falling over. > If we can detect a Mongo failover we could query Mongo which revisions are > fully replicated to the new primary and discard the potentially > half-replicated revisions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (OAK-1449) Troubleshooting tool to inspect hidden items
[ https://issues.apache.org/jira/browse/OAK-1449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Marth resolved OAK-1449. Resolution: Won't Fix Fix Version/s: (was: 1.2) > Troubleshooting tool to inspect hidden items > > > Key: OAK-1449 > URL: https://issues.apache.org/jira/browse/OAK-1449 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core >Reporter: Michael Marth >Priority: Minor > Labels: production, resilience, tools > > For troubleshooting borked instances we should have a way to inspect (and > write to?) the complete tree, including items that are hidden. Permission > tree, version tree, indexes, etc come to mind. > We need to design this tool in a way that does not compromise security, but > on the other hand I think we need write capabilities in order to fix broken > items if needed. Not sure, how to best do this... > The tool ideally can be limited to look at > * specific paths only and > * ranges of MVCC revisions -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (OAK-1450) Run repair tool if repo cannot start (automatically or explicitly)
[ https://issues.apache.org/jira/browse/OAK-1450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Marth resolved OAK-1450. Resolution: Won't Fix > Run repair tool if repo cannot start (automatically or explicitly) > -- > > Key: OAK-1450 > URL: https://issues.apache.org/jira/browse/OAK-1450 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: segmentmk >Reporter: Michael Marth >Priority: Minor > Labels: production, resilience, tools > Fix For: 1.2 > > > If the repo does not come up should we automatically run the repair tool > (OAK-1446) and then retry...? Not sure about this one. Maybe starting the > repo again with an explicit flag "--repair" would be better. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-1451) Expose index size
[ https://issues.apache.org/jira/browse/OAK-1451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Marth updated OAK-1451: --- Fix Version/s: (was: 1.2) 1.3.1 > Expose index size > - > > Key: OAK-1451 > URL: https://issues.apache.org/jira/browse/OAK-1451 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: query >Reporter: Michael Marth >Priority: Minor > Labels: production, resilience > Fix For: 1.3.1 > > > At the moment some MK's disc needs are largely from the indexes. Maybe we can > do something about this, but in the meantime it would be helpful if we could > expose the index sizes (num of indexed nodes) via JMX so that they could be > easily monitored. > This would also be helpful to see at which point an index becomes useless (if > the majority of content nodes are indexed one might as well not have an index) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-1451) Expose index size
[ https://issues.apache.org/jira/browse/OAK-1451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14355351#comment-14355351 ] Michael Marth commented on OAK-1451: [~tmueller], afair we now use the index size to in the query planning. Maybe we can implement this MBean in 1.3.x? > Expose index size > - > > Key: OAK-1451 > URL: https://issues.apache.org/jira/browse/OAK-1451 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: query >Reporter: Michael Marth >Priority: Minor > Labels: production, resilience > Fix For: 1.3.1 > > > At the moment some MK's disc needs are largely from the indexes. Maybe we can > do something about this, but in the meantime it would be helpful if we could > expose the index sizes (num of indexed nodes) via JMX so that they could be > easily monitored. > This would also be helpful to see at which point an index becomes useless (if > the majority of content nodes are indexed one might as well not have an index) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2605) Support for additional encodings needed in ReversedLinesFileReader
[ https://issues.apache.org/jira/browse/OAK-2605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Dürig updated OAK-2605: --- Fix Version/s: 1.1.8 > Support for additional encodings needed in ReversedLinesFileReader > -- > > Key: OAK-2605 > URL: https://issues.apache.org/jira/browse/OAK-2605 > Project: Jackrabbit Oak > Issue Type: Bug > Components: segmentmk >Affects Versions: 1.1.7 > Environment: Windows 2012 R2 Japanese > Windows 2012 R2 Korean > Windows 2012 R2 Simplified Chinese > Windows 2012 R2 Traditional Chinese >Reporter: Leandro Reis >Assignee: Michael Dürig > Fix For: 1.1.8 > > > I'm working on a product that uses Commons IO via Jackrabbit Oak. In the > process of testing the launch of such product on Japanese Windows 2012 > Server R2, I came across the following exception: > "(java.io.UnsupportedEncodingException: Encoding windows-31j is not supported > yet (feel free to submit a patch))" > windows-31j is the IANA name for Windows code page 932 (Japanese), and is > returned by Charset.defaultCharset(), used in > org.apache.commons.io.input.ReversedLinesFileReader [0]. > A patch for this issue was provided in > https://issues.apache.org/jira/browse/IO-471 . > It also includes changes needed to support Chinese Simplified, Chinese > Traditional and Korean. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (OAK-2605) Support for additional encodings needed in ReversedLinesFileReader
[ https://issues.apache.org/jira/browse/OAK-2605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Dürig reassigned OAK-2605: -- Assignee: Michael Dürig > Support for additional encodings needed in ReversedLinesFileReader > -- > > Key: OAK-2605 > URL: https://issues.apache.org/jira/browse/OAK-2605 > Project: Jackrabbit Oak > Issue Type: Bug > Components: segmentmk >Affects Versions: 1.1.7 > Environment: Windows 2012 R2 Japanese > Windows 2012 R2 Korean > Windows 2012 R2 Simplified Chinese > Windows 2012 R2 Traditional Chinese >Reporter: Leandro Reis >Assignee: Michael Dürig > Fix For: 1.1.8 > > > I'm working on a product that uses Commons IO via Jackrabbit Oak. In the > process of testing the launch of such product on Japanese Windows 2012 > Server R2, I came across the following exception: > "(java.io.UnsupportedEncodingException: Encoding windows-31j is not supported > yet (feel free to submit a patch))" > windows-31j is the IANA name for Windows code page 932 (Japanese), and is > returned by Charset.defaultCharset(), used in > org.apache.commons.io.input.ReversedLinesFileReader [0]. > A patch for this issue was provided in > https://issues.apache.org/jira/browse/IO-471 . > It also includes changes needed to support Chinese Simplified, Chinese > Traditional and Korean. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2605) Support for additional encodings needed in ReversedLinesFileReader
[
https://issues.apache.org/jira/browse/OAK-2605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig updated OAK-2605:
---
Attachment: OAK-2605.patch
Proposed patch [^OAK-2605.patch].
The patch copies {{ReversedLinesFileReader}} from {{commons-io}} including the
fix for IO-471.
[~lreis], could you quickly cross check the patch whether I correctly
incorporated all your changes. Also do you think we could add a unit test for
this? Could you attach one to IO-471 if so? I would then port it over to Oak
until we get IO-471 fixed.
> Support for additional encodings needed in ReversedLinesFileReader
> --
>
> Key: OAK-2605
> URL: https://issues.apache.org/jira/browse/OAK-2605
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segmentmk
>Affects Versions: 1.1.7
> Environment: Windows 2012 R2 Japanese
> Windows 2012 R2 Korean
> Windows 2012 R2 Simplified Chinese
> Windows 2012 R2 Traditional Chinese
>Reporter: Leandro Reis
>Assignee: Michael Dürig
> Fix For: 1.1.8
>
> Attachments: OAK-2605.patch
>
>
> I'm working on a product that uses Commons IO via Jackrabbit Oak. In the
> process of testing the launch of such product on Japanese Windows 2012
> Server R2, I came across the following exception:
> "(java.io.UnsupportedEncodingException: Encoding windows-31j is not supported
> yet (feel free to submit a patch))"
> windows-31j is the IANA name for Windows code page 932 (Japanese), and is
> returned by Charset.defaultCharset(), used in
> org.apache.commons.io.input.ReversedLinesFileReader [0].
> A patch for this issue was provided in
> https://issues.apache.org/jira/browse/IO-471 .
> It also includes changes needed to support Chinese Simplified, Chinese
> Traditional and Korean.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-2605) Support for additional encodings needed in ReversedLinesFileReader
[ https://issues.apache.org/jira/browse/OAK-2605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14356107#comment-14356107 ] Leandro Reis commented on OAK-2605: --- @Michael Dürig, your patch correctly includes my changes. There are 3 unit tests (*) for the ReversedLinesFileReader class, I uploaded a patch to https://issues.apache.org/jira/browse/IO-471 that adds tests for the 4 encodings I added support for, as well as test files. (*) http://svn.apache.org/viewvc/commons/proper/io/trunk/src/test/java/org/apache/commons/io/input/ReversedLinesFileReaderTestSimple.java http://svn.apache.org/viewvc/commons/proper/io/trunk/src/test/java/org/apache/commons/io/input/ReversedLinesFileReaderTestParamFile.java http://svn.apache.org/viewvc/commons/proper/io/trunk/src/test/java/org/apache/commons/io/input/ReversedLinesFileReaderTestParamBlockSize.java test files - http://svn.apache.org/viewvc/commons/proper/io/trunk/src/test/resources/ > Support for additional encodings needed in ReversedLinesFileReader > -- > > Key: OAK-2605 > URL: https://issues.apache.org/jira/browse/OAK-2605 > Project: Jackrabbit Oak > Issue Type: Bug > Components: segmentmk >Affects Versions: 1.1.7 > Environment: Windows 2012 R2 Japanese > Windows 2012 R2 Korean > Windows 2012 R2 Simplified Chinese > Windows 2012 R2 Traditional Chinese >Reporter: Leandro Reis >Assignee: Michael Dürig > Fix For: 1.1.8 > > Attachments: OAK-2605.patch > > > I'm working on a product that uses Commons IO via Jackrabbit Oak. In the > process of testing the launch of such product on Japanese Windows 2012 > Server R2, I came across the following exception: > "(java.io.UnsupportedEncodingException: Encoding windows-31j is not supported > yet (feel free to submit a patch))" > windows-31j is the IANA name for Windows code page 932 (Japanese), and is > returned by Charset.defaultCharset(), used in > org.apache.commons.io.input.ReversedLinesFileReader [0]. > A patch for this issue was provided in > https://issues.apache.org/jira/browse/IO-471 . > It also includes changes needed to support Chinese Simplified, Chinese > Traditional and Korean. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (OAK-2605) Support for additional encodings needed in ReversedLinesFileReader
[ https://issues.apache.org/jira/browse/OAK-2605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14356107#comment-14356107 ] Leandro Reis edited comment on OAK-2605 at 3/11/15 2:16 AM: @Michael Dürig, your patch correctly includes my changes. There are 3 unit tests [*] for the ReversedLinesFileReader class, I uploaded a patch to https://issues.apache.org/jira/browse/IO-471 that adds tests for the 4 encodings I added support for, as well as test files. [*] http://svn.apache.org/viewvc/commons/proper/io/trunk/src/test/java/org/apache/commons/io/input/ReversedLinesFileReaderTestSimple.java http://svn.apache.org/viewvc/commons/proper/io/trunk/src/test/java/org/apache/commons/io/input/ReversedLinesFileReaderTestParamFile.java http://svn.apache.org/viewvc/commons/proper/io/trunk/src/test/java/org/apache/commons/io/input/ReversedLinesFileReaderTestParamBlockSize.java test files - http://svn.apache.org/viewvc/commons/proper/io/trunk/src/test/resources/ was (Author: lreis): @Michael Dürig, your patch correctly includes my changes. There are 3 unit tests (*) for the ReversedLinesFileReader class, I uploaded a patch to https://issues.apache.org/jira/browse/IO-471 that adds tests for the 4 encodings I added support for, as well as test files. (*) http://svn.apache.org/viewvc/commons/proper/io/trunk/src/test/java/org/apache/commons/io/input/ReversedLinesFileReaderTestSimple.java http://svn.apache.org/viewvc/commons/proper/io/trunk/src/test/java/org/apache/commons/io/input/ReversedLinesFileReaderTestParamFile.java http://svn.apache.org/viewvc/commons/proper/io/trunk/src/test/java/org/apache/commons/io/input/ReversedLinesFileReaderTestParamBlockSize.java test files - http://svn.apache.org/viewvc/commons/proper/io/trunk/src/test/resources/ > Support for additional encodings needed in ReversedLinesFileReader > -- > > Key: OAK-2605 > URL: https://issues.apache.org/jira/browse/OAK-2605 > Project: Jackrabbit Oak > Issue Type: Bug > Components: segmentmk >Affects Versions: 1.1.7 > Environment: Windows 2012 R2 Japanese > Windows 2012 R2 Korean > Windows 2012 R2 Simplified Chinese > Windows 2012 R2 Traditional Chinese >Reporter: Leandro Reis >Assignee: Michael Dürig > Fix For: 1.1.8 > > Attachments: OAK-2605.patch > > > I'm working on a product that uses Commons IO via Jackrabbit Oak. In the > process of testing the launch of such product on Japanese Windows 2012 > Server R2, I came across the following exception: > "(java.io.UnsupportedEncodingException: Encoding windows-31j is not supported > yet (feel free to submit a patch))" > windows-31j is the IANA name for Windows code page 932 (Japanese), and is > returned by Charset.defaultCharset(), used in > org.apache.commons.io.input.ReversedLinesFileReader [0]. > A patch for this issue was provided in > https://issues.apache.org/jira/browse/IO-471 . > It also includes changes needed to support Chinese Simplified, Chinese > Traditional and Korean. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (OAK-2605) Support for additional encodings needed in ReversedLinesFileReader
[ https://issues.apache.org/jira/browse/OAK-2605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14356107#comment-14356107 ] Leandro Reis edited comment on OAK-2605 at 3/11/15 2:17 AM: [~mduerig], your patch correctly includes my changes. There are 3 unit tests [*] for the ReversedLinesFileReader class, I uploaded a patch to https://issues.apache.org/jira/browse/IO-471 that adds tests for the 4 encodings I added support for, as well as test files. [*] http://svn.apache.org/viewvc/commons/proper/io/trunk/src/test/java/org/apache/commons/io/input/ReversedLinesFileReaderTestSimple.java http://svn.apache.org/viewvc/commons/proper/io/trunk/src/test/java/org/apache/commons/io/input/ReversedLinesFileReaderTestParamFile.java http://svn.apache.org/viewvc/commons/proper/io/trunk/src/test/java/org/apache/commons/io/input/ReversedLinesFileReaderTestParamBlockSize.java test files - http://svn.apache.org/viewvc/commons/proper/io/trunk/src/test/resources/ was (Author: lreis): @Michael Dürig, your patch correctly includes my changes. There are 3 unit tests [*] for the ReversedLinesFileReader class, I uploaded a patch to https://issues.apache.org/jira/browse/IO-471 that adds tests for the 4 encodings I added support for, as well as test files. [*] http://svn.apache.org/viewvc/commons/proper/io/trunk/src/test/java/org/apache/commons/io/input/ReversedLinesFileReaderTestSimple.java http://svn.apache.org/viewvc/commons/proper/io/trunk/src/test/java/org/apache/commons/io/input/ReversedLinesFileReaderTestParamFile.java http://svn.apache.org/viewvc/commons/proper/io/trunk/src/test/java/org/apache/commons/io/input/ReversedLinesFileReaderTestParamBlockSize.java test files - http://svn.apache.org/viewvc/commons/proper/io/trunk/src/test/resources/ > Support for additional encodings needed in ReversedLinesFileReader > -- > > Key: OAK-2605 > URL: https://issues.apache.org/jira/browse/OAK-2605 > Project: Jackrabbit Oak > Issue Type: Bug > Components: segmentmk >Affects Versions: 1.1.7 > Environment: Windows 2012 R2 Japanese > Windows 2012 R2 Korean > Windows 2012 R2 Simplified Chinese > Windows 2012 R2 Traditional Chinese >Reporter: Leandro Reis >Assignee: Michael Dürig > Fix For: 1.1.8 > > Attachments: OAK-2605.patch > > > I'm working on a product that uses Commons IO via Jackrabbit Oak. In the > process of testing the launch of such product on Japanese Windows 2012 > Server R2, I came across the following exception: > "(java.io.UnsupportedEncodingException: Encoding windows-31j is not supported > yet (feel free to submit a patch))" > windows-31j is the IANA name for Windows code page 932 (Japanese), and is > returned by Charset.defaultCharset(), used in > org.apache.commons.io.input.ReversedLinesFileReader [0]. > A patch for this issue was provided in > https://issues.apache.org/jira/browse/IO-471 . > It also includes changes needed to support Chinese Simplified, Chinese > Traditional and Korean. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-2557) VersionGC uses way too much memory if there is a large pile of garbage
[
https://issues.apache.org/jira/browse/OAK-2557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14356267#comment-14356267
]
Chetan Mehrotra commented on OAK-2557:
--
Version GC currently ensures that query fired is made against the Secondary (if
present). However having some throttling in such background task would be good
thing to have. But first we need to have some {{SystemLoadIndicator}} notion in
Oak which can be provide details say in percentage 1..100 about system load. We
can then expose configurable threshold which VersionGC would listen for and
adjust its working accordingly.
It can be a JMX bean which emits notification and we have our components listen
to those notification (or use OSGi SR/Events). That can be used in other places
like Observation processing, Blob GC etc
> VersionGC uses way too much memory if there is a large pile of garbage
> --
>
> Key: OAK-2557
> URL: https://issues.apache.org/jira/browse/OAK-2557
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: core, mongomk
>Affects Versions: 1.0.11
>Reporter: Stefan Egli
>Assignee: Chetan Mehrotra
>Priority: Blocker
> Fix For: 1.0.13, 1.2
>
> Attachments: OAK-2557.patch
>
>
> It has been noticed that on a system where revision-gc
> (VersionGarbageCollector of mongomk) did not run for a few days (due to not
> interfering with some tests/large bulk operations) that there was such a
> large pile of garbage accumulating, that the following code
> {code}
> VersionGarbageCollector.collectDeletedDocuments
> {code}
> in the for loop, creates such a large list of NodeDocuments to delete
> (docIdsToDelete) that it uses up too much memory, causing the JVM's GC to
> constantly spin in Full-GCs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (OAK-2557) VersionGC uses way too much memory if there is a large pile of garbage
[
https://issues.apache.org/jira/browse/OAK-2557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14352867#comment-14352867
]
Chetan Mehrotra edited comment on OAK-2557 at 3/11/15 4:25 AM:
---
After an offline discussion with [~mreutegg] we come to following conclusion
# Deletion logic has to work like creation logic and has to perform deletion in
bottom to top way. Creation currently works from top to bottom i.e. ensures
that parent get created first and child thereafter. So deletion logic has to
complement that
# Current logic also has a potential issue where if the system performing GC
crashes in between then it might lead to a state where parent would have got
removed before child and in that case such child document can never be GCed. So
as a fix we should first sort the batch via {{PathComparator}} in reverse and
then perform deletion from child to parent. -{color:brown}Open a new issue for
that{color}- OAK-2603
# For large deletion (like current case) we make use of {{ExternalSort}} where
sorting is performed on disk and we read for paths stored in file. This would
make use of all the support developed in Blob GC in
{{MarkSweepGarbageCollector}}
All in all this would not be a simple fix that I initially thought :(
was (Author: chetanm):
After an offline discussion with [~mreutegg] we come to following conclusion
# Deletion logic has to work like creation logic and has to perform deletion in
bottom to top way. Creation currently works from top to bottom i.e. ensures
that parent get created first and child thereafter. So deletion logic has to
complement that
# Current logic also has a potential issue where if the system performing GC
crashes in between then it might lead to a state where parent would have got
removed before child and in that case such child document can never be GCed. So
as a fix we should first sort the batch via {{PathComparator}} in reverse and
then perform deletion from child to parent. {color:brown}Open a new issue for
that{color}
# For large deletion (like current case) we make use of {{ExternalSort}} where
sorting is performed on disk and we read for paths stored in file. This would
make use of all the support developed in Blob GC in
{{MarkSweepGarbageCollector}}
All in all this would not be a simple fix that I initially thought :(
> VersionGC uses way too much memory if there is a large pile of garbage
> --
>
> Key: OAK-2557
> URL: https://issues.apache.org/jira/browse/OAK-2557
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: core, mongomk
>Affects Versions: 1.0.11
>Reporter: Stefan Egli
>Assignee: Chetan Mehrotra
>Priority: Blocker
> Fix For: 1.0.13, 1.2
>
> Attachments: OAK-2557.patch
>
>
> It has been noticed that on a system where revision-gc
> (VersionGarbageCollector of mongomk) did not run for a few days (due to not
> interfering with some tests/large bulk operations) that there was such a
> large pile of garbage accumulating, that the following code
> {code}
> VersionGarbageCollector.collectDeletedDocuments
> {code}
> in the for loop, creates such a large list of NodeDocuments to delete
> (docIdsToDelete) that it uses up too much memory, causing the JVM's GC to
> constantly spin in Full-GCs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (OAK-2603) Failure in one of the batch in VersionGC might lead to orphaned nodes
[
https://issues.apache.org/jira/browse/OAK-2603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chetan Mehrotra resolved OAK-2603.
--
Resolution: Fixed
Fixed the issue by sorting the list before passing to {{DocumentStore}} for
deletion. This ensures that child get deleted before parent
* trunk - http://svn.apache.org/r1665758
* 1.0 - http://svn.apache.org/r1665759
> Failure in one of the batch in VersionGC might lead to orphaned nodes
> -
>
> Key: OAK-2603
> URL: https://issues.apache.org/jira/browse/OAK-2603
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: mongomk
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
> Fix For: 1.1.8, 1.0.13
>
>
> VersionGC logic currently performs deletion of nodes in batches. For GC to
> work properly NodeDocument should always be removed in bottom-up mode i.e.
> parent node should be removed *after* child has been removed
> Currently the GC logic deletes the NodeDocument in undefined order. In such
> mode if one of the batch fails then its possible that parent might have got
> deleted but the child was not deleted.
> Now in next run the child node would not be recognized as a deleted node
> because the commit root would not be found.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Reopened] (OAK-2603) Failure in one of the batch in VersionGC might lead to orphaned nodes
[ https://issues.apache.org/jira/browse/OAK-2603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Mehrotra reopened OAK-2603: -- Reopening as the fix used incorrect comparator i.e. using PathComparator for Ids > Failure in one of the batch in VersionGC might lead to orphaned nodes > - > > Key: OAK-2603 > URL: https://issues.apache.org/jira/browse/OAK-2603 > Project: Jackrabbit Oak > Issue Type: Bug > Components: mongomk >Reporter: Chetan Mehrotra >Assignee: Chetan Mehrotra > Fix For: 1.1.8, 1.0.13 > > > VersionGC logic currently performs deletion of nodes in batches. For GC to > work properly NodeDocument should always be removed in bottom-up mode i.e. > parent node should be removed *after* child has been removed > Currently the GC logic deletes the NodeDocument in undefined order. In such > mode if one of the batch fails then its possible that parent might have got > deleted but the child was not deleted. > Now in next run the child node would not be recognized as a deleted node > because the commit root would not be found. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
