[jira] [Created] (OAK-7287) Public JcrNameParser.parse() requires package protected Listener
Marcel Reutegger created OAK-7287: - Summary: Public JcrNameParser.parse() requires package protected Listener Key: OAK-7287 URL: https://issues.apache.org/jira/browse/OAK-7287 Project: Jackrabbit Oak Issue Type: Bug Components: core-spi Affects Versions: 1.8.0 Reporter: Marcel Reutegger For consistency either the method should be made package private or the listener public. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OAK-7286) DocumentNodeStoreBranch handling of non-recoverable DocumentStoreExceptions
[
https://issues.apache.org/jira/browse/OAK-7286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Reschke updated OAK-7286:
Description:
In {{DocumentNodeStoreBranch.merge()}}, any {{DocumentStoreException}} is
mapped to a {{DocumentStoreException}} to a {{CommitFailedException}} of type
"MERGE", which leads to the operation being retried, and a non-helpful
exception being generated.
The effect can be observed by enabling a test in {{ValidNamesTest}}:
{noformat}
--- oak-jcr/src/test/java/org/apache/jackrabbit/oak/jcr/ValidNamesTest.java
(Revision 1825371)
+++ oak-jcr/src/test/java/org/apache/jackrabbit/oak/jcr/ValidNamesTest.java
(Arbeitskopie)
@@ -300,7 +300,6 @@
public void testUnpairedHighSurrogateEnd() {
// see OAK-5506
org.junit.Assume.assumeFalse(super.fixture.toString().toLowerCase().contains("segment"));
-
org.junit.Assume.assumeFalse(super.fixture.toString().toLowerCase().contains("rdb"));
nameTest("foo" + SURROGATE_PAIR[0]);
}
@@ -336,6 +335,7 @@
assertEquals("paths should be equal", p.getPath(), n.getPath());
return p;
} catch (RepositoryException ex) {
+ex.printStackTrace();
fail(ex.getMessage());
return null;
}
{noformat}
The underlying issue is that {{RDBDocumentStore}} is throwing a
{{DocumentStoreException}} due to the invalid ID, and repeating the call will
not help.
We probably should have a way to dinstinguish between different types of
problems.
I hacked {{DocumentNodeStoreBranch}} like that:
{noformat}
---
oak-store-document/src/main/java/org/apache/jackrabbit/oak/plugins/document/DocumentNodeStoreBranch.java
(Revision 1825371)
+++
oak-store-document/src/main/java/org/apache/jackrabbit/oak/plugins/document/DocumentNodeStoreBranch.java
(Arbeitskopie)
@@ -520,8 +520,12 @@
} catch (ConflictException e) {
throw e.asCommitFailedException();
} catch(DocumentStoreException e) {
-throw new CommitFailedException(MERGE, 1,
-"Failed to merge changes to the underlying store",
e);
+if (e.getMessage().contains("Invalid ID")) {
+throw new CommitFailedException(OAK, 123,
+"Failed to store changes in the underlying
store: " + e.getMessage(), e);
+} else {
+throw new CommitFailedException(MERGE, 1, "Failed to
merge changes to the underlying store", e);
+}
} catch (Exception e) {
throw new CommitFailedException(OAK, 1,
"Failed to merge changes to the underlying store",
e);
{noformat}
...which causes the exception to surface immediately (see
https://issues.apache.org/jira/secure/attachment/12912117/OAK-7286.diff).
(cc [~mreutegg])
was:
In {{DocumentNodeStoreBranch.merge()}}, any {{DocumentStoreException}} is
mapped to a {{DocumentStoreException}} to a {{CommitFailedException}} of type
"MERGE", which leads to the operation being retried, and a non-helpful
exception being generated.
The effect can be observed by enabling a test in {{ValidNamesTest}}:
{noformat}
--- oak-jcr/src/test/java/org/apache/jackrabbit/oak/jcr/ValidNamesTest.java
(Revision 1825371)
+++ oak-jcr/src/test/java/org/apache/jackrabbit/oak/jcr/ValidNamesTest.java
(Arbeitskopie)
@@ -300,7 +300,6 @@
public void testUnpairedHighSurrogateEnd() {
// see OAK-5506
org.junit.Assume.assumeFalse(super.fixture.toString().toLowerCase().contains("segment"));
-
org.junit.Assume.assumeFalse(super.fixture.toString().toLowerCase().contains("rdb"));
nameTest("foo" + SURROGATE_PAIR[0]);
}
@@ -336,6 +335,7 @@
assertEquals("paths should be equal", p.getPath(), n.getPath());
return p;
} catch (RepositoryException ex) {
+ex.printStackTrace();
fail(ex.getMessage());
return null;
}
{noformat}
The underlying issue is that {{RDBDocumentStore}} is throwing a
{{DocumentStoreException}} due to the invalid ID, and repeating the call will
not help.
We probably should have a way to dinstinguish between different types of
problems.
I hacked {{DocumentNodeStoreBranch}} like that:
{noformat}
---
oak-store-document/src/main/java/org/apache/jackrabbit/oak/plugins/document/DocumentNodeStoreBranch.java
(Revision 1825371)
+++
oak-store-document/src/main/java/org/apache/jackrabbit/oak/plugins/document/DocumentNodeStoreBranch.java
(Arbeitskopie)
@@ -520,8 +520,12 @@
} catch (ConflictException e) {
throw e.asCommitFailedException();
} catch(DocumentStoreException e) {
-throw new
[jira] [Updated] (OAK-7286) DocumentNodeStoreBranch handling of non-recoverable DocumentStoreExceptions
[
https://issues.apache.org/jira/browse/OAK-7286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Reschke updated OAK-7286:
Description:
In {{DocumentNodeStoreBranch.merge()}}, any {{DocumentStoreException}} is
mapped to a {{DocumentStoreException}} to a {{CommitFailedException}} of type
"MERGE", which leads to the operation being retried, and a non-helpful
exception being generated.
The effect can be observed by enabling a test in {{ValidNamesTest}}:
{noformat}
--- oak-jcr/src/test/java/org/apache/jackrabbit/oak/jcr/ValidNamesTest.java
(Revision 1825371)
+++ oak-jcr/src/test/java/org/apache/jackrabbit/oak/jcr/ValidNamesTest.java
(Arbeitskopie)
@@ -300,7 +300,6 @@
public void testUnpairedHighSurrogateEnd() {
// see OAK-5506
org.junit.Assume.assumeFalse(super.fixture.toString().toLowerCase().contains("segment"));
-
org.junit.Assume.assumeFalse(super.fixture.toString().toLowerCase().contains("rdb"));
nameTest("foo" + SURROGATE_PAIR[0]);
}
@@ -336,6 +335,7 @@
assertEquals("paths should be equal", p.getPath(), n.getPath());
return p;
} catch (RepositoryException ex) {
+ex.printStackTrace();
fail(ex.getMessage());
return null;
}
{noformat}
The underlying issue is that {{RDBDocumentStore}} is throwing a
{{DocumentStoreException}} due to the invalid ID, and repeating the call will
not help.
We probably should have a way to dinstinguish between different types of
problems.
I hacked {{DocumentNodeStoreBranch}} like that:
{noformat}
---
oak-store-document/src/main/java/org/apache/jackrabbit/oak/plugins/document/DocumentNodeStoreBranch.java
(Revision 1825371)
+++
oak-store-document/src/main/java/org/apache/jackrabbit/oak/plugins/document/DocumentNodeStoreBranch.java
(Arbeitskopie)
@@ -520,8 +520,12 @@
} catch (ConflictException e) {
throw e.asCommitFailedException();
} catch(DocumentStoreException e) {
-throw new CommitFailedException(MERGE, 1,
-"Failed to merge changes to the underlying store",
e);
+if (e.getMessage().contains("Invalid ID")) {
+throw new CommitFailedException(OAK, 123,
+"Failed to store changes in the underlying
store: " + e.getMessage(), e);
+} else {
+throw new CommitFailedException(MERGE, 1, "Failed to
merge changes to the underlying store", e);
+}
} catch (Exception e) {
throw new CommitFailedException(OAK, 1,
"Failed to merge changes to the underlying store",
e);
{noformat}
...which causes the exception to surface immediately (see
https://issues.apache.org/jira/secure/attachment/12912117/OAK-7286.diff).
(cc [~mreutegg]])
was:
In {{DocumentNodeStoreBranch.merge()}}, any {{DocumentStoreException}} is
mapped to a {{DocumentStoreException}} to a {{CommitFailedException}} of type
"MERGE", which leads to the operation being retried, and a non-helpful
exception being generated.
The effect can be observed by enabling a test in {{ValidNamesTest}}:
{noformat}
--- oak-jcr/src/test/java/org/apache/jackrabbit/oak/jcr/ValidNamesTest.java
(Revision 1825371)
+++ oak-jcr/src/test/java/org/apache/jackrabbit/oak/jcr/ValidNamesTest.java
(Arbeitskopie)
@@ -300,7 +300,6 @@
public void testUnpairedHighSurrogateEnd() {
// see OAK-5506
org.junit.Assume.assumeFalse(super.fixture.toString().toLowerCase().contains("segment"));
-
org.junit.Assume.assumeFalse(super.fixture.toString().toLowerCase().contains("rdb"));
nameTest("foo" + SURROGATE_PAIR[0]);
}
@@ -336,6 +335,7 @@
assertEquals("paths should be equal", p.getPath(), n.getPath());
return p;
} catch (RepositoryException ex) {
+ex.printStackTrace();
fail(ex.getMessage());
return null;
}
{noformat}
The underlying issue is that {{RDBDocumentStore}} is throwing a
{{DocumentStoreException}} due to the invalid ID, and repeating the call will
not help.
We probably should have a way to dinstinguish between different types of
problems.
I hacked {{DocumentNodeStoreBranch}} like that:
{noformat}
---
oak-store-document/src/main/java/org/apache/jackrabbit/oak/plugins/document/DocumentNodeStoreBranch.java
(Revision 1825371)
+++
oak-store-document/src/main/java/org/apache/jackrabbit/oak/plugins/document/DocumentNodeStoreBranch.java
(Arbeitskopie)
@@ -520,8 +520,12 @@
} catch (ConflictException e) {
throw e.asCommitFailedException();
} catch(DocumentStoreException e) {
-throw new
[jira] [Updated] (OAK-7286) DocumentNodeStoreBranch handling of non-recoverable DocumentStoreExceptions
[
https://issues.apache.org/jira/browse/OAK-7286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Reschke updated OAK-7286:
Attachment: OAK-7286.diff
> DocumentNodeStoreBranch handling of non-recoverable DocumentStoreExceptions
> ---
>
> Key: OAK-7286
> URL: https://issues.apache.org/jira/browse/OAK-7286
> Project: Jackrabbit Oak
> Issue Type: Task
> Components: documentmk
>Reporter: Julian Reschke
>Priority: Major
> Attachments: OAK-7286.diff
>
>
> In {{DocumentNodeStoreBranch.merge()}}, any {{DocumentStoreException}} is
> mapped to a {{DocumentStoreException}} to a {{CommitFailedException}} of type
> "MERGE", which leads to the operation being retried, and a non-helpful
> exception being generated.
> The effect can be observed by enabling a test in {{ValidNamesTest}}:
> {noformat}
> --- oak-jcr/src/test/java/org/apache/jackrabbit/oak/jcr/ValidNamesTest.java
> (Revision 1825371)
> +++ oak-jcr/src/test/java/org/apache/jackrabbit/oak/jcr/ValidNamesTest.java
> (Arbeitskopie)
> @@ -300,7 +300,6 @@
> public void testUnpairedHighSurrogateEnd() {
> // see OAK-5506
>
> org.junit.Assume.assumeFalse(super.fixture.toString().toLowerCase().contains("segment"));
> -
> org.junit.Assume.assumeFalse(super.fixture.toString().toLowerCase().contains("rdb"));
> nameTest("foo" + SURROGATE_PAIR[0]);
> }
> @@ -336,6 +335,7 @@
> assertEquals("paths should be equal", p.getPath(), n.getPath());
> return p;
> } catch (RepositoryException ex) {
> +ex.printStackTrace();
> fail(ex.getMessage());
> return null;
> }
> {noformat}
> The underlying issue is that {{RDBDocumentStore}} is throwing a
> {{DocumentStoreException}} due to the invalid ID, and repeating the call will
> not help.
> We probably should have a way to dinstinguish between different types of
> problems.
> I hacked {{DocumentNodeStoreBranch}} like that:
> {noformat}
> ---
> oak-store-document/src/main/java/org/apache/jackrabbit/oak/plugins/document/DocumentNodeStoreBranch.java
> (Revision 1825371)
> +++
> oak-store-document/src/main/java/org/apache/jackrabbit/oak/plugins/document/DocumentNodeStoreBranch.java
> (Arbeitskopie)
> @@ -520,8 +520,12 @@
> } catch (ConflictException e) {
> throw e.asCommitFailedException();
> } catch(DocumentStoreException e) {
> -throw new CommitFailedException(MERGE, 1,
> -"Failed to merge changes to the underlying
> store", e);
> +if (e.getMessage().contains("Invalid ID")) {
> +throw new CommitFailedException(OAK, 123,
> +"Failed to store changes in the underlying
> store: " + e.getMessage(), e);
> +} else {
> +throw new CommitFailedException(MERGE, 1, "Failed to
> merge changes to the underlying store", e);
> +}
> } catch (Exception e) {
> throw new CommitFailedException(OAK, 1,
> "Failed to merge changes to the underlying
> store", e);
> {noformat}
> ...which causes the exception to surface immediately.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (OAK-7286) DocumentNodeStoreBranch handling of non-recoverable DocumentStoreExceptions
Julian Reschke created OAK-7286:
---
Summary: DocumentNodeStoreBranch handling of non-recoverable
DocumentStoreExceptions
Key: OAK-7286
URL: https://issues.apache.org/jira/browse/OAK-7286
Project: Jackrabbit Oak
Issue Type: Task
Components: documentmk
Reporter: Julian Reschke
In {{DocumentNodeStoreBranch.merge()}}, any {{DocumentStoreException}} is
mapped to a {{DocumentStoreException}} to a {{CommitFailedException}} of type
"MERGE", which leads to the operation being retried, and a non-helpful
exception being generated.
The effect can be observed by enabling a test in {{ValidNamesTest}}:
{noformat}
--- oak-jcr/src/test/java/org/apache/jackrabbit/oak/jcr/ValidNamesTest.java
(Revision 1825371)
+++ oak-jcr/src/test/java/org/apache/jackrabbit/oak/jcr/ValidNamesTest.java
(Arbeitskopie)
@@ -300,7 +300,6 @@
public void testUnpairedHighSurrogateEnd() {
// see OAK-5506
org.junit.Assume.assumeFalse(super.fixture.toString().toLowerCase().contains("segment"));
-
org.junit.Assume.assumeFalse(super.fixture.toString().toLowerCase().contains("rdb"));
nameTest("foo" + SURROGATE_PAIR[0]);
}
@@ -336,6 +335,7 @@
assertEquals("paths should be equal", p.getPath(), n.getPath());
return p;
} catch (RepositoryException ex) {
+ex.printStackTrace();
fail(ex.getMessage());
return null;
}
{noformat}
The underlying issue is that {{RDBDocumentStore}} is throwing a
{{DocumentStoreException}} due to the invalid ID, and repeating the call will
not help.
We probably should have a way to dinstinguish between different types of
problems.
I hacked {{DocumentNodeStoreBranch}} like that:
{noformat}
---
oak-store-document/src/main/java/org/apache/jackrabbit/oak/plugins/document/DocumentNodeStoreBranch.java
(Revision 1825371)
+++
oak-store-document/src/main/java/org/apache/jackrabbit/oak/plugins/document/DocumentNodeStoreBranch.java
(Arbeitskopie)
@@ -520,8 +520,12 @@
} catch (ConflictException e) {
throw e.asCommitFailedException();
} catch(DocumentStoreException e) {
-throw new CommitFailedException(MERGE, 1,
-"Failed to merge changes to the underlying store",
e);
+if (e.getMessage().contains("Invalid ID")) {
+throw new CommitFailedException(OAK, 123,
+"Failed to store changes in the underlying
store: " + e.getMessage(), e);
+} else {
+throw new CommitFailedException(MERGE, 1, "Failed to
merge changes to the underlying store", e);
+}
} catch (Exception e) {
throw new CommitFailedException(OAK, 1,
"Failed to merge changes to the underlying store",
e);
{noformat}
...which causes the exception to surface immediately.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-7276) Build Jackrabbit Oak #1256 failed
[ https://issues.apache.org/jira/browse/OAK-7276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16377159#comment-16377159 ] Hudson commented on OAK-7276: - Previously failing build now is OK. Passed run: [Jackrabbit Oak #1262|https://builds.apache.org/job/Jackrabbit%20Oak/1262/] [console log|https://builds.apache.org/job/Jackrabbit%20Oak/1262/console] > Build Jackrabbit Oak #1256 failed > - > > Key: OAK-7276 > URL: https://issues.apache.org/jira/browse/OAK-7276 > Project: Jackrabbit Oak > Issue Type: Bug > Components: continuous integration >Reporter: Hudson >Priority: Major > > No description is provided > The build Jackrabbit Oak #1256 has failed. > First failed run: [Jackrabbit Oak > #1256|https://builds.apache.org/job/Jackrabbit%20Oak/1256/] [console > log|https://builds.apache.org/job/Jackrabbit%20Oak/1256/console] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (OAK-6770) Convert oak-segment-tar to OSGi R6 annotations
[ https://issues.apache.org/jira/browse/OAK-6770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francesco Mari reassigned OAK-6770: --- Assignee: Francesco Mari > Convert oak-segment-tar to OSGi R6 annotations > -- > > Key: OAK-6770 > URL: https://issues.apache.org/jira/browse/OAK-6770 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: segment-tar >Reporter: Robert Munteanu >Assignee: Francesco Mari >Priority: Minor > Labels: osgi > Fix For: 1.9.0, 1.10 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (OAK-7058) oak-run compact reports success even when it was cancelled
[
https://issues.apache.org/jira/browse/OAK-7058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig reassigned OAK-7058:
--
Assignee: Michael Dürig
> oak-run compact reports success even when it was cancelled
> --
>
> Key: OAK-7058
> URL: https://issues.apache.org/jira/browse/OAK-7058
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: run, segment-tar
>Reporter: Michael Dürig
>Assignee: Michael Dürig
>Priority: Major
> Labels: production, tooling
> Fix For: 1.9.0, 1.10
>
>
> When {{oak-run compact}} gets cancelled because running out of disk space it
> will send a corresponding warning to the logs and bail out. However on the
> console it will still report success.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OAK-6770) Convert oak-segment-tar to OSGi R6 annotations
[ https://issues.apache.org/jira/browse/OAK-6770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Dürig updated OAK-6770: --- Fix Version/s: 1.9.0 > Convert oak-segment-tar to OSGi R6 annotations > -- > > Key: OAK-6770 > URL: https://issues.apache.org/jira/browse/OAK-6770 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: segment-tar >Reporter: Robert Munteanu >Priority: Minor > Labels: osgi > Fix For: 1.9.0, 1.10 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (OAK-7188) guava: ListenableFuture.transform() changes to transformAsync in version 20
[ https://issues.apache.org/jira/browse/OAK-7188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Dürig reassigned OAK-7188: -- Assignee: Michael Dürig > guava: ListenableFuture.transform() changes to transformAsync in version 20 > --- > > Key: OAK-7188 > URL: https://issues.apache.org/jira/browse/OAK-7188 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: segment-tar >Reporter: Julian Reschke >Assignee: Michael Dürig >Priority: Major > Labels: technical_debt > Fix For: 1.9.0, 1.10 > > Attachments: OAK-7188.diff, OAK-7188.diff > > > See > https://google.github.io/guava/releases/19.0/api/docs/com/google/common/util/concurrent/Futures.html#transform(com.google.common.util.concurrent.ListenableFuture,%20com.google.common.util.concurrent.AsyncFunction) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OAK-7058) oak-run compact reports success even when it was cancelled
[
https://issues.apache.org/jira/browse/OAK-7058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig updated OAK-7058:
---
Fix Version/s: 1.9.0
> oak-run compact reports success even when it was cancelled
> --
>
> Key: OAK-7058
> URL: https://issues.apache.org/jira/browse/OAK-7058
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: run, segment-tar
>Reporter: Michael Dürig
>Priority: Major
> Labels: production, tooling
> Fix For: 1.9.0, 1.10
>
>
> When {{oak-run compact}} gets cancelled because running out of disk space it
> will send a corresponding warning to the logs and bail out. However on the
> console it will still report success.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OAK-7188) guava: ListenableFuture.transform() changes to transformAsync in version 20
[ https://issues.apache.org/jira/browse/OAK-7188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Dürig updated OAK-7188: --- Fix Version/s: 1.9.0 > guava: ListenableFuture.transform() changes to transformAsync in version 20 > --- > > Key: OAK-7188 > URL: https://issues.apache.org/jira/browse/OAK-7188 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: segment-tar >Reporter: Julian Reschke >Assignee: Michael Dürig >Priority: Major > Labels: technical_debt > Fix For: 1.9.0, 1.10 > > Attachments: OAK-7188.diff, OAK-7188.diff > > > See > https://google.github.io/guava/releases/19.0/api/docs/com/google/common/util/concurrent/Futures.html#transform(com.google.common.util.concurrent.ListenableFuture,%20com.google.common.util.concurrent.AsyncFunction) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OAK-7057) Segment.toString: Record table should include an index into the hexdump
[
https://issues.apache.org/jira/browse/OAK-7057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig updated OAK-7057:
---
Fix Version/s: 1.9.0
> Segment.toString: Record table should include an index into the hexdump
> ---
>
> Key: OAK-7057
> URL: https://issues.apache.org/jira/browse/OAK-7057
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Michael Dürig
>Priority: Minor
> Labels: tooling
> Fix For: 1.9.0, 1.10
>
>
> Currently the Segment dump created in {{Segment.toString}} includes a list of
> records with their offsets. However these offsets do no match the ones in the
> subsequent raw byte dump of the segment. We should add a raw offsets to the
> list of records so finding the actual data that belongs to a record doesn't
> involve manually fiddling with logical / physical offset translation.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (OAK-6707) TarWriter.close() must not throw an exception on subsequent invocations
[
https://issues.apache.org/jira/browse/OAK-6707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig reassigned OAK-6707:
--
Assignee: Michael Dürig
> TarWriter.close() must not throw an exception on subsequent invocations
> ---
>
> Key: OAK-6707
> URL: https://issues.apache.org/jira/browse/OAK-6707
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Michael Dürig
>Priority: Minor
> Labels: technical_debt
> Fix For: 1.9.0, 1.10
>
>
> Invoking TarWriter.close() on an already closed writer throws an {{ISE}}.
> According to the general contract this is not allowed:
> {code}
> * Closes this stream and releases any system resources associated
> * with it. If the stream is already closed then invoking this
> * method has no effect.
> {code}
> We should adjust the behvaviour of that method accordingly.
> Failing to comply with that general contract causes {{TarWriter}} instances
> to fail in try-resource statements when multiple wrapped streams are involved.
> Consider
> {code}
> try (
> StringWriter string = new StringWriter();
> PrintWriter writer = new PrintWriter(string);
> WriterOutputStream out = new WriterOutputStream(writer, Charsets.UTF_8))
> {
> dumpHeader(out);
> writer.println("");
> dumpHex(out);
> writer.println("");
> return string.toString();
> }
> {code}
> This code would cause exceptions to be thrown if e.g. the
> {{PrintWriter.close}} method would not be idempotent.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (OAK-7057) Segment.toString: Record table should include an index into the hexdump
[
https://issues.apache.org/jira/browse/OAK-7057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig reassigned OAK-7057:
--
Assignee: Michael Dürig
> Segment.toString: Record table should include an index into the hexdump
> ---
>
> Key: OAK-7057
> URL: https://issues.apache.org/jira/browse/OAK-7057
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Michael Dürig
>Priority: Minor
> Labels: tooling
> Fix For: 1.9.0, 1.10
>
>
> Currently the Segment dump created in {{Segment.toString}} includes a list of
> records with their offsets. However these offsets do no match the ones in the
> subsequent raw byte dump of the segment. We should add a raw offsets to the
> list of records so finding the actual data that belongs to a record doesn't
> involve manually fiddling with logical / physical offset translation.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OAK-6707) TarWriter.close() must not throw an exception on subsequent invocations
[
https://issues.apache.org/jira/browse/OAK-6707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig updated OAK-6707:
---
Fix Version/s: 1.9.0
> TarWriter.close() must not throw an exception on subsequent invocations
> ---
>
> Key: OAK-6707
> URL: https://issues.apache.org/jira/browse/OAK-6707
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar
>Reporter: Michael Dürig
>Priority: Minor
> Labels: technical_debt
> Fix For: 1.9.0, 1.10
>
>
> Invoking TarWriter.close() on an already closed writer throws an {{ISE}}.
> According to the general contract this is not allowed:
> {code}
> * Closes this stream and releases any system resources associated
> * with it. If the stream is already closed then invoking this
> * method has no effect.
> {code}
> We should adjust the behvaviour of that method accordingly.
> Failing to comply with that general contract causes {{TarWriter}} instances
> to fail in try-resource statements when multiple wrapped streams are involved.
> Consider
> {code}
> try (
> StringWriter string = new StringWriter();
> PrintWriter writer = new PrintWriter(string);
> WriterOutputStream out = new WriterOutputStream(writer, Charsets.UTF_8))
> {
> dumpHeader(out);
> writer.println("");
> dumpHex(out);
> writer.println("");
> return string.toString();
> }
> {code}
> This code would cause exceptions to be thrown if e.g. the
> {{PrintWriter.close}} method would not be idempotent.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-7285) Reindexing using --doc-traversal-mode can OOM while aggregation in some cases
[
https://issues.apache.org/jira/browse/OAK-7285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16377067#comment-16377067
]
Vikas Saurabh commented on OAK-7285:
Following test in {{FaltFileStoreIteratorTest}} shows the behavior (not
committing ignored test, as we'd probably need to restructure
{{FlatFileStoreIteartor}} ctor to take in preferred list instead of just size:
{noformat}
@Test
public void getChildNodeForNonPreferred() {
// have more than 1 preferred names
Set preferred = ImmutableSet.of("j:c", "md");
CountingIterable citr = createList(preferred,
asList("/a", "/a/b", "/a/c"));
FlatFileStoreIterator fitr = new FlatFileStoreIterator(citr.iterator(),
preferred.size());
NodeStateEntry a = fitr.next();
assertEquals("/a", a.getPath());
NodeState aNS = a.getNodeState();
aNS.getChildNode("j:c");
// Don't read whole tree to conclude that "j:c" doesn't exist (reading
/a/b should imply that it doesn't exist)
assertEquals(1, fitr.getBufferSize());
}
{noformat}
> Reindexing using --doc-traversal-mode can OOM while aggregation in some cases
> -
>
> Key: OAK-7285
> URL: https://issues.apache.org/jira/browse/OAK-7285
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: lucene, mongomk
>Affects Versions: 1.8.0
>Reporter: Vikas Saurabh
>Assignee: Vikas Saurabh
>Priority: Major
> Labels: candidate_oak_1_8
> Fix For: 1.10
>
>
> {{--doc-traversal-mode}} works on the notion of {{preferred}} children which
> is computed using path fragments that form aggregate rules.
> The idea is reading through aggregated paths should avoid keeping non useful
> nodes (for path being currently indexed) in memory.
> But, currently, in case, say when there multiple preferred children -
> {{jcr:content}}, {{metadata}}, then an index defn indexing parent of a very
> deep tree root would try to read in the whole tree before concluding that it
> doesn't have preferred children
> e.g. with preferred list - {{j:c}} and {{md} and index looking for {{j:c}}
> indexing following structure
> {noformat}
> + /path/being/indexed
>+ very
> + very
> + very
>+ deep
>+ tree
> + /some-sibling
> {noformat}
> Currently, while looking for {{j:c}}, the code concludes that it doesn't
> exist only after reaching {{/some-sibling}} (or if number of children read of
> {{/path/being/indexed}} is >= num_preferred_children).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (OAK-7285) Reindexing using --doc-traversal-mode can OOM while aggregation in some cases
Vikas Saurabh created OAK-7285:
--
Summary: Reindexing using --doc-traversal-mode can OOM while
aggregation in some cases
Key: OAK-7285
URL: https://issues.apache.org/jira/browse/OAK-7285
Project: Jackrabbit Oak
Issue Type: Bug
Components: lucene, mongomk
Affects Versions: 1.8.0
Reporter: Vikas Saurabh
Assignee: Vikas Saurabh
Fix For: 1.10
{{--doc-traversal-mode}} works on the notion of {{preferred}} children which is
computed using path fragments that form aggregate rules.
The idea is reading through aggregated paths should avoid keeping non useful
nodes (for path being currently indexed) in memory.
But, currently, in case, say when there multiple preferred children -
{{jcr:content}}, {{metadata}}, then an index defn indexing parent of a very
deep tree root would try to read in the whole tree before concluding that it
doesn't have preferred children
e.g. with preferred list - {{j:c}} and {{md} and index looking for {{j:c}}
indexing following structure
{noformat}
+ /path/being/indexed
+ very
+ very
+ very
+ deep
+ tree
+ /some-sibling
{noformat}
Currently, while looking for {{j:c}}, the code concludes that it doesn't exist
only after reaching {{/some-sibling}} (or if number of children read of
{{/path/being/indexed}} is >= num_preferred_children).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OAK-6312) Unify NodeStore/DataStore configurations
[ https://issues.apache.org/jira/browse/OAK-6312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Dürig updated OAK-6312: --- Fix Version/s: (was: 1.10) > Unify NodeStore/DataStore configurations > > > Key: OAK-6312 > URL: https://issues.apache.org/jira/browse/OAK-6312 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: blob, blob-plugins, composite, documentmk, rdbmk, > segment-tar >Reporter: Arek Kita >Priority: Major > Labels: configuration, production > > I've noticed recently that with many different NodeStore > implementation (Segment, Document, Composite) but also DataStore > implementation (File, S3, Azure) and some composite ones like > (Hierarchical, Federated) it > becomes more and more difficult to set up everything correctly and be > able to know the current persistence state of repository (especially > with pretty aged repos). The factory code/required options are more complex > not only from user perspective but also from maintenance point. > We should have the same means of *describing* layouts of Oak repository no > matter if it is simple or more layered/composite instance. > Some work has already been done in scope of OAK-6210 so I guess we have good > foundations to continue working in that direction. > /cc [~mattvryan], [~chetanm] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OAK-7188) guava: ListenableFuture.transform() changes to transformAsync in version 20
[ https://issues.apache.org/jira/browse/OAK-7188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Dürig updated OAK-7188: --- Fix Version/s: 1.10 > guava: ListenableFuture.transform() changes to transformAsync in version 20 > --- > > Key: OAK-7188 > URL: https://issues.apache.org/jira/browse/OAK-7188 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: segment-tar >Reporter: Julian Reschke >Priority: Major > Labels: technical_debt > Fix For: 1.10 > > Attachments: OAK-7188.diff, OAK-7188.diff > > > See > https://google.github.io/guava/releases/19.0/api/docs/com/google/common/util/concurrent/Futures.html#transform(com.google.common.util.concurrent.ListenableFuture,%20com.google.common.util.concurrent.AsyncFunction) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OAK-6737) Standby server should send timely responses to all client requests
[
https://issues.apache.org/jira/browse/OAK-6737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig updated OAK-6737:
---
Fix Version/s: (was: 1.10)
(was: 1.9.0)
> Standby server should send timely responses to all client requests
> --
>
> Key: OAK-6737
> URL: https://issues.apache.org/jira/browse/OAK-6737
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: segment-tar, tarmk-standby
>Reporter: Andrei Dulceanu
>Assignee: Andrei Dulceanu
>Priority: Minor
> Labels: candidate_oak_1_8, cold-standby
>
> Currently all the {{GetXXXRequestHandler}} (where XXX stands for Blob, Head,
> References and Segment), on the server discard client requests which cannot
> be satisfied (i.e. the requested object does not exist (yet) on the server).
> A more transparent approach would be to timely respond to all client
> requests, clearly stating that the object was not found. This would improve a
> lot debugging for example, because all requests and their responses could be
> easily followed from the client log, without needing to know what actually
> happened on the server.
> Below, a possible implementation for {{GetHeadRequestHandler}}, suggested by
> [~frm] in a comment on OAK-6678:
> {noformat}
> String id = reader.readHeadRecordId();
> if (id == null) {
> ctx.writeAndFlush(new NotFoundGetHeadResponse(msg.getClientId(), id));
> return;
> }
> ctx.writeAndFlush(new GetHeadResponse(msg.getClientId(), id));
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OAK-6674) Create a more complex IT for cold standby
[
https://issues.apache.org/jira/browse/OAK-6674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig updated OAK-6674:
---
Fix Version/s: (was: 1.10)
(was: 1.9.0)
> Create a more complex IT for cold standby
> -
>
> Key: OAK-6674
> URL: https://issues.apache.org/jira/browse/OAK-6674
> Project: Jackrabbit Oak
> Issue Type: Task
> Components: segment-tar, tarmk-standby
>Reporter: Andrei Dulceanu
>Assignee: Andrei Dulceanu
>Priority: Major
> Labels: cold-standby, test
>
> At the moment all integration tests for cold standby are using the same
> scenario in their tests: some content is created on the server (including
> binaries), a standby sync cycle is started and then the content is checked on
> the client. The only twist here is using/not using a data store for storing
> binaries.
> Although good, this model could be extended to cover many more cases. For
> example, {{StandbyDiff}} covers the following 6 cases node/property
> added/changed/deleted. From these, with the scenario described, the removal
> part is never tested (and the change part is covered in only one test).
> It would be nice to have an IT which would add content on the server, do a
> sync, remove some of the content, do a sync and then call OnRC. This way all
> cases will be covered, including if cleanup works as expected on the client.
> /cc [~frm]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (OAK-5884) Evaluate utility of RepositoryGrowthTest benchmark
[
https://issues.apache.org/jira/browse/OAK-5884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig resolved OAK-5884.
Resolution: Later
Fix Version/s: (was: 1.10)
(was: 1.9.0)
> Evaluate utility of RepositoryGrowthTest benchmark
> --
>
> Key: OAK-5884
> URL: https://issues.apache.org/jira/browse/OAK-5884
> Project: Jackrabbit Oak
> Issue Type: Task
> Components: run, segment-tar
>Reporter: Andrei Dulceanu
>Assignee: Andrei Dulceanu
>Priority: Minor
> Labels: benchmark
>
> {{RepositoryGrowthTest}} is a benchmark which makes use of the deprecated
> {{SegmentFixture}}. Since OAK-5834 removes the old {{oak-segment}} module and
> the code associated with it, {{RepositoryGrowthTest}} was also removed. If
> there's value in it, we can adapt it to work with the new
> {{SegmentTarFixture}}.
> /cc [~chetanm]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-7284) Reindexing using --doc-traversal-mode can hit ConcurrentModificationException during aggregation
[ https://issues.apache.org/jira/browse/OAK-7284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16377037#comment-16377037 ] Vikas Saurabh commented on OAK-7284: Added ignored test at [r1825381|https://svn.apache.org/r1825381]. > Reindexing using --doc-traversal-mode can hit ConcurrentModificationException > during aggregation > > > Key: OAK-7284 > URL: https://issues.apache.org/jira/browse/OAK-7284 > Project: Jackrabbit Oak > Issue Type: Bug > Components: lucene, mongomk >Affects Versions: 1.8.0 >Reporter: Vikas Saurabh >Assignee: Vikas Saurabh >Priority: Major > Labels: candidate_oak_1_8 > Fix For: 1.10 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (OAK-2113) TarMK cold standby: ensure client integrity
[ https://issues.apache.org/jira/browse/OAK-2113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Dürig resolved OAK-2113. Resolution: Later > TarMK cold standby: ensure client integrity > > > Key: OAK-2113 > URL: https://issues.apache.org/jira/browse/OAK-2113 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: segment-tar >Reporter: Manfred Baedke >Priority: Minor > Labels: resilience > > TarMK cold standby needs measures to ensure the integrity of each segment on > the slave and that all segments that are reachable on the master from a given > segment are available on the slave. > To ensure the integrity of a given segment on the slave, we can just use the > checksum stored in the segment, so there is no need to change the > communication protocol for this. > To ensure that all segments that are reachable from a given segment are the > same on the master and on the slave, we need a new request to calculate a > suitable checksum on the master and send it back to the slave. > If missing or broken segments are detected, the slave will pull them from the > master again. > Both measures combined should be scheduled to run regularly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (OAK-7284) Reindexing using --doc-traversal-mode can hit ConcurrentModificationException during aggregation
Vikas Saurabh created OAK-7284: -- Summary: Reindexing using --doc-traversal-mode can hit ConcurrentModificationException during aggregation Key: OAK-7284 URL: https://issues.apache.org/jira/browse/OAK-7284 Project: Jackrabbit Oak Issue Type: Bug Components: lucene, mongomk Affects Versions: 1.8.0 Reporter: Vikas Saurabh Assignee: Vikas Saurabh Fix For: 1.10 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (OAK-6922) Azure support for the segment-tar
[
https://issues.apache.org/jira/browse/OAK-6922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig reassigned OAK-6922:
--
Assignee: Tomek Rękawek
> Azure support for the segment-tar
> -
>
> Key: OAK-6922
> URL: https://issues.apache.org/jira/browse/OAK-6922
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: segment-tar
>Reporter: Tomek Rękawek
>Assignee: Tomek Rękawek
>Priority: Major
> Fix For: 1.9.0, 1.10
>
> Attachments: OAK-6922.patch
>
>
> An Azure Blob Storage implementation of the segment storage, based on the
> OAK-6921 work.
> h3. Segment files layout
> Thew new implementation doesn't use tar files. They are replaced with
> directories, storing segments, named after their UUIDs. This approach has
> following advantages:
> * no need to call seek(), which may be expensive on a remote file system.
> Rather than that we can read the whole file (=segment) at once.
> * it's possible to send multiple segments at once, asynchronously, which
> reduces the performance overhead (see below).
> The file structure is as follows:
> {noformat}
> [~]$ az storage blob list -c oak --output table
> Name Blob Type
> Blob TierLengthContent Type Last Modified
> ---
> --- -
> oak/data0a.tar/.ca1326d1-edf4-4d53-aef0-0f14a6d05b63 BlockBlob
> 192 application/octet-stream 2018-01-31T10:59:14+00:00
> oak/data0a.tar/0001.c6e03426-db9d-4315-a20a-12559e6aee54 BlockBlob
> 262144application/octet-stream 2018-01-31T10:59:14+00:00
> oak/data0a.tar/0002.b3784e27-6d16-4f80-afc1-6f3703f6bdb9 BlockBlob
> 262144application/octet-stream 2018-01-31T10:59:14+00:00
> oak/data0a.tar/0003.5d2f9588-0c92-4547-abf7-0263ee7c37bb BlockBlob
> 259216application/octet-stream 2018-01-31T10:59:14+00:00
> ...
> oak/data0a.tar/006e.7b8cf63d-849a-4120-aa7c-47c3dde25e48 BlockBlob
> 4368 application/octet-stream 2018-01-31T12:01:09+00:00
> oak/data0a.tar/006f.93799ae9-288e-4b32-afc2-bbc676fad7e5 BlockBlob
> 3792 application/octet-stream 2018-01-31T12:01:14+00:00
> oak/data0a.tar/0070.8b2d5ff2-6a74-4ac3-a3cc-cc439367c2aa BlockBlob
> 3680 application/octet-stream 2018-01-31T12:01:14+00:00
> oak/data0a.tar/0071.2a1c49f0-ce33-4777-a042-8aa8a704d202 BlockBlob
> 7760 application/octet-stream 2018-01-31T12:10:54+00:00
> oak/journal.log.001 AppendBlob
> 1010 application/octet-stream 2018-01-31T12:10:54+00:00
> oak/manifest BlockBlob
> 46application/octet-stream 2018-01-31T10:59:14+00:00
> oak/repo.lock BlockBlob
> application/octet-stream 2018-01-31T10:59:14+00:00
> {noformat}
> For the segment files, each name is prefixed with the index number. This
> allows to maintain an order, as in the tar archive. This order is normally
> stored in the index files as well, but if it's missing, the recovery process
> needs it.
> Each file contains the raw segment data, with no padding/headers. Apart from
> the segment files, there are 3 special files: binary references (.brf),
> segment graph (.gph) and segment index (.idx).
> h3. Asynchronous writes
> Normally, all the TarWriter writes are synchronous, appending the segments to
> the tar file. In case of Azure Blob Storage each write involves a network
> latency. That's why the SegmentWriteQueue was introduced. The segments are
> added to the blocking dequeue, which is served by a number of the consumer
> threads, writing the segments to the cloud. There's also a map UUID->Segment,
> which allows to return the segments in case they are requested by the
> readSegment() method before they are actually persisted. Segments are removed
> from the map only after a successful write operation.
> The flush() method blocks accepting the new segments and returns after all
> waiting segments are written. The close() method waits until the current
> operations are finished and stops all threads.
> The asynchronous mode can be disabled by setting the number of threads to 0.
> h5. Queue recovery mode
> If the Azure Blob Storage write() operation fails, the segment will be
> re-added and the queue is switched to an "recovery mode". In this mode, all
> the threads are suspended and new segments are not accepted (active waiting).
[jira] [Resolved] (OAK-3893) SegmentWriter records cache could use thinner keys
[
https://issues.apache.org/jira/browse/OAK-3893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig resolved OAK-3893.
Resolution: Later
> SegmentWriter records cache could use thinner keys
> --
>
> Key: OAK-3893
> URL: https://issues.apache.org/jira/browse/OAK-3893
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: segment-tar
>Reporter: Alex Deparvu
>Assignee: Alex Deparvu
>Priority: Minor
> Labels: performance
> Attachments: OAK-3893.patch
>
>
> The SegmentWriter keeps a records deduplication cache ('records' map) that
> maintains 2 types of mappings:
> * template -> recordid
> * strings -> recordid
> For the first one (template-> recordid) we can come up with a thinner
> representation of a template (a hash function that is fast and not very
> collision prone) so we don't have to keep a reference to each template object.
> Same applies for second one, similar to what is happening in the StringsCache
> now, we could keep the string value up to a certain size and beyond that,
> hash it and use that for the deduplication map.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (OAK-5786) Memory based FileStore bootstrap
[ https://issues.apache.org/jira/browse/OAK-5786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Dürig resolved OAK-5786. Resolution: Later > Memory based FileStore bootstrap > > > Key: OAK-5786 > URL: https://issues.apache.org/jira/browse/OAK-5786 > Project: Jackrabbit Oak > Issue Type: New Feature > Components: segment-tar >Reporter: Alex Deparvu >Assignee: Alex Deparvu >Priority: Major > > A memory-based FileStore boot would reduce IO to zero (or as close as > possible) and provide more insight into the code bottlenecks (locks, > concurrency control, etc). > A very basic idea (with minimal changes to current code) is to boot a > FileStore that would > * disable mmaping > * have an unbounded segment cache (and/or other caches), with no size checks > * eagerly load all application state in memory on startup > * optionally flush everything to disk on shutdown only so the state is > persistent -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (OAK-5635) Revisit FileStoreStats mbean stats format
[ https://issues.apache.org/jira/browse/OAK-5635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Dürig resolved OAK-5635. Resolution: Later Fix Version/s: (was: 1.10) > Revisit FileStoreStats mbean stats format > - > > Key: OAK-5635 > URL: https://issues.apache.org/jira/browse/OAK-5635 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: segment-tar >Reporter: Alex Deparvu >Assignee: Michael Dürig >Priority: Major > Labels: monitoring > > This is a bigger refactoring item to revisit the format of the exposed data, > moving towards having it in a more machine consumable friendly format. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7272) improve BackgroundLeaseUpdate warning messages
[
https://issues.apache.org/jira/browse/OAK-7272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16376910#comment-16376910
]
Stefan Egli commented on OAK-7272:
--
[~reschke], +1 to these changes
> improve BackgroundLeaseUpdate warning messages
> --
>
> Key: OAK-7272
> URL: https://issues.apache.org/jira/browse/OAK-7272
> Project: Jackrabbit Oak
> Issue Type: Task
> Components: documentmk
>Reporter: Julian Reschke
>Assignee: Julian Reschke
>Priority: Minor
> Labels: candidate_oak_1_8
> Fix For: 1.9.0, 1.10
>
> Attachments: OAK-7272.diff, OAK-7272.diff
>
>
> Example for current logging:
> {noformat}
> *WARN* [DocumentNodeStore lease update thread (1)]
> org.apache.jackrabbit.oak.plugins.document.DocumentNodeStore
> BackgroundLeaseUpdate.execute: time since last renewClusterIdLease() call
> longer than expected: 5338ms
> {noformat}
> Source:
> {noformat}
> @Override
> protected void execute(@Nonnull DocumentNodeStore nodeStore) {
> // OAK-4859 : keep track of invocation time of renewClusterIdLease
> // and warn if time since last call is longer than 5sec
> final long now = System.currentTimeMillis();
> if (lastRenewClusterIdLeaseCall <= 0) {
> lastRenewClusterIdLeaseCall = now;
> } else {
> final long diff = now - lastRenewClusterIdLeaseCall;
> if (diff > 5000) {
> LOG.warn("BackgroundLeaseUpdate.execute: time since last
> renewClusterIdLease() call longer than expected: {}ms", diff);
> }
> lastRenewClusterIdLeaseCall = now;
> }
> // first renew the clusterId lease
> nodeStore.renewClusterIdLease();
> }
> {noformat}
> Observations:
> - the warning message doesn't actually say what the expected delay is
> - we only log when it's exceeded by factor 5
> - the threshold is hardwired; it should be computed based on the actual
> config (I think)
> Also:
> - we don't measure the time of the actual update operation, so we don't know
> whether it's a thread scheduling problem or a persistence problem (again, I
> think)
> [~egli], [~mreutegg] - feedback appreciated.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-7276) Build Jackrabbit Oak #1256 failed
[ https://issues.apache.org/jira/browse/OAK-7276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16376871#comment-16376871 ] Hudson commented on OAK-7276: - Previously failing build now is OK. Passed run: [Jackrabbit Oak #1261|https://builds.apache.org/job/Jackrabbit%20Oak/1261/] [console log|https://builds.apache.org/job/Jackrabbit%20Oak/1261/console] > Build Jackrabbit Oak #1256 failed > - > > Key: OAK-7276 > URL: https://issues.apache.org/jira/browse/OAK-7276 > Project: Jackrabbit Oak > Issue Type: Bug > Components: continuous integration >Reporter: Hudson >Priority: Major > > No description is provided > The build Jackrabbit Oak #1256 has failed. > First failed run: [Jackrabbit Oak > #1256|https://builds.apache.org/job/Jackrabbit%20Oak/1256/] [console > log|https://builds.apache.org/job/Jackrabbit%20Oak/1256/console] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (OAK-6397) Move record implementations to their own package
[
https://issues.apache.org/jira/browse/OAK-6397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francesco Mari resolved OAK-6397.
-
Resolution: Later
> Move record implementations to their own package
>
>
> Key: OAK-6397
> URL: https://issues.apache.org/jira/browse/OAK-6397
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: segment-tar
>Reporter: Francesco Mari
>Assignee: Francesco Mari
>Priority: Major
> Labels: technical_debt
>
> Given the work done for OAK-6378, it is now possible to move the record
> implementation to their own package. These implementations can be implemented
> on top of the {{SegmentReader}}, {{SegmentWriter}} and {{BlobStore}}
> interfaces, and detached from other implementation classes from the
> {{o.a.j.o.segment}} package. I have already started working in [this
> branch|https://github.com/francescomari/jackrabbit-oak/tree/record-package]
> in GitHub.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (OAK-4582) Split Segment in a read-only and a read-write implementations
[
https://issues.apache.org/jira/browse/OAK-4582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francesco Mari resolved OAK-4582.
-
Resolution: Later
> Split Segment in a read-only and a read-write implementations
> -
>
> Key: OAK-4582
> URL: https://issues.apache.org/jira/browse/OAK-4582
> Project: Jackrabbit Oak
> Issue Type: Task
> Components: segment-tar
>Reporter: Francesco Mari
>Assignee: Francesco Mari
>Priority: Major
> Labels: technical_debt
> Attachments: benchmark-01.png, benchmark-01.txt
>
>
> {{Segment}} is central to the working of the Segment Store, but it currently
> serves two purposes:
> # It is a temporary storage location for the currently written segment,
> waiting to be full and flushed to disk.
> # It is a way to parse serialzed segments read from disk.
> To distinguish these two use cases, I suggest to promote {{Segment}} to the
> status of interface, and to create two different implementations for a
> read-only and a read-write segments.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (OAK-4649) Move index files outside of the TAR files
[ https://issues.apache.org/jira/browse/OAK-4649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francesco Mari resolved OAK-4649. - Resolution: Later > Move index files outside of the TAR files > - > > Key: OAK-4649 > URL: https://issues.apache.org/jira/browse/OAK-4649 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: segment-tar >Reporter: Francesco Mari >Assignee: Francesco Mari >Priority: Major > Labels: technical_debt > > TAR files currently embed three indexes: an index of the segments contained > in the TAR files, a graph index and an index of external binary references. > Index files are checked for consistency purposes at the startup of the > system. Normally, if an index file is corrupted it is recreated. Since the > index file is contained inside the TAR file, recreating them implies > rewriting the whole TAR file and appending the new index. > This process creates unnecessary backups, since the biggest part of the TAR > file is effectively immutable. Moreover, because index files are stored in > the TAR files, we can't treat TAR files as true read-only files. There is > always the possibility that they have to be opened again in write mode for > the recovery of the index file. > I propose to move those index files outside of the TAR files. TAR files will > end up being truly read-only files containing immutable data, and index files > will be granted their own physical files on the file system. Being index > files derived data, they now can be recreated at will without impacting the > read-only part of the segment store. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (OAK-5437) Add a persistence-dependent namespace when running CLI commands
[
https://issues.apache.org/jira/browse/OAK-5437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francesco Mari resolved OAK-5437.
-
Resolution: Later
> Add a persistence-dependent namespace when running CLI commands
> ---
>
> Key: OAK-5437
> URL: https://issues.apache.org/jira/browse/OAK-5437
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: run
>Reporter: Francesco Mari
>Assignee: Francesco Mari
>Priority: Major
> Labels: tooling
>
> Commands in oak-run currently live in a flat namespace. If a command is
> specific to only one implementation, it will leave along other
> implementation-specific commands without any means of distinguishing what
> belongs where.
> I would like to add a layer of indirection to the oak-run command line
> interface, so to parse commands in the following fashion:
> {noformat}
> oak-run segment debug /path/to/folder
> oak-run mongo debug mongodb://host:12345
> oak-run rdb debug jdbc:oracle:oci8:scott/tiger@myhost
> {noformat}
> In this scenario, oak-run would become a simple entry point that would
> delegate to implementation-specific command line utilities based on the first
> argument. In the previous example, {{segment}}, {{mongo}} and {{rdb}} would
> delegate to three different implementation specific CLI utilities. Each of
> these CLI utilities will understand the {{debug}} command and will collect
> command-line parameters as it sees fit.
> If the code for a command is so generic that can be reused from different
> commands, it can be parameterised and reused from different
> implementation-specific commands.
> The benefit of this approach is that we can start moving commands closer to
> the implementations. This approach would benefit oak-run as well, which is
> overloaded with many commands from many different implementations.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (OAK-6405) Cleanup the o.a.j.o.segment.file.tar package
[
https://issues.apache.org/jira/browse/OAK-6405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francesco Mari resolved OAK-6405.
-
Resolution: Later
> Cleanup the o.a.j.o.segment.file.tar package
>
>
> Key: OAK-6405
> URL: https://issues.apache.org/jira/browse/OAK-6405
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: segment-tar
>Reporter: Francesco Mari
>Assignee: Francesco Mari
>Priority: Major
> Labels: technical_debt
>
> This issue tracks the cleanup and rearrangement of the internals of the
> {{o.a.j.o.segment.file.tar}} package.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (OAK-4558) SegmentNodeState.fastEquals() can trigger two I/O operations
[
https://issues.apache.org/jira/browse/OAK-4558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francesco Mari resolved OAK-4558.
-
Resolution: Later
> SegmentNodeState.fastEquals() can trigger two I/O operations
>
>
> Key: OAK-4558
> URL: https://issues.apache.org/jira/browse/OAK-4558
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: segment-tar
>Reporter: Francesco Mari
>Assignee: Francesco Mari
>Priority: Minor
> Labels: performance
>
> The implementation of {{SegmentNodeState.fastEquals()}} compares the stable
> IDs of two instances of {{SegmentNodeState}}. In some cases, reading the
> stable ID would trigger a read of an additional record, the block record
> containing the serialized version of the segment ID.
> This issue is about evaluating the performance implications of this strategy
> and, in particular, if it would be better to store the serialized stable ID
> in the node record itself.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-6031) Add TarFiles to the architecture diagram
[
https://issues.apache.org/jira/browse/OAK-6031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16376762#comment-16376762
]
Francesco Mari commented on OAK-6031:
-
Backported to 1.8 at r1825364.
> Add TarFiles to the architecture diagram
>
>
> Key: OAK-6031
> URL: https://issues.apache.org/jira/browse/OAK-6031
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: doc, segment-tar
>Affects Versions: 1.8.0
>Reporter: Francesco Mari
>Assignee: Francesco Mari
>Priority: Major
> Labels: documentation
> Fix For: 1.9.0, 1.10, 1.8.3
>
>
> The newly created {{TarFiles}} should be added to the architecture diagram
> for oak-segment-tar.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (OAK-6031) Add TarFiles to the architecture diagram
[
https://issues.apache.org/jira/browse/OAK-6031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francesco Mari resolved OAK-6031.
-
Resolution: Fixed
> Add TarFiles to the architecture diagram
>
>
> Key: OAK-6031
> URL: https://issues.apache.org/jira/browse/OAK-6031
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: doc, segment-tar
>Affects Versions: 1.8.0
>Reporter: Francesco Mari
>Assignee: Francesco Mari
>Priority: Major
> Labels: documentation
> Fix For: 1.9.0, 1.10, 1.8.3
>
>
> The newly created {{TarFiles}} should be added to the architecture diagram
> for oak-segment-tar.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-6031) Add TarFiles to the architecture diagram
[
https://issues.apache.org/jira/browse/OAK-6031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16376755#comment-16376755
]
Francesco Mari commented on OAK-6031:
-
Fixed on trunk at r1825362.
> Add TarFiles to the architecture diagram
>
>
> Key: OAK-6031
> URL: https://issues.apache.org/jira/browse/OAK-6031
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: doc, segment-tar
>Affects Versions: 1.8.0
>Reporter: Francesco Mari
>Assignee: Francesco Mari
>Priority: Major
> Labels: documentation
> Fix For: 1.9.0, 1.10, 1.8.3
>
>
> The newly created {{TarFiles}} should be added to the architecture diagram
> for oak-segment-tar.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
