[jira] [Resolved] (OAK-10581) Remove mock stubbing at the end of the test method in AzureArchiveManagerTest.testWriteAfterLosingRepoLock
[ https://issues.apache.org/jira/browse/OAK-10581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miroslav Smiljanic resolved OAK-10581. -- Fix Version/s: 1.62.0 Resolution: Fixed > Remove mock stubbing at the end of the test method in > AzureArchiveManagerTest.testWriteAfterLosingRepoLock > -- > > Key: OAK-10581 > URL: https://issues.apache.org/jira/browse/OAK-10581 > Project: Jackrabbit Oak > Issue Type: Task > Components: segment-azure >Affects Versions: 1.60.0 >Reporter: Miroslav Smiljanic >Assignee: Miroslav Smiljanic >Priority: Major > Fix For: 1.62.0 > > > Mock stubbing at the end of method > [AzureArchiveManagerTest#testWriteAfterLosingRepoLock|https://github.com/apache/jackrabbit-oak/blob/4c2cac63eef3d9b538c8b855729f02b2090760a8/oak-segment-azure/src/test/java/org/apache/jackrabbit/oak/segment/azure/AzureArchiveManagerTest.java#L551] > is causing the test to fail sometimes: > https://ci-builds.apache.org/job/Jackrabbit/job/jackrabbit-oak-trunk/org.apache.jackrabbit$oak-segment-azure/1301/testReport/junit/org.apache.jackrabbit.oak.segment.azure/AzureArchiveManagerTest/testWriteAfterLosingRepoLock/ > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OAK-10581) Remove mock stubbing at the end of the test method in AzureArchiveManagerTest.testWriteAfterLosingRepoLock
[ https://issues.apache.org/jira/browse/OAK-10581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17798644#comment-17798644 ] Miroslav Smiljanic commented on OAK-10581: -- Change pushed to trunk [https://github.com/apache/jackrabbit-oak/commit/0b8f4ab2e736c6561ae745a5fe6040a59911eeb3] > Remove mock stubbing at the end of the test method in > AzureArchiveManagerTest.testWriteAfterLosingRepoLock > -- > > Key: OAK-10581 > URL: https://issues.apache.org/jira/browse/OAK-10581 > Project: Jackrabbit Oak > Issue Type: Task > Components: segment-azure >Affects Versions: 1.60.0 >Reporter: Miroslav Smiljanic >Assignee: Miroslav Smiljanic >Priority: Major > > Mock stubbing at the end of method > [AzureArchiveManagerTest#testWriteAfterLosingRepoLock|https://github.com/apache/jackrabbit-oak/blob/4c2cac63eef3d9b538c8b855729f02b2090760a8/oak-segment-azure/src/test/java/org/apache/jackrabbit/oak/segment/azure/AzureArchiveManagerTest.java#L551] > is causing the test to fail sometimes: > https://ci-builds.apache.org/job/Jackrabbit/job/jackrabbit-oak-trunk/org.apache.jackrabbit$oak-segment-azure/1301/testReport/junit/org.apache.jackrabbit.oak.segment.azure/AzureArchiveManagerTest/testWriteAfterLosingRepoLock/ > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (OAK-10591) Bump netty dependency from 4.1.96.Final to 4.1.104.Final
[ https://issues.apache.org/jira/browse/OAK-10591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrei Dulceanu updated OAK-10591: -- Description: *File Matche(s):* /netty-common-4.1.96.Final.jar *Vulnerabilitie(s)* This artifact embeds Netty Project 4.1.96.Final which contains the following vulnerabilitie(s): *BDSA-2023-2732/CVE-2023-44487* in version 4.1.96.Final (CVSS 7.5 High): The HTTP/2 protocol contains a flaw related to the stream multiplexing feature that can allow for excessive resource consumption on servers operating implementations of the HTTP/2 protocol. The HTTP/2 protocol allows clients to signal to a server to cancel a previously opened stream by sending an `RST_STREAM` frame. Attackers can abuse this stream canceling ability by opening a large number of streams at once immediately followed by `RST_STREAM` frames. In most HTTP/2 implementations this bypasses concurrent open stream limits and causes servers to spend processing time first handling request frames and then performing stream tear downs. For the server, these operations can pile up whereas the attacker client paid minuscule bandwidth and processing costs. [Amazon](https://aws.amazon.com/security/security-bulletins/AWS-2023-011/), [Cloudflare](https://blog.cloudflare.com/technical-breakdown-http2-rapid-reset-ddos-attack/) and [Google](https://cloud.google.com/blog/products/identity-security/google-cloud-mitigated-largest-ddos-attack-peaking-above-398-million-rps/) have reported that this vulnerability has been exploited in the wild from August to October 2023. This vulnerability is listed as exploitable by the Cybersecurity & Infrastructure Security Agency in their [Known Exploited Vulnerabilities Catalog](https://www.cisa.gov/known-exploited-vulnerabilities-catalog). was: io.netty : netty-codec : 4.1.52.Final sonatype-2021-0789 *Summary*: sonatype-2021-0789 Explanation The netty-codec package contains a Buffer Overflow vulnerability. The finishEncode function in the Lz4FrameEncoder.class class incorrectly estimates the buffer size when writing a footer for the last header. An attacker could abuse this behavior by sending a payload to the flawed application that will overwrite contiguous memory chunks in the heap, resulting in a Denial of Service (DoS) condition or other unintended behavior. Detection The application is vulnerable by using this component. Recommendation We recommend upgrading to a version of this component that is not vulnerable to this specific issue. Note: If this component is included as a bundled/transitive dependency of another component, there may not be an upgrade path. In this instance, we recommend contacting the maintainers who included the vulnerable package. Alternatively, we recommend investigating alternative components or a potential mitigating control. Root Cause netty-codec-4.1.52.Final.jar <= io/netty/handler/codec/compression/Lz4FrameEncoder.class:[4.1.0.Beta2 , 4.1.66.Final) Advisories Project: [https://github.com/netty/netty/pull/11429] > Bump netty dependency from 4.1.96.Final to 4.1.104.Final > > > Key: OAK-10591 > URL: https://issues.apache.org/jira/browse/OAK-10591 > Project: Jackrabbit Oak > Issue Type: Task > Components: segment-tar >Reporter: Andrei Dulceanu >Assignee: Andrei Dulceanu >Priority: Major > Labels: vulnerability > Fix For: 1.62.0 > > > *File Matche(s):* > /netty-common-4.1.96.Final.jar > *Vulnerabilitie(s)* > This artifact embeds Netty Project 4.1.96.Final which contains the following > vulnerabilitie(s): > *BDSA-2023-2732/CVE-2023-44487* in version 4.1.96.Final (CVSS 7.5 High): The > HTTP/2 protocol contains a flaw related to the stream multiplexing feature > that can allow for excessive resource consumption on servers operating > implementations of the HTTP/2 protocol. The HTTP/2 protocol allows clients to > signal to a server to cancel a previously opened stream by sending an > `RST_STREAM` frame. Attackers can abuse this stream canceling ability by > opening a large number of streams at once immediately followed by > `RST_STREAM` frames. In most HTTP/2 implementations this bypasses concurrent > open stream limits and causes servers to spend processing time first handling > request frames and then performing stream tear downs. For the server, these > operations can pile up whereas the attacker client paid minuscule bandwidth > and processing costs. > [Amazon](https://aws.amazon.com/security/security-bulletins/AWS-2023-011/), > [Cloudflare](https://blog.cloudflare.com/technical-breakdown-http2-rapid-reset-ddos-attack/) > and >
[jira] [Updated] (OAK-10591) Bump netty dependency from 4.1.96.Final to 4.1.104.Final
[ https://issues.apache.org/jira/browse/OAK-10591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrei Dulceanu updated OAK-10591: -- Summary: Bump netty dependency from 4.1.96.Final to 4.1.104.Final (was: Bump netty dependency from 4.1.96.Final to 4.1.66.Final) > Bump netty dependency from 4.1.96.Final to 4.1.104.Final > > > Key: OAK-10591 > URL: https://issues.apache.org/jira/browse/OAK-10591 > Project: Jackrabbit Oak > Issue Type: Task > Components: segment-tar >Reporter: Andrei Dulceanu >Assignee: Andrei Dulceanu >Priority: Major > Labels: vulnerability > Fix For: 1.62.0 > > > io.netty : netty-codec : 4.1.52.Final sonatype-2021-0789 > *Summary*: > sonatype-2021-0789 > Explanation > The netty-codec package contains a Buffer Overflow vulnerability. The > finishEncode function in the Lz4FrameEncoder.class class incorrectly > estimates the buffer size when writing a footer for the last header. An > attacker could abuse this behavior by sending a payload to the flawed > application that will overwrite contiguous memory chunks in the heap, > resulting in a Denial of Service (DoS) condition or other unintended behavior. > Detection > The application is vulnerable by using this component. > Recommendation > We recommend upgrading to a version of this component that is not vulnerable > to this specific issue. > Note: If this component is included as a bundled/transitive dependency of > another component, there may not be an upgrade path. In this instance, we > recommend contacting the maintainers who included the vulnerable package. > Alternatively, we recommend investigating alternative components or a > potential mitigating control. > Root Cause > netty-codec-4.1.52.Final.jar <= > io/netty/handler/codec/compression/Lz4FrameEncoder.class:[4.1.0.Beta2 , > 4.1.66.Final) > Advisories > Project: > [https://github.com/netty/netty/pull/11429] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (OAK-10591) Bump netty dependency from 4.1.96.Final to 4.1.66.Final
[ https://issues.apache.org/jira/browse/OAK-10591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrei Dulceanu updated OAK-10591: -- Reporter: Andrei Dulceanu (was: Arun Kumar Ram) > Bump netty dependency from 4.1.96.Final to 4.1.66.Final > --- > > Key: OAK-10591 > URL: https://issues.apache.org/jira/browse/OAK-10591 > Project: Jackrabbit Oak > Issue Type: Task > Components: segment-tar >Reporter: Andrei Dulceanu >Assignee: Andrei Dulceanu >Priority: Major > Labels: vulnerability > Fix For: 1.42.0, 1.22.9 > > > io.netty : netty-codec : 4.1.52.Final sonatype-2021-0789 > *Summary*: > sonatype-2021-0789 > Explanation > The netty-codec package contains a Buffer Overflow vulnerability. The > finishEncode function in the Lz4FrameEncoder.class class incorrectly > estimates the buffer size when writing a footer for the last header. An > attacker could abuse this behavior by sending a payload to the flawed > application that will overwrite contiguous memory chunks in the heap, > resulting in a Denial of Service (DoS) condition or other unintended behavior. > Detection > The application is vulnerable by using this component. > Recommendation > We recommend upgrading to a version of this component that is not vulnerable > to this specific issue. > Note: If this component is included as a bundled/transitive dependency of > another component, there may not be an upgrade path. In this instance, we > recommend contacting the maintainers who included the vulnerable package. > Alternatively, we recommend investigating alternative components or a > potential mitigating control. > Root Cause > netty-codec-4.1.52.Final.jar <= > io/netty/handler/codec/compression/Lz4FrameEncoder.class:[4.1.0.Beta2 , > 4.1.66.Final) > Advisories > Project: > [https://github.com/netty/netty/pull/11429] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (OAK-10591) Bump netty dependency from 4.1.96.Final to 4.1.66.Final
[ https://issues.apache.org/jira/browse/OAK-10591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrei Dulceanu updated OAK-10591: -- Fix Version/s: 1.62.0 (was: 1.42.0) (was: 1.22.9) > Bump netty dependency from 4.1.96.Final to 4.1.66.Final > --- > > Key: OAK-10591 > URL: https://issues.apache.org/jira/browse/OAK-10591 > Project: Jackrabbit Oak > Issue Type: Task > Components: segment-tar >Reporter: Andrei Dulceanu >Assignee: Andrei Dulceanu >Priority: Major > Labels: vulnerability > Fix For: 1.62.0 > > > io.netty : netty-codec : 4.1.52.Final sonatype-2021-0789 > *Summary*: > sonatype-2021-0789 > Explanation > The netty-codec package contains a Buffer Overflow vulnerability. The > finishEncode function in the Lz4FrameEncoder.class class incorrectly > estimates the buffer size when writing a footer for the last header. An > attacker could abuse this behavior by sending a payload to the flawed > application that will overwrite contiguous memory chunks in the heap, > resulting in a Denial of Service (DoS) condition or other unintended behavior. > Detection > The application is vulnerable by using this component. > Recommendation > We recommend upgrading to a version of this component that is not vulnerable > to this specific issue. > Note: If this component is included as a bundled/transitive dependency of > another component, there may not be an upgrade path. In this instance, we > recommend contacting the maintainers who included the vulnerable package. > Alternatively, we recommend investigating alternative components or a > potential mitigating control. > Root Cause > netty-codec-4.1.52.Final.jar <= > io/netty/handler/codec/compression/Lz4FrameEncoder.class:[4.1.0.Beta2 , > 4.1.66.Final) > Advisories > Project: > [https://github.com/netty/netty/pull/11429] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (OAK-10591) Bump netty dependency from 4.1.96.Final to 4.1.66.Final
[ https://issues.apache.org/jira/browse/OAK-10591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrei Dulceanu updated OAK-10591: -- Summary: Bump netty dependency from 4.1.96.Final to 4.1.66.Final (was: CLONE - Bump netty dependency from 4.1.52.Final to 4.1.66.Final) > Bump netty dependency from 4.1.96.Final to 4.1.66.Final > --- > > Key: OAK-10591 > URL: https://issues.apache.org/jira/browse/OAK-10591 > Project: Jackrabbit Oak > Issue Type: Task > Components: segment-tar >Reporter: Arun Kumar Ram >Assignee: Andrei Dulceanu >Priority: Major > Labels: vulnerability > Fix For: 1.42.0, 1.22.9 > > > io.netty : netty-codec : 4.1.52.Final sonatype-2021-0789 > *Summary*: > sonatype-2021-0789 > Explanation > The netty-codec package contains a Buffer Overflow vulnerability. The > finishEncode function in the Lz4FrameEncoder.class class incorrectly > estimates the buffer size when writing a footer for the last header. An > attacker could abuse this behavior by sending a payload to the flawed > application that will overwrite contiguous memory chunks in the heap, > resulting in a Denial of Service (DoS) condition or other unintended behavior. > Detection > The application is vulnerable by using this component. > Recommendation > We recommend upgrading to a version of this component that is not vulnerable > to this specific issue. > Note: If this component is included as a bundled/transitive dependency of > another component, there may not be an upgrade path. In this instance, we > recommend contacting the maintainers who included the vulnerable package. > Alternatively, we recommend investigating alternative components or a > potential mitigating control. > Root Cause > netty-codec-4.1.52.Final.jar <= > io/netty/handler/codec/compression/Lz4FrameEncoder.class:[4.1.0.Beta2 , > 4.1.66.Final) > Advisories > Project: > [https://github.com/netty/netty/pull/11429] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (OAK-10591) CLONE - Bump netty dependency from 4.1.52.Final to 4.1.66.Final
Andrei Dulceanu created OAK-10591: - Summary: CLONE - Bump netty dependency from 4.1.52.Final to 4.1.66.Final Key: OAK-10591 URL: https://issues.apache.org/jira/browse/OAK-10591 Project: Jackrabbit Oak Issue Type: Task Components: segment-tar Reporter: Arun Kumar Ram Assignee: Andrei Dulceanu Fix For: 1.42.0, 1.22.9 io.netty : netty-codec : 4.1.52.Final sonatype-2021-0789 *Summary*: sonatype-2021-0789 Explanation The netty-codec package contains a Buffer Overflow vulnerability. The finishEncode function in the Lz4FrameEncoder.class class incorrectly estimates the buffer size when writing a footer for the last header. An attacker could abuse this behavior by sending a payload to the flawed application that will overwrite contiguous memory chunks in the heap, resulting in a Denial of Service (DoS) condition or other unintended behavior. Detection The application is vulnerable by using this component. Recommendation We recommend upgrading to a version of this component that is not vulnerable to this specific issue. Note: If this component is included as a bundled/transitive dependency of another component, there may not be an upgrade path. In this instance, we recommend contacting the maintainers who included the vulnerable package. Alternatively, we recommend investigating alternative components or a potential mitigating control. Root Cause netty-codec-4.1.52.Final.jar <= io/netty/handler/codec/compression/Lz4FrameEncoder.class:[4.1.0.Beta2 , 4.1.66.Final) Advisories Project: [https://github.com/netty/netty/pull/11429] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (OAK-10587) Exception in bulk update operations in finalizing forced reindex
[
https://issues.apache.org/jira/browse/OAK-10587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mohammad Mahdavi resolved OAK-10587.
Resolution: Duplicate
> Exception in bulk update operations in finalizing forced reindex
> -
>
> Key: OAK-10587
> URL: https://issues.apache.org/jira/browse/OAK-10587
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: indexing, solr
>Affects Versions: 1.58.0
>Reporter: Mohammad Mahdavi
>Priority: Major
> Attachments: log.txt
>
>
> I used Oak as a repository to save our records since 3 years ago. Now our
> nodes table contains more than 4 billion records. About 1 week ago we had a
> crash in the application host and after restarting I saw the below error in
> one of our Oak instances:
> {code:java}
> WARN 1 --- [d-executor-2429] o.a.j.o.plugins.index.AsyncIndexUpdate :
> [async] Failed to retrieve previously indexed checkpoint r18bb4ef8f7c-0-b9;
> re-running the initial index update
> INFO 1 --- [ex-update-async] o.a.j.oak.plugins.index.IndexUpdate : Found
> a new index node [solr]. Reindexing is requested
> INFO 1 --- [ex-update-async] o.a.j.oak.plugins.index.IndexUpdate :
> Reindexing will be performed for following indexes: [/oak:index/solr]
> 2023-12-16 11:25:42.102 INFO 1 --- [ex-update-async]
> o.a.j.oak.plugins.index.IndexUpdate : Estimated node count to be
> traversed for reindexing under / is [387210240]{code}
> So I should wait until it finishes the reindexing but after more than 6 days
> of waiting I got:
> {quote}DocumentStoreException: Following exceptions occurred during the bulk
> update operations
> {quote}
> The full log is:
> {code:java}
> 2023-12-13 18:44:10.203 INFO [ex-update-async]
> o.a.j.oak.plugins.index.IndexUpdate /oak:index/solr => Indexed 34647
> nodes in 11.07 s ...2023-12-13 18:44:21.554 INFO [ex-update-async]
> o.a.j.oak.plugins.index.IndexUpdate Reindexing Traversed #34648
> /jcr:system/jcr:versionStorage/59/8a/a6/598aa69c-3d3c-4a75-a7af-f7795164960c/1.2/jcr:frozenNode
> [1087.98 nodes/s, 3916737.05 nodes/hr] (Elapsed 3.686 d, Expected 10.40 h,
> Completed 89.48%)2023-12-13 18:44:21.560 INFO [ex-update-async]
> o.a.j.oak.plugins.index.IndexUpdate /oak:index/solr => Indexed 34648
> nodes in 11.36 s ...2023-12-13 18:44:32.446 INFO [ex-update-async]
> o.a.j.oak.plugins.index.IndexUpdate Reindexing Traversed #34649
> /jcr:system/jcr:versionStorage/6f/01 [1087.98 nodes/s, 3916714.80 nodes/hr]
> (Elapsed 3.686 d, Expected 10.40 h, Completed 89.48%)2023-12-13 18:44:32.456
> INFO [ex-update-async] o.a.j.oak.plugins.index.IndexUpdate /oak:index/solr
> => Indexed 34649 nodes in 10.90 s ...2023-12-13 18:44:43.236 INFO
> [ex-update-async] o.a.j.oak.plugins.index.IndexUpdate Reindexing Traversed
> #34650
> /jcr:system/jcr:versionStorage/86/2e/61/862e61c4-7a25-4fe6-b669-689f8ddf51fa/1.3/jcr:frozenNode
> [1087.97 nodes/s, 3916704.86 nodes/hr] (Elapsed 3.686 d, Expected 10.39 h,
> Completed 89.49%)2023-12-13 18:44:43.244 INFO [ex-update-async]
> o.a.j.oak.plugins.index.IndexUpdate /oak:index/solr => Indexed 34650
> nodes in 10.79 s ...2023-12-13 18:44:55.676 INFO [ex-update-async]
> o.a.j.oak.plugins.index.IndexUpdate Reindexing Traversed #34651
> /jcr:system/jcr:versionStorage/9d/93/b7/9d93b723-ea4d-40cf-a7e0-6c81e5bf9fad/1.1
> [1087.96 nodes/s, 3916658.02 nodes/hr] (Elapsed 3.686 d, Expected 10.39 h,
> Completed 89.49%)2023-12-13 18:44:55.681 INFO [ex-update-async]
> o.a.j.oak.plugins.index.IndexUpdate /oak:index/solr => Indexed 34651
> nodes in 12.44 s ...2023-12-13 18:45:07.915 INFO [ex-update-async]
> o.a.j.oak.plugins.index.IndexUpdate Reindexing Traversed #34652
> /jcr:system/jcr:versionStorage/b3/7b/9f/b37b9f8c-9d8c-42b2-8dc6-fe8732769ea8/jcr:rootVersion
> [1087.95 nodes/s, 3916623.49 nodes/hr] (Elapsed 3.686 d, Expected 10.39 h,
> Completed 89.49%)2023-12-13 18:45:07.927 INFO [ex-update-async]
> o.a.j.oak.plugins.index.IndexUpdate /oak:index/solr => Indexed 34652
> nodes in 12.25 s ...2023-12-13 18:45:19.325 INFO [ex-update-async]
> o.a.j.oak.plugins.index.IndexUpdate Reindexing Traversed #34653
> /jcr:system/jcr:versionStorage/c9/57/24 [1087.94 nodes/s, 3916588.95
> nodes/hr] (Elapsed 3.687 d, Expected 10.39 h, Completed 89.49%)2023-12-13
> 18:45:19.338 INFO [ex-update-async] o.a.j.oak.plugins.index.IndexUpdate
> /oak:index/solr => Indexed 34653 nodes in 11.41 s ...2023-12-13
> 18:45:29.118 INFO [ex-update-async] o.a.j.oak.plugins.index.IndexUpdate
> Reindexing Traversed #34654
> /jcr:system/jcr:versionStorage/df/fc/4f/dffc4fa4-e531-4059-86b0-4c37fe41ace5/1.4/jcr:frozenNode
> [1087.94 nodes/s, 3916591.31 nodes/hr] (Elapsed 3.687 d, Expected
[jira] [Commented] (OAK-10587) Exception in bulk update operations in finalizing forced reindex
[
https://issues.apache.org/jira/browse/OAK-10587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17798556#comment-17798556
]
Mohammad Mahdavi commented on OAK-10587:
It's a duplicate one and should be removed.
> Exception in bulk update operations in finalizing forced reindex
> -
>
> Key: OAK-10587
> URL: https://issues.apache.org/jira/browse/OAK-10587
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: indexing, solr
>Affects Versions: 1.58.0
>Reporter: Mohammad Mahdavi
>Priority: Major
> Attachments: log.txt
>
>
> I used Oak as a repository to save our records since 3 years ago. Now our
> nodes table contains more than 4 billion records. About 1 week ago we had a
> crash in the application host and after restarting I saw the below error in
> one of our Oak instances:
> {code:java}
> WARN 1 --- [d-executor-2429] o.a.j.o.plugins.index.AsyncIndexUpdate :
> [async] Failed to retrieve previously indexed checkpoint r18bb4ef8f7c-0-b9;
> re-running the initial index update
> INFO 1 --- [ex-update-async] o.a.j.oak.plugins.index.IndexUpdate : Found
> a new index node [solr]. Reindexing is requested
> INFO 1 --- [ex-update-async] o.a.j.oak.plugins.index.IndexUpdate :
> Reindexing will be performed for following indexes: [/oak:index/solr]
> 2023-12-16 11:25:42.102 INFO 1 --- [ex-update-async]
> o.a.j.oak.plugins.index.IndexUpdate : Estimated node count to be
> traversed for reindexing under / is [387210240]{code}
> So I should wait until it finishes the reindexing but after more than 6 days
> of waiting I got:
> {quote}DocumentStoreException: Following exceptions occurred during the bulk
> update operations
> {quote}
> The full log is:
> {code:java}
> 2023-12-13 18:44:10.203 INFO [ex-update-async]
> o.a.j.oak.plugins.index.IndexUpdate /oak:index/solr => Indexed 34647
> nodes in 11.07 s ...2023-12-13 18:44:21.554 INFO [ex-update-async]
> o.a.j.oak.plugins.index.IndexUpdate Reindexing Traversed #34648
> /jcr:system/jcr:versionStorage/59/8a/a6/598aa69c-3d3c-4a75-a7af-f7795164960c/1.2/jcr:frozenNode
> [1087.98 nodes/s, 3916737.05 nodes/hr] (Elapsed 3.686 d, Expected 10.40 h,
> Completed 89.48%)2023-12-13 18:44:21.560 INFO [ex-update-async]
> o.a.j.oak.plugins.index.IndexUpdate /oak:index/solr => Indexed 34648
> nodes in 11.36 s ...2023-12-13 18:44:32.446 INFO [ex-update-async]
> o.a.j.oak.plugins.index.IndexUpdate Reindexing Traversed #34649
> /jcr:system/jcr:versionStorage/6f/01 [1087.98 nodes/s, 3916714.80 nodes/hr]
> (Elapsed 3.686 d, Expected 10.40 h, Completed 89.48%)2023-12-13 18:44:32.456
> INFO [ex-update-async] o.a.j.oak.plugins.index.IndexUpdate /oak:index/solr
> => Indexed 34649 nodes in 10.90 s ...2023-12-13 18:44:43.236 INFO
> [ex-update-async] o.a.j.oak.plugins.index.IndexUpdate Reindexing Traversed
> #34650
> /jcr:system/jcr:versionStorage/86/2e/61/862e61c4-7a25-4fe6-b669-689f8ddf51fa/1.3/jcr:frozenNode
> [1087.97 nodes/s, 3916704.86 nodes/hr] (Elapsed 3.686 d, Expected 10.39 h,
> Completed 89.49%)2023-12-13 18:44:43.244 INFO [ex-update-async]
> o.a.j.oak.plugins.index.IndexUpdate /oak:index/solr => Indexed 34650
> nodes in 10.79 s ...2023-12-13 18:44:55.676 INFO [ex-update-async]
> o.a.j.oak.plugins.index.IndexUpdate Reindexing Traversed #34651
> /jcr:system/jcr:versionStorage/9d/93/b7/9d93b723-ea4d-40cf-a7e0-6c81e5bf9fad/1.1
> [1087.96 nodes/s, 3916658.02 nodes/hr] (Elapsed 3.686 d, Expected 10.39 h,
> Completed 89.49%)2023-12-13 18:44:55.681 INFO [ex-update-async]
> o.a.j.oak.plugins.index.IndexUpdate /oak:index/solr => Indexed 34651
> nodes in 12.44 s ...2023-12-13 18:45:07.915 INFO [ex-update-async]
> o.a.j.oak.plugins.index.IndexUpdate Reindexing Traversed #34652
> /jcr:system/jcr:versionStorage/b3/7b/9f/b37b9f8c-9d8c-42b2-8dc6-fe8732769ea8/jcr:rootVersion
> [1087.95 nodes/s, 3916623.49 nodes/hr] (Elapsed 3.686 d, Expected 10.39 h,
> Completed 89.49%)2023-12-13 18:45:07.927 INFO [ex-update-async]
> o.a.j.oak.plugins.index.IndexUpdate /oak:index/solr => Indexed 34652
> nodes in 12.25 s ...2023-12-13 18:45:19.325 INFO [ex-update-async]
> o.a.j.oak.plugins.index.IndexUpdate Reindexing Traversed #34653
> /jcr:system/jcr:versionStorage/c9/57/24 [1087.94 nodes/s, 3916588.95
> nodes/hr] (Elapsed 3.687 d, Expected 10.39 h, Completed 89.49%)2023-12-13
> 18:45:19.338 INFO [ex-update-async] o.a.j.oak.plugins.index.IndexUpdate
> /oak:index/solr => Indexed 34653 nodes in 11.41 s ...2023-12-13
> 18:45:29.118 INFO [ex-update-async] o.a.j.oak.plugins.index.IndexUpdate
> Reindexing Traversed #34654
> /jcr:system/jcr:versionStorage/df/fc/4f/dffc4fa4-e531-4059-86b0-4c37fe41ace5/1.4/jcr:frozenNode
> [1087.94
[jira] [Created] (OAK-10590) Indexing job downloads and creates FFS with full node store if includedPaths is specified as a string instead of array of strings
Nuno Santos created OAK-10590:
-
Summary: Indexing job downloads and creates FFS with full node
store if includedPaths is specified as a string instead of array of strings
Key: OAK-10590
URL: https://issues.apache.org/jira/browse/OAK-10590
Project: Jackrabbit Oak
Issue Type: Bug
Components: indexing
Reporter: Nuno Santos
The {{includedPaths}} property of an index definition should be an array of
strings.
If it is instead specified as a String, like in this example:
{noformat}
"includedPaths": "/a/b", {noformat}
The indexing job defaults to using the {{/}} as the value for includedPaths,
and therefore downloads the full node store and creates an FFS containing
everything except the hidden paths. The logic that handles this case is here:
[https://github.com/apache/jackrabbit-oak/blob/0b8f4ab2e736c6561ae745a5fe6040a59911eeb3/oak-store-spi/src/main/java/org/apache/jackrabbit/oak/spi/filter/PathFilter.java#L95-L103]
This will slow down significantly the indexing, as it will negate any benefits
from using regex filtering. And even if regex filtering is not enabled or
cannot be used, using / as includedPaths will also result in the FFS containing
more nodes than it should, which will once again slow down the indexing job.
Suggested fix: if includedPaths is a String, treat it as a one element array
and at the same time log a warning.
Additionally, apply the same fix to other properties in the index definition:
* {{excludedPaths}}
* {{includedPaths}}
* {{queryPaths}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
