[jira] [Commented] (OAK-9094) Error message "Could not access the Lucene index" / FileNotFoundException
[
https://issues.apache.org/jira/browse/OAK-9094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17186269#comment-17186269
]
Mohit Kataria commented on OAK-9094:
Updated fix version.
> Error message "Could not access the Lucene index" / FileNotFoundException
> -
>
> Key: OAK-9094
> URL: https://issues.apache.org/jira/browse/OAK-9094
> Project: Jackrabbit Oak
> Issue Type: Improvement
>Reporter: Mohit Kataria
>Assignee: Mohit Kataria
>Priority: Minor
> Fix For: 1.32.0
>
>
> It may happen that between retrieving files from remote and evaluating size
> of file, the remote file may get get removed leading to IO exception. Adding
> a check to handle this case.
> {noformat}
> IndexSanityChecker
> private boolean isThereASizeMismatch() throws IOException {
> for (String fileName : remote.listAll()){
> long localLength = DirectoryUtils.getFileLength(local, fileName);
> long remoteLength = remote.fileLength(fileName); // <== line 81
> //This is a weak check based on length.
> if (localLength > 0 && localLength != remoteLength){
> log.warn("[{}] Found local copy for {} in {} but size of
> local {} differs from remote {}. ",
> indexPath, fileName, local, localLength,
> remoteLength);
> return true;
> }
> if (localLength > 0) {
> localDirSize += localLength;
> localFileCount++;
> }
> remoteDirSize += remoteLength;
> remoteFileCount++;
> }
> return false;
> }
> OakDirectory:
>@Override
> public long fileLength(String name) throws IOException {
> NodeBuilder file = directoryBuilder.getChildNode(name);
> if (!file.exists()) {
> String msg = String.format("[%s] %s", indexName, name);
> throw new FileNotFoundException(msg);
> }
> OakIndexInput input = new OakIndexInput(name, file, indexName,
> blobFactory);
> try {
> return input.length();
> } finally {
> input.close();
> }
> }
> {noformat}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (OAK-9094) Error message "Could not access the Lucene index" / FileNotFoundException
[
https://issues.apache.org/jira/browse/OAK-9094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mohit Kataria updated OAK-9094:
---
Fix Version/s: 1.32.0
> Error message "Could not access the Lucene index" / FileNotFoundException
> -
>
> Key: OAK-9094
> URL: https://issues.apache.org/jira/browse/OAK-9094
> Project: Jackrabbit Oak
> Issue Type: Improvement
>Reporter: Mohit Kataria
>Assignee: Mohit Kataria
>Priority: Minor
> Fix For: 1.32.0
>
>
> It may happen that between retrieving files from remote and evaluating size
> of file, the remote file may get get removed leading to IO exception. Adding
> a check to handle this case.
> {noformat}
> IndexSanityChecker
> private boolean isThereASizeMismatch() throws IOException {
> for (String fileName : remote.listAll()){
> long localLength = DirectoryUtils.getFileLength(local, fileName);
> long remoteLength = remote.fileLength(fileName); // <== line 81
> //This is a weak check based on length.
> if (localLength > 0 && localLength != remoteLength){
> log.warn("[{}] Found local copy for {} in {} but size of
> local {} differs from remote {}. ",
> indexPath, fileName, local, localLength,
> remoteLength);
> return true;
> }
> if (localLength > 0) {
> localDirSize += localLength;
> localFileCount++;
> }
> remoteDirSize += remoteLength;
> remoteFileCount++;
> }
> return false;
> }
> OakDirectory:
>@Override
> public long fileLength(String name) throws IOException {
> NodeBuilder file = directoryBuilder.getChildNode(name);
> if (!file.exists()) {
> String msg = String.format("[%s] %s", indexName, name);
> throw new FileNotFoundException(msg);
> }
> OakIndexInput input = new OakIndexInput(name, file, indexName,
> blobFactory);
> try {
> return input.length();
> } finally {
> input.close();
> }
> }
> {noformat}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (OAK-9028) upgrade to mockito-core 3.3.3
[ https://issues.apache.org/jira/browse/OAK-9028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Reschke updated OAK-9028: Fix Version/s: 1.30.0 > upgrade to mockito-core 3.3.3 > - > > Key: OAK-9028 > URL: https://issues.apache.org/jira/browse/OAK-9028 > Project: Jackrabbit Oak > Issue Type: Task > Components: parent >Reporter: Julian Reschke >Assignee: Julian Reschke >Priority: Minor > Labels: candidate_oak_1_22 > Fix For: 1.30.0 > > > Needed für certain mocking features to work with Java 14. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (OAK-9043) The Oak parent pom should override the version of the Maven Dependency Plugin
[
https://issues.apache.org/jira/browse/OAK-9043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185962#comment-17185962
]
Julian Reschke commented on OAK-9043:
-
fixVersion?
> The Oak parent pom should override the version of the Maven Dependency Plugin
> -
>
> Key: OAK-9043
> URL: https://issues.apache.org/jira/browse/OAK-9043
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: parent
>Reporter: Manfred Baedke
>Assignee: Manfred Baedke
>Priority: Minor
>
> Oak is current using
> https://github.com/apache/maven-apache-parent/blob/apache-18/pom.xml, which
> specifies version 2.10 of the Maven Dependency Plugin.
> This should be overridden with version 3.1.2 in the Oak parent pom.
> Note that with v2.10,
> {code}mvn dependency:tree -Dverbose{code}
> gives the warning
> {code}Using Maven 2 dependency tree to get verbose output, which may be
> inconsistent with actual Maven 3 resolution{code}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (OAK-9109) Upgrade to Elasticsearch 7.7.1
[ https://issues.apache.org/jira/browse/OAK-9109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185959#comment-17185959 ] Julian Reschke commented on OAK-9109: - fixVersion? > Upgrade to Elasticsearch 7.7.1 > -- > > Key: OAK-9109 > URL: https://issues.apache.org/jira/browse/OAK-9109 > Project: Jackrabbit Oak > Issue Type: Task > Components: elastic-search, indexing, oak-search >Reporter: Fabrizio Fortino >Assignee: Fabrizio Fortino >Priority: Major > > > Release Notes > [https://www.elastic.co/guide/en/elasticsearch/reference/7.7/release-notes-7.7.1.html|https://www.elastic.co/guide/en/elasticsearch/reference/7.7/release-notes-7.7.0.html] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (OAK-9091) oak-search-elastic: review query code
[ https://issues.apache.org/jira/browse/OAK-9091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185961#comment-17185961 ] Julian Reschke commented on OAK-9091: - fixVersion? > oak-search-elastic: review query code > - > > Key: OAK-9091 > URL: https://issues.apache.org/jira/browse/OAK-9091 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: elastic-search, indexing >Reporter: Fabrizio Fortino >Assignee: Fabrizio Fortino >Priority: Major > > The code to execute queries in the elastic module needs to be improved. > It has been inherited from the first prototype but has never been > reviewed/refactored. > Here is what needs to be done: > * ElasticsearchRowIterator [0] is the central point. We need to break it up > and separate the query building phase from the interpretation of the response > * Centralize how we resolve property names. This affects also mapping and > indexing phases > * We need to review how the queries are composed to increase performance. As > a rule of thumb, we should always create a BoolQuery and leverage as much as > possible the use of filters over matching queries. This allows us to use ES > caching efficiently > * Increase test coverage. Most of the testing in this area requires ES > integration. We should add more unit tests checking that input query plans > generate the expected ES requests. > [0] > [https://github.com/oak-indexing/jackrabbit-oak/blob/8cc86c4d142004bf2550cf2ac66400b54093209d/oak-search-elastic/src/main/java/org/apache/jackrabbit/oak/plugins/index/elasticsearch/query/ElasticsearchResultRowIterator.java] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (OAK-9111) Default Analyzer for remote index / elastic search
[
https://issues.apache.org/jira/browse/OAK-9111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185958#comment-17185958
]
Julian Reschke commented on OAK-9111:
-
fixVersion?
> Default Analyzer for remote index / elastic search
> --
>
> Key: OAK-9111
> URL: https://issues.apache.org/jira/browse/OAK-9111
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: indexing, search, search-elastic
>Reporter: Fabrizio Fortino
>Assignee: Fabrizio Fortino
>Priority: Major
>
> Need to find out what needs to be done / if this is possible / how to do it /
> what are our options.
> The current default OakAnalyzer uses the following configuration:
> {noformat}
> TokenStreamComponents(
> WordDelimiterFilter(
> LowerCaseFilter(
> StandardTokenizer(
> Reader
> )
> ),
> GENERATE_WORD_PARTS
> | STEM_ENGLISH_POSSESSIVE
> | (indexOriginalTerm ? PRESERVE_ORIGINAL : 0)
> | GENERATE_NUMBER_PARTS
> )
> ){noformat}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (OAK-9094) Error message "Could not access the Lucene index" / FileNotFoundException
[
https://issues.apache.org/jira/browse/OAK-9094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185960#comment-17185960
]
Julian Reschke commented on OAK-9094:
-
fixVersion?
> Error message "Could not access the Lucene index" / FileNotFoundException
> -
>
> Key: OAK-9094
> URL: https://issues.apache.org/jira/browse/OAK-9094
> Project: Jackrabbit Oak
> Issue Type: Improvement
>Reporter: Mohit Kataria
>Assignee: Mohit Kataria
>Priority: Minor
>
> It may happen that between retrieving files from remote and evaluating size
> of file, the remote file may get get removed leading to IO exception. Adding
> a check to handle this case.
> {noformat}
> IndexSanityChecker
> private boolean isThereASizeMismatch() throws IOException {
> for (String fileName : remote.listAll()){
> long localLength = DirectoryUtils.getFileLength(local, fileName);
> long remoteLength = remote.fileLength(fileName); // <== line 81
> //This is a weak check based on length.
> if (localLength > 0 && localLength != remoteLength){
> log.warn("[{}] Found local copy for {} in {} but size of
> local {} differs from remote {}. ",
> indexPath, fileName, local, localLength,
> remoteLength);
> return true;
> }
> if (localLength > 0) {
> localDirSize += localLength;
> localFileCount++;
> }
> remoteDirSize += remoteLength;
> remoteFileCount++;
> }
> return false;
> }
> OakDirectory:
>@Override
> public long fileLength(String name) throws IOException {
> NodeBuilder file = directoryBuilder.getChildNode(name);
> if (!file.exists()) {
> String msg = String.format("[%s] %s", indexName, name);
> throw new FileNotFoundException(msg);
> }
> OakIndexInput input = new OakIndexInput(name, file, indexName,
> blobFactory);
> try {
> return input.length();
> } finally {
> input.close();
> }
> }
> {noformat}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (OAK-9119) oak-search-elastic: support order by clause
[ https://issues.apache.org/jira/browse/OAK-9119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185957#comment-17185957 ] Julian Reschke commented on OAK-9119: - fixVersion? > oak-search-elastic: support order by clause > --- > > Key: OAK-9119 > URL: https://issues.apache.org/jira/browse/OAK-9119 > Project: Jackrabbit Oak > Issue Type: Task > Components: elastic-search, indexing, search >Reporter: Fabrizio Fortino >Assignee: Fabrizio Fortino >Priority: Major > > Support `order by` clause with the same lucene behaviour -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (OAK-9123) Error: Document contains at least one immense term
[
https://issues.apache.org/jira/browse/OAK-9123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185955#comment-17185955
]
Julian Reschke commented on OAK-9123:
-
fixVersion?
> Error: Document contains at least one immense term
> --
>
> Key: OAK-9123
> URL: https://issues.apache.org/jira/browse/OAK-9123
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: elastic-search, indexing, search
>Reporter: Fabrizio Fortino
>Assignee: Fabrizio Fortino
>Priority: Major
>
> {code:java}
> 11:35:09.400 [I/O dispatcher 1] ERROR o.a.j.o.p.i.e.i.ElasticIndexWriter -
> Bulk item with id /wikipedia/76/84/National Palace (Mexico) failed
> org.elasticsearch.ElasticsearchException: Elasticsearch exception
> [type=illegal_argument_exception, reason=Document contains at least one
> immense term in field="text.keyword" (whose UTF8 encoding is longer than the
> max length 32766), all of which were skipped. Please correct the analyzer to
> not produce such terms. The prefix of the first immense term is: '[123, 123,
> 73, 110, 102, 111, 98, 111, 120, 32, 104, 105, 115, 116, 111, 114, 105, 99,
> 32, 98, 117, 105, 108, 100, 105, 110, 103, 10, 124, 110]...', original
> message: bytes can be at most 32766 in length; got 33409]
> at
> org.elasticsearch.ElasticsearchException.innerFromXContent(ElasticsearchException.java:496)
> at
> org.elasticsearch.ElasticsearchException.fromXContent(ElasticsearchException.java:407)
> at
> org.elasticsearch.action.bulk.BulkItemResponse.fromXContent(BulkItemResponse.java:138)
> at
> org.elasticsearch.action.bulk.BulkResponse.fromXContent(BulkResponse.java:196)
> at
> org.elasticsearch.client.RestHighLevelClient.parseEntity(RestHighLevelClient.java:1888)
> at
> org.elasticsearch.client.RestHighLevelClient.lambda$performRequestAsyncAndParseEntity$10(RestHighLevelClient.java:1676)
> at
> org.elasticsearch.client.RestHighLevelClient$1.onSuccess(RestHighLevelClient.java:1758)
> at
> org.elasticsearch.client.RestClient$FailureTrackingResponseListener.onSuccess(RestClient.java:590)
> at org.elasticsearch.client.RestClient$1.completed(RestClient.java:333)
> at org.elasticsearch.client.RestClient$1.completed(RestClient.java:327)
> at org.apache.http.concurrent.BasicFuture.completed(BasicFuture.java:122)
> at
> org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.responseCompleted(DefaultClientExchangeHandlerImpl.java:181)
> at
> org.apache.http.nio.protocol.HttpAsyncRequestExecutor.processResponse(HttpAsyncRequestExecutor.java:448)
> at
> org.apache.http.nio.protocol.HttpAsyncRequestExecutor.inputReady(HttpAsyncRequestExecutor.java:338)
> at
> org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:265)
> at
> org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:81)
> at
> org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:39)
> at
> org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:114)
> at
> org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162)
> at
> org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337)
> at
> org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315)
> at
> org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276)
> at
> org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104)
> at
> org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:591)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.elasticsearch.ElasticsearchException: Elasticsearch exception
> [type=max_bytes_length_exceeded_exception, reason=bytes can be at most 32766
> in length; got 33409]
> at
> org.elasticsearch.ElasticsearchException.innerFromXContent(ElasticsearchException.java:496)
> at
> org.elasticsearch.ElasticsearchException.fromXContent(ElasticsearchException.java:407)
> at
> org.elasticsearch.ElasticsearchException.innerFromXContent(ElasticsearchException.java:437)
> ... 24 common frames omitted{code}
> This happens with huge keyword fields since Lucene doesn't allow terms with
> more than 32k bytes.
> See
> [https://discuss.elastic.co/t/error-document-contains-at-least-one-immense-term-in-field/66486]
> We have decided to always create keyword fields to remove the need to specify
> properties like ordered or facet. In this way every field can be sorted or
> used as facet.
> In this specific case the keyword field won't be needed at all but it would
> be hard to decide when include it or not. To solve this we are going to use
> `ignore_above=256` so huge keyword fields will be ignored.
[jira] [Updated] (OAK-9129) Upgrade to Elasticsearch 7.8.0
[ https://issues.apache.org/jira/browse/OAK-9129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fabrizio Fortino updated OAK-9129: -- Fix Version/s: 1.32.0 > Upgrade to Elasticsearch 7.8.0 > -- > > Key: OAK-9129 > URL: https://issues.apache.org/jira/browse/OAK-9129 > Project: Jackrabbit Oak > Issue Type: Task > Components: elastic-search, indexing, oak-search >Reporter: Fabrizio Fortino >Assignee: Fabrizio Fortino >Priority: Major > Fix For: 1.32.0 > > > > Release Notes > [https://www.elastic.co/guide/en/elasticsearch/reference/7.8/release-notes-7.8.0.html] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (OAK-9129) Upgrade to Elasticsearch 7.8.0
[ https://issues.apache.org/jira/browse/OAK-9129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185926#comment-17185926 ] Fabrizio Fortino commented on OAK-9129: --- My bad. Fixed. > Upgrade to Elasticsearch 7.8.0 > -- > > Key: OAK-9129 > URL: https://issues.apache.org/jira/browse/OAK-9129 > Project: Jackrabbit Oak > Issue Type: Task > Components: elastic-search, indexing, oak-search >Reporter: Fabrizio Fortino >Assignee: Fabrizio Fortino >Priority: Major > Fix For: 1.32.0 > > > > Release Notes > [https://www.elastic.co/guide/en/elasticsearch/reference/7.8/release-notes-7.8.0.html] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (OAK-9188) Upgrade to Elasticsearch 7.9.0
[ https://issues.apache.org/jira/browse/OAK-9188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fabrizio Fortino updated OAK-9188: -- Fix Version/s: 1.34.0 > Upgrade to Elasticsearch 7.9.0 > -- > > Key: OAK-9188 > URL: https://issues.apache.org/jira/browse/OAK-9188 > Project: Jackrabbit Oak > Issue Type: Task > Components: elastic-search, indexing, oak-search >Reporter: Fabrizio Fortino >Assignee: Fabrizio Fortino >Priority: Major > Fix For: 1.34.0 > > > > Release Notes > [https://www.elastic.co/guide/en/elasticsearch/reference/7.9/release-notes-7.9.0.html] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (OAK-9188) Upgrade to Elasticsearch 7.9.0
[ https://issues.apache.org/jira/browse/OAK-9188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fabrizio Fortino updated OAK-9188: -- Fix Version/s: (was: 1.8.24) > Upgrade to Elasticsearch 7.9.0 > -- > > Key: OAK-9188 > URL: https://issues.apache.org/jira/browse/OAK-9188 > Project: Jackrabbit Oak > Issue Type: Task > Components: elastic-search, indexing, oak-search >Reporter: Fabrizio Fortino >Assignee: Fabrizio Fortino >Priority: Major > > > Release Notes > [https://www.elastic.co/guide/en/elasticsearch/reference/7.9/release-notes-7.9.0.html] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (OAK-9129) Upgrade to Elasticsearch 7.8.0
[ https://issues.apache.org/jira/browse/OAK-9129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fabrizio Fortino updated OAK-9129: -- Fix Version/s: (was: 1.8.23) > Upgrade to Elasticsearch 7.8.0 > -- > > Key: OAK-9129 > URL: https://issues.apache.org/jira/browse/OAK-9129 > Project: Jackrabbit Oak > Issue Type: Task > Components: elastic-search, indexing, oak-search >Reporter: Fabrizio Fortino >Assignee: Fabrizio Fortino >Priority: Major > > > Release Notes > [https://www.elastic.co/guide/en/elasticsearch/reference/7.8/release-notes-7.8.0.html] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (OAK-9188) Upgrade to Elasticsearch 7.9.0
[ https://issues.apache.org/jira/browse/OAK-9188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fabrizio Fortino resolved OAK-9188. --- Fix Version/s: 1.8.24 Resolution: Fixed Fixed with commit 1881247 > Upgrade to Elasticsearch 7.9.0 > -- > > Key: OAK-9188 > URL: https://issues.apache.org/jira/browse/OAK-9188 > Project: Jackrabbit Oak > Issue Type: Task > Components: elastic-search, indexing, oak-search >Reporter: Fabrizio Fortino >Assignee: Fabrizio Fortino >Priority: Major > Fix For: 1.8.24 > > > > Release Notes > [https://www.elastic.co/guide/en/elasticsearch/reference/7.9/release-notes-7.9.0.html] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (OAK-9129) Upgrade to Elasticsearch 7.8.0
[ https://issues.apache.org/jira/browse/OAK-9129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185912#comment-17185912 ] Julian Reschke commented on OAK-9129: - But the change is in trunk, how can it affect 1.8.*??? > Upgrade to Elasticsearch 7.8.0 > -- > > Key: OAK-9129 > URL: https://issues.apache.org/jira/browse/OAK-9129 > Project: Jackrabbit Oak > Issue Type: Task > Components: elastic-search, indexing, oak-search >Reporter: Fabrizio Fortino >Assignee: Fabrizio Fortino >Priority: Major > Fix For: 1.8.23 > > > > Release Notes > [https://www.elastic.co/guide/en/elasticsearch/reference/7.8/release-notes-7.8.0.html] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (OAK-9129) Upgrade to Elasticsearch 7.8.0
[ https://issues.apache.org/jira/browse/OAK-9129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185910#comment-17185910 ] Fabrizio Fortino commented on OAK-9129: --- Ok, thanks for pointing it out [~reschke] > Upgrade to Elasticsearch 7.8.0 > -- > > Key: OAK-9129 > URL: https://issues.apache.org/jira/browse/OAK-9129 > Project: Jackrabbit Oak > Issue Type: Task > Components: elastic-search, indexing, oak-search >Reporter: Fabrizio Fortino >Assignee: Fabrizio Fortino >Priority: Major > Fix For: 1.8.23 > > > > Release Notes > [https://www.elastic.co/guide/en/elasticsearch/reference/7.8/release-notes-7.8.0.html] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (OAK-9129) Upgrade to Elasticsearch 7.8.0
[ https://issues.apache.org/jira/browse/OAK-9129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fabrizio Fortino updated OAK-9129: -- Fix Version/s: 1.8.23 > Upgrade to Elasticsearch 7.8.0 > -- > > Key: OAK-9129 > URL: https://issues.apache.org/jira/browse/OAK-9129 > Project: Jackrabbit Oak > Issue Type: Task > Components: elastic-search, indexing, oak-search >Reporter: Fabrizio Fortino >Assignee: Fabrizio Fortino >Priority: Major > Fix For: 1.8.23 > > > > Release Notes > [https://www.elastic.co/guide/en/elasticsearch/reference/7.8/release-notes-7.8.0.html] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (OAK-9129) Upgrade to Elasticsearch 7.8.0
[ https://issues.apache.org/jira/browse/OAK-9129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185885#comment-17185885 ] Julian Reschke commented on OAK-9129: - [~fortino] - you need to set "fixVersion", otherwise the change will not appear in release notes, nor will the ticket change to "closed" after the release. > Upgrade to Elasticsearch 7.8.0 > -- > > Key: OAK-9129 > URL: https://issues.apache.org/jira/browse/OAK-9129 > Project: Jackrabbit Oak > Issue Type: Task > Components: elastic-search, indexing, oak-search >Reporter: Fabrizio Fortino >Assignee: Fabrizio Fortino >Priority: Major > > > Release Notes > [https://www.elastic.co/guide/en/elasticsearch/reference/7.8/release-notes-7.8.0.html] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (OAK-9188) Upgrade to Elasticsearch 7.9.0
[ https://issues.apache.org/jira/browse/OAK-9188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fabrizio Fortino updated OAK-9188: -- Description: Release Notes [https://www.elastic.co/guide/en/elasticsearch/reference/7.9/release-notes-7.9.0.html] was: Release Notes [https://www.elastic.co/guide/en/elasticsearch/reference/7.8/release-notes-7.8.0.html] > Upgrade to Elasticsearch 7.9.0 > -- > > Key: OAK-9188 > URL: https://issues.apache.org/jira/browse/OAK-9188 > Project: Jackrabbit Oak > Issue Type: Task > Components: elastic-search, indexing, oak-search >Reporter: Fabrizio Fortino >Assignee: Fabrizio Fortino >Priority: Major > > > Release Notes > [https://www.elastic.co/guide/en/elasticsearch/reference/7.9/release-notes-7.9.0.html] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (OAK-9188) Upgrade to Elasticsearch 7.9.0
Fabrizio Fortino created OAK-9188: - Summary: Upgrade to Elasticsearch 7.9.0 Key: OAK-9188 URL: https://issues.apache.org/jira/browse/OAK-9188 Project: Jackrabbit Oak Issue Type: Task Components: elastic-search, indexing, oak-search Reporter: Fabrizio Fortino Assignee: Fabrizio Fortino Release Notes [https://www.elastic.co/guide/en/elasticsearch/reference/7.8/release-notes-7.8.0.html] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (OAK-9125) Oak-run indexing with MongoDB logs errors in a background thread
[
https://issues.apache.org/jira/browse/OAK-9125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nitin Gupta closed OAK-9125.
> Oak-run indexing with MongoDB logs errors in a background thread
>
>
> Key: OAK-9125
> URL: https://issues.apache.org/jira/browse/OAK-9125
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: indexing, oak-run
>Affects Versions: 1.8.22
>Reporter: Thomas Mueller
>Assignee: Marcel Reutegger
>Priority: Major
> Fix For: 1.8.23
>
> Attachments: OAK-9125.patch
>
>
> The following error message can be logged in a background thread:
> {noformat}
> Caused by: com.mongodb.MongoQueryException:
> Query failed with error code 13 and error message
> 'not authorized on ... to execute command
> { find: "nodes", filter: { _id: "0:/" }, limit: 1, singleBatch: true }'
> on server ... at
> com.mongodb.operation.FindOperation$1.call(FindOperation.java:722)
> {noformat}
> Those error messages don't have a negative impact on the index command.
> However, they are annoying.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (OAK-9184) Very slow, potential endless loop in LucenePropertyIndex.loadDocs()
[
https://issues.apache.org/jira/browse/OAK-9184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185676#comment-17185676
]
Thomas Mueller commented on OAK-9184:
-
* Logs a warning after 30 seconds.
* If repeated more than once, and after 3 minutes, an error is logged (with
stack trace) and the loop stops.
* Configurable over system properties.
* No test case, but I tested it manually using a breakpoint.
> Very slow, potential endless loop in LucenePropertyIndex.loadDocs()
> ---
>
> Key: OAK-9184
> URL: https://issues.apache.org/jira/browse/OAK-9184
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: lucene
>Affects Versions: 1.22.4, 1.32.0, 1.8.23
>Reporter: Thomas Mueller
>Assignee: Thomas Mueller
>Priority: Major
>
> We found one case of a very slow, possibly endless loop in this method.
> {noformat}
> java.lang.Thread.State: RUNNABLE at
> org.apache.lucene.search.TopScoreDocCollector$InOrderPagingScoreDocCollector.collect(TopScoreDocCollector.java:85)
> at org.apache.lucene.search.Scorer.score(Scorer.java:65) at
> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:621) at
> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:491) at
> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:448) at
> org.apache.lucene.search.IndexSearcher.searchAfter(IndexSearcher.java:243) at
> org.apache.jackrabbit.oak.plugins.index.lucene.LucenePropertyIndex$1.loadDocs(LucenePropertyIndex.java:434)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.LucenePropertyIndex$1.computeNext(LucenePropertyIndex.java:353)
> {noformat}
> Background: Many threads are waiting to lock here:
> {noformat}
> - waiting to lock <0x7fbd804a9448> (a
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexTracker)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexTracker.acquireIndexNode(IndexTracker.java:202)
> public IndexNode acquireIndexNode(String path) {
> IndexNodeManager index = indices.get(path);
> IndexNode indexNode = index != null ? index.acquire() : null;
> if (indexNode != null) {
> return indexNode;
> } else {
> return findIndexNode(path); <<= synchronized method
> }
> }
> {noformat}
> And the thread that is holding that (synchronized) lock on IndexTracker is
> also waiting to get a write lock here:
> {noformat}
> "oak-lucene-992" #21559 daemon prio=1 os_prio=0 tid=0x7ed0280ad000
> nid=0x1be8 waiting on condition [0x7fb96c352000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x7fbedad77210> (a
> java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
> at
> java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:943)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexNodeManager.close(IndexNodeManager.java:174)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexTracker.close(IndexTracker.java:104)
> - locked <0x7fbd804a9448> (a
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexTracker)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexTracker.update(IndexTracker.java:114)
> - locked <0x7fbd804a9448> (a
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexTracker)
> void close() throws IOException {
> lock.writeLock().lock(); <<== waiting here
> try {
> checkState(!closed);
> closed = true;
> } finally {
> lock.writeLock().unlock();
> }
> {noformat}
> It looks like a read lock is not released. The places where a read lock is
> acquired:
> {noformat}
> IndexNode acquire() {
> lock.readLock().lock();
> (released if the method returns null or throws an exception)
> private void release() {
> lock.readLock().unlock();
> }
> {noformat}
> acquire() is called in acquireIndexNode:
> {noformat}
> public IndexNode acquireIndexNode(String path) {
> IndexNodeManager index = indices.get(path);
> IndexNode indexNode = index != null ? index.acquire() : null; <<==
> here
> if (indexNode != null) {
> return indexNode;
> } else {
> return
[jira] [Commented] (OAK-9184) Very slow, potential endless loop in LucenePropertyIndex.loadDocs()
[
https://issues.apache.org/jira/browse/OAK-9184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185673#comment-17185673
]
Thomas Mueller commented on OAK-9184:
-
[https://github.com/oak-indexing/jackrabbit-oak/pull/274]
> Very slow, potential endless loop in LucenePropertyIndex.loadDocs()
> ---
>
> Key: OAK-9184
> URL: https://issues.apache.org/jira/browse/OAK-9184
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: lucene
>Affects Versions: 1.22.4, 1.32.0, 1.8.23
>Reporter: Thomas Mueller
>Assignee: Thomas Mueller
>Priority: Major
>
> We found one case of a very slow, possibly endless loop in this method.
> {noformat}
> java.lang.Thread.State: RUNNABLE at
> org.apache.lucene.search.TopScoreDocCollector$InOrderPagingScoreDocCollector.collect(TopScoreDocCollector.java:85)
> at org.apache.lucene.search.Scorer.score(Scorer.java:65) at
> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:621) at
> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:491) at
> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:448) at
> org.apache.lucene.search.IndexSearcher.searchAfter(IndexSearcher.java:243) at
> org.apache.jackrabbit.oak.plugins.index.lucene.LucenePropertyIndex$1.loadDocs(LucenePropertyIndex.java:434)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.LucenePropertyIndex$1.computeNext(LucenePropertyIndex.java:353)
> {noformat}
> Background: Many threads are waiting to lock here:
> {noformat}
> - waiting to lock <0x7fbd804a9448> (a
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexTracker)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexTracker.acquireIndexNode(IndexTracker.java:202)
> public IndexNode acquireIndexNode(String path) {
> IndexNodeManager index = indices.get(path);
> IndexNode indexNode = index != null ? index.acquire() : null;
> if (indexNode != null) {
> return indexNode;
> } else {
> return findIndexNode(path); <<= synchronized method
> }
> }
> {noformat}
> And the thread that is holding that (synchronized) lock on IndexTracker is
> also waiting to get a write lock here:
> {noformat}
> "oak-lucene-992" #21559 daemon prio=1 os_prio=0 tid=0x7ed0280ad000
> nid=0x1be8 waiting on condition [0x7fb96c352000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x7fbedad77210> (a
> java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
> at
> java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:943)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexNodeManager.close(IndexNodeManager.java:174)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexTracker.close(IndexTracker.java:104)
> - locked <0x7fbd804a9448> (a
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexTracker)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexTracker.update(IndexTracker.java:114)
> - locked <0x7fbd804a9448> (a
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexTracker)
> void close() throws IOException {
> lock.writeLock().lock(); <<== waiting here
> try {
> checkState(!closed);
> closed = true;
> } finally {
> lock.writeLock().unlock();
> }
> {noformat}
> It looks like a read lock is not released. The places where a read lock is
> acquired:
> {noformat}
> IndexNode acquire() {
> lock.readLock().lock();
> (released if the method returns null or throws an exception)
> private void release() {
> lock.readLock().unlock();
> }
> {noformat}
> acquire() is called in acquireIndexNode:
> {noformat}
> public IndexNode acquireIndexNode(String path) {
> IndexNodeManager index = indices.get(path);
> IndexNode indexNode = index != null ? index.acquire() : null; <<==
> here
> if (indexNode != null) {
> return indexNode;
> } else {
> return findIndexNode(path);
> }
> }
> private synchronized IndexNode findIndexNode(String path) {
> // Retry the lookup from acquireIndexNode now that we're
> //
