[jira] [Commented] (SOLR-9269) Ability to create/delete/list snapshots for a solr core
[
https://issues.apache.org/jira/browse/SOLR-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388937#comment-15388937
]
ASF GitHub Bot commented on SOLR-9269:
--
Github user hgadre commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/52#discussion_r71828221
--- Diff:
solr/core/src/java/org/apache/solr/core/snapshots/SolrSnapshotManager.java ---
@@ -0,0 +1,132 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.core.snapshots;
+
+import java.io.IOException;
+import java.lang.invoke.MethodHandles;
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.function.Function;
+import java.util.stream.Collectors;
+
+import org.apache.lucene.index.DirectoryReader;
+import org.apache.lucene.index.IndexCommit;
+import org.apache.lucene.store.Directory;
+import
org.apache.solr.core.snapshots.SolrSnapshotMetaDataManager.SnapshotMetaData;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This class provides functionality required to handle the data files
corresponding to Solr snapshots.
+ */
+public class SolrSnapshotManager {
+ private static final Logger log =
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
+
+ /**
+ * This method deletes index files of the {@linkplain IndexCommit} for
the specified generation number.
+ *
+ * @param dir The index directory storing the snapshot.
+ * @param gen The generation number for the {@linkplain IndexCommit}
+ * @throws IOException in case of I/O errors.
+ */
+ public static synchronized void deleteIndexFiles ( Directory dir,
Collection snapshots, long gen ) throws IOException {
+List commits = DirectoryReader.listCommits(dir);
+Map refCounts = buildRefCounts(snapshots,commits);
+for (IndexCommit ic : commits) {
+ if (ic.getGeneration() == gen) {
+deleteIndexFiles(dir,refCounts, ic);
+break;
+ }
+}
+ }
+
+ /**
+ * This method deletes all files not corresponding to a configured
snapshot in the specified index directory.
+ *
+ * @param dir The index directory to search for.
+ * @throws IOException in case of I/O errors.
+ */
+ public static synchronized void deleteNonSnapshotIndexFiles (Directory
dir, Collection snapshots) throws IOException {
+List commits = DirectoryReader.listCommits(dir);
+Map refCounts = buildRefCounts(snapshots, commits);
+Set snapshotGenNumbers = snapshots.stream()
+
.map(SnapshotMetaData::getGenerationNumber)
+.collect(Collectors.toSet());
+for (IndexCommit ic : commits) {
+ if (!snapshotGenNumbers.contains(ic.getGeneration())) {
+deleteIndexFiles(dir,refCounts, ic);
+ }
+}
+ }
+
+ /**
+ * This method computes reference count for the index files by taking
into consideration
+ * (a) configured snapshots and (b) files sharing between two or more
{@linkplain IndexCommit} instances.
+ *
+ * @param snapshots A collection of user configured snapshots
+ * @param commits A list of {@linkplain IndexCommit} instances
+ * @return A map containing reference count for each index file referred
in one of the {@linkplain IndexCommit} instances.
+ * @throws IOException in case of I/O error.
+ */
+ public static Map buildRefCounts
(Collection snapshots, List commits) throws
IOException {
--- End diff --
Actually we need to know the actual reference count for the unit test. But
now I changed this method to be package protected. Hence we should be OK with
this.
> Ability to create/delete/list snapshots for a solr core
>
[GitHub] lucene-solr pull request #52: [SOLR-9269] Ability to create/delete/list snap...
Github user hgadre commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/52#discussion_r71828221
--- Diff:
solr/core/src/java/org/apache/solr/core/snapshots/SolrSnapshotManager.java ---
@@ -0,0 +1,132 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.core.snapshots;
+
+import java.io.IOException;
+import java.lang.invoke.MethodHandles;
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.function.Function;
+import java.util.stream.Collectors;
+
+import org.apache.lucene.index.DirectoryReader;
+import org.apache.lucene.index.IndexCommit;
+import org.apache.lucene.store.Directory;
+import
org.apache.solr.core.snapshots.SolrSnapshotMetaDataManager.SnapshotMetaData;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This class provides functionality required to handle the data files
corresponding to Solr snapshots.
+ */
+public class SolrSnapshotManager {
+ private static final Logger log =
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
+
+ /**
+ * This method deletes index files of the {@linkplain IndexCommit} for
the specified generation number.
+ *
+ * @param dir The index directory storing the snapshot.
+ * @param gen The generation number for the {@linkplain IndexCommit}
+ * @throws IOException in case of I/O errors.
+ */
+ public static synchronized void deleteIndexFiles ( Directory dir,
Collection snapshots, long gen ) throws IOException {
+List commits = DirectoryReader.listCommits(dir);
+Map refCounts = buildRefCounts(snapshots,commits);
+for (IndexCommit ic : commits) {
+ if (ic.getGeneration() == gen) {
+deleteIndexFiles(dir,refCounts, ic);
+break;
+ }
+}
+ }
+
+ /**
+ * This method deletes all files not corresponding to a configured
snapshot in the specified index directory.
+ *
+ * @param dir The index directory to search for.
+ * @throws IOException in case of I/O errors.
+ */
+ public static synchronized void deleteNonSnapshotIndexFiles (Directory
dir, Collection snapshots) throws IOException {
+List commits = DirectoryReader.listCommits(dir);
+Map refCounts = buildRefCounts(snapshots, commits);
+Set snapshotGenNumbers = snapshots.stream()
+
.map(SnapshotMetaData::getGenerationNumber)
+.collect(Collectors.toSet());
+for (IndexCommit ic : commits) {
+ if (!snapshotGenNumbers.contains(ic.getGeneration())) {
+deleteIndexFiles(dir,refCounts, ic);
+ }
+}
+ }
+
+ /**
+ * This method computes reference count for the index files by taking
into consideration
+ * (a) configured snapshots and (b) files sharing between two or more
{@linkplain IndexCommit} instances.
+ *
+ * @param snapshots A collection of user configured snapshots
+ * @param commits A list of {@linkplain IndexCommit} instances
+ * @return A map containing reference count for each index file referred
in one of the {@linkplain IndexCommit} instances.
+ * @throws IOException in case of I/O error.
+ */
+ public static Map buildRefCounts
(Collection snapshots, List commits) throws
IOException {
--- End diff --
Actually we need to know the actual reference count for the unit test. But
now I changed this method to be package protected. Hence we should be OK with
this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA
[jira] [Commented] (SOLR-9269) Ability to create/delete/list snapshots for a solr core
[
https://issues.apache.org/jira/browse/SOLR-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388936#comment-15388936
]
ASF GitHub Bot commented on SOLR-9269:
--
Github user hgadre commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/52#discussion_r71828065
--- Diff:
solr/core/src/java/org/apache/solr/core/snapshots/SolrSnapshotManager.java ---
@@ -0,0 +1,132 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.core.snapshots;
+
+import java.io.IOException;
+import java.lang.invoke.MethodHandles;
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.function.Function;
+import java.util.stream.Collectors;
+
+import org.apache.lucene.index.DirectoryReader;
+import org.apache.lucene.index.IndexCommit;
+import org.apache.lucene.store.Directory;
+import
org.apache.solr.core.snapshots.SolrSnapshotMetaDataManager.SnapshotMetaData;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This class provides functionality required to handle the data files
corresponding to Solr snapshots.
+ */
+public class SolrSnapshotManager {
+ private static final Logger log =
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
+
+ /**
+ * This method deletes index files of the {@linkplain IndexCommit} for
the specified generation number.
+ *
+ * @param dir The index directory storing the snapshot.
+ * @param gen The generation number for the {@linkplain IndexCommit}
+ * @throws IOException in case of I/O errors.
+ */
+ public static synchronized void deleteIndexFiles ( Directory dir,
Collection snapshots, long gen ) throws IOException {
+List commits = DirectoryReader.listCommits(dir);
+Map refCounts = buildRefCounts(snapshots,commits);
+for (IndexCommit ic : commits) {
+ if (ic.getGeneration() == gen) {
+deleteIndexFiles(dir,refCounts, ic);
+break;
+ }
+}
+ }
+
+ /**
+ * This method deletes all files not corresponding to a configured
snapshot in the specified index directory.
+ *
+ * @param dir The index directory to search for.
+ * @throws IOException in case of I/O errors.
+ */
+ public static synchronized void deleteNonSnapshotIndexFiles (Directory
dir, Collection snapshots) throws IOException {
+List commits = DirectoryReader.listCommits(dir);
+Map refCounts = buildRefCounts(snapshots, commits);
+Set snapshotGenNumbers = snapshots.stream()
+
.map(SnapshotMetaData::getGenerationNumber)
+.collect(Collectors.toSet());
+for (IndexCommit ic : commits) {
+ if (!snapshotGenNumbers.contains(ic.getGeneration())) {
+deleteIndexFiles(dir,refCounts, ic);
+ }
+}
+ }
+
+ /**
+ * This method computes reference count for the index files by taking
into consideration
+ * (a) configured snapshots and (b) files sharing between two or more
{@linkplain IndexCommit} instances.
+ *
+ * @param snapshots A collection of user configured snapshots
+ * @param commits A list of {@linkplain IndexCommit} instances
+ * @return A map containing reference count for each index file referred
in one of the {@linkplain IndexCommit} instances.
+ * @throws IOException in case of I/O error.
+ */
+ public static Map buildRefCounts
(Collection snapshots, List commits) throws
IOException {
+Map result = new HashMap<>();
+Map commitsByGen = commits.stream().collect(
+Collectors.toMap(IndexCommit::getGeneration, Function.identity()));
+
+for(SnapshotMetaData md :
[GitHub] lucene-solr pull request #52: [SOLR-9269] Ability to create/delete/list snap...
Github user hgadre commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/52#discussion_r71828065
--- Diff:
solr/core/src/java/org/apache/solr/core/snapshots/SolrSnapshotManager.java ---
@@ -0,0 +1,132 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.core.snapshots;
+
+import java.io.IOException;
+import java.lang.invoke.MethodHandles;
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.function.Function;
+import java.util.stream.Collectors;
+
+import org.apache.lucene.index.DirectoryReader;
+import org.apache.lucene.index.IndexCommit;
+import org.apache.lucene.store.Directory;
+import
org.apache.solr.core.snapshots.SolrSnapshotMetaDataManager.SnapshotMetaData;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This class provides functionality required to handle the data files
corresponding to Solr snapshots.
+ */
+public class SolrSnapshotManager {
+ private static final Logger log =
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
+
+ /**
+ * This method deletes index files of the {@linkplain IndexCommit} for
the specified generation number.
+ *
+ * @param dir The index directory storing the snapshot.
+ * @param gen The generation number for the {@linkplain IndexCommit}
+ * @throws IOException in case of I/O errors.
+ */
+ public static synchronized void deleteIndexFiles ( Directory dir,
Collection snapshots, long gen ) throws IOException {
+List commits = DirectoryReader.listCommits(dir);
+Map refCounts = buildRefCounts(snapshots,commits);
+for (IndexCommit ic : commits) {
+ if (ic.getGeneration() == gen) {
+deleteIndexFiles(dir,refCounts, ic);
+break;
+ }
+}
+ }
+
+ /**
+ * This method deletes all files not corresponding to a configured
snapshot in the specified index directory.
+ *
+ * @param dir The index directory to search for.
+ * @throws IOException in case of I/O errors.
+ */
+ public static synchronized void deleteNonSnapshotIndexFiles (Directory
dir, Collection snapshots) throws IOException {
+List commits = DirectoryReader.listCommits(dir);
+Map refCounts = buildRefCounts(snapshots, commits);
+Set snapshotGenNumbers = snapshots.stream()
+
.map(SnapshotMetaData::getGenerationNumber)
+.collect(Collectors.toSet());
+for (IndexCommit ic : commits) {
+ if (!snapshotGenNumbers.contains(ic.getGeneration())) {
+deleteIndexFiles(dir,refCounts, ic);
+ }
+}
+ }
+
+ /**
+ * This method computes reference count for the index files by taking
into consideration
+ * (a) configured snapshots and (b) files sharing between two or more
{@linkplain IndexCommit} instances.
+ *
+ * @param snapshots A collection of user configured snapshots
+ * @param commits A list of {@linkplain IndexCommit} instances
+ * @return A map containing reference count for each index file referred
in one of the {@linkplain IndexCommit} instances.
+ * @throws IOException in case of I/O error.
+ */
+ public static Map buildRefCounts
(Collection snapshots, List commits) throws
IOException {
+Map result = new HashMap<>();
+Map commitsByGen = commits.stream().collect(
+Collectors.toMap(IndexCommit::getGeneration, Function.identity()));
+
+for(SnapshotMetaData md : snapshots) {
+ IndexCommit ic = commitsByGen.get(md.getGenerationNumber());
+ if (ic != null) {
+Collection fileNames = ic.getFileNames();
+for(String fileName : fileNames) {
+ int
[jira] [Commented] (SOLR-9269) Ability to create/delete/list snapshots for a solr core
[
https://issues.apache.org/jira/browse/SOLR-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388925#comment-15388925
]
ASF GitHub Bot commented on SOLR-9269:
--
Github user hgadre commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/52#discussion_r71827952
--- Diff: solr/core/src/java/org/apache/solr/core/SolrCore.java ---
@@ -414,7 +416,19 @@ private IndexDeletionPolicyWrapper
initDeletionPolicy(IndexDeletionPolicyWrapper
} else {
delPolicy = new SolrDeletionPolicy();
}
-return new IndexDeletionPolicyWrapper(delPolicy);
+
+return new IndexDeletionPolicyWrapper(delPolicy, snapshotMgr);
+ }
+
+ private SolrSnapshotMetaDataManager initSnapshotMetaDataManager() {
--- End diff --
I don't think directory creation should have any significant performance
impact. Also lazy creation just adds a bit of complexity in the code which I
wanted to avoid. But I am open to this idea if you insist.
> Ability to create/delete/list snapshots for a solr core
> ---
>
> Key: SOLR-9269
> URL: https://issues.apache.org/jira/browse/SOLR-9269
> Project: Solr
> Issue Type: Sub-task
> Components: SolrCloud
>Reporter: Hrishikesh Gadre
>Assignee: David Smiley
> Attachments: SOLR-9269.patch
>
>
> Support snapshot create/delete/list functionality @ the Solr core level.
> Please refer to parent JIRA for more details.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] lucene-solr pull request #52: [SOLR-9269] Ability to create/delete/list snap...
Github user hgadre commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/52#discussion_r71827952
--- Diff: solr/core/src/java/org/apache/solr/core/SolrCore.java ---
@@ -414,7 +416,19 @@ private IndexDeletionPolicyWrapper
initDeletionPolicy(IndexDeletionPolicyWrapper
} else {
delPolicy = new SolrDeletionPolicy();
}
-return new IndexDeletionPolicyWrapper(delPolicy);
+
+return new IndexDeletionPolicyWrapper(delPolicy, snapshotMgr);
+ }
+
+ private SolrSnapshotMetaDataManager initSnapshotMetaDataManager() {
--- End diff --
I don't think directory creation should have any significant performance
impact. Also lazy creation just adds a bit of complexity in the code which I
wanted to avoid. But I am open to this idea if you insist.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-3554) Skins/Themes for Solr UI
[ https://issues.apache.org/jira/browse/SOLR-3554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388893#comment-15388893 ] Alexandre Rafalovitch commented on SOLR-3554: - Can we close this? Admin UI is not really for end-users, the focus is on functional rather than eye-candy. > Skins/Themes for Solr UI > > > Key: SOLR-3554 > URL: https://issues.apache.org/jira/browse/SOLR-3554 > Project: Solr > Issue Type: Improvement > Components: web gui >Reporter: Lance Norskog >Priority: Minor > > Request the ability to create 'skins' or 'themes' for the UI. > Some people love designing themes, which will help recruit UI developers. > Also, no color set can please everyone. And, a black theme makes for > more readable screenshots in bug reports. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-3232) Admin UI: query form should have a menu to pick a request handler
[ https://issues.apache.org/jira/browse/SOLR-3232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388890#comment-15388890 ] Alexandre Rafalovitch commented on SOLR-3232: - Is this still an issue to keep around? It is an interesting discussion but all underlying API changed, as well as the actual admin UI. Interestingly, I think this is still a hard problem as we now have even more request handlers that do not make sense in the query screen. > Admin UI: query form should have a menu to pick a request handler > - > > Key: SOLR-3232 > URL: https://issues.apache.org/jira/browse/SOLR-3232 > Project: Solr > Issue Type: Improvement > Components: web gui >Reporter: David Smiley > Attachments: SOLR-3232.patch > > > The query form in the admin UI could use an improvement regarding how the > request handler is chosen; presently all there is is a text input for 'qt'. > The first choice to make in the form above the query should really be the > request handler since it actually handles the request before any other > parameters do anything. It'd be great if it was a dynamically driven menu > defaulting to "/select". Similar to how the DIH page finds DIH request > handlers, this page could find the request handlers with a class of > "SearchHandler". Their names would be added to a list, and if the name > didn't start with a '/' then it would be prefixed with '/select?qt='. > I did something similar (without the menu) to the old 3x UI in a patch to > SOLR-3161 which will hopefully get committed. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4157) Add more conventional search functionality to the Admin UI
[ https://issues.apache.org/jira/browse/SOLR-4157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1533#comment-1533 ] Alexandre Rafalovitch commented on SOLR-4157: - Can we close this? The /browse interface has been around for quite a while now :-) > Add more conventional search functionality to the Admin UI > -- > > Key: SOLR-4157 > URL: https://issues.apache.org/jira/browse/SOLR-4157 > Project: Solr > Issue Type: Improvement > Components: web gui >Reporter: Upayavira >Priority: Minor > Attachments: SOLR-4157.patch > > > The admin UI has a 'query' pane which allows searching the index. However, > this is currently an 'expert' level feature, as you must specify exact > request parameters and interpret output XML or JSON. > I suggest we add simple versions of each. A simple query pane would give a > more conventional search interface for running queries. A simple results pane > would give HTML formatted results with features to nicely display > hightlighting, explains, etc. > To give an idea of what this might look like, I've attached a rudimentary > patch that gives an HTML option for wt which formats the query results as > (somewhat minimal) HTML. > The challenge will be in producing a search interface that is schema > agnostic, as to be really useful, it should work with any index, and not just > with the fields in the default schema (perhaps Erik Hatcher is right, this > should be backed by the velocityResponseWriter). > Thoughts welcome. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-3986) index version and generation not changed in admin UI after delete by query on master
[ https://issues.apache.org/jira/browse/SOLR-3986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388878#comment-15388878 ] Alexandre Rafalovitch commented on SOLR-3986: - can this be closed? Old non-reproduced bug report against old UI in ancient version. > index version and generation not changed in admin UI after delete by query on > master > > > Key: SOLR-3986 > URL: https://issues.apache.org/jira/browse/SOLR-3986 > Project: Solr > Issue Type: Bug > Components: web gui >Affects Versions: 4.0 >Reporter: Bill Au >Priority: Minor > > Here are the steps to reproduce this: > - follow steps in Solr 4.0 tutorial to set up a master and a slave to use > Java/HTTP replication > - index example documents on master: > java -jar post.jar *.xml > - make a note of the index version and generation the on both the replication > section of the summary screen of core collection1 and the replication screen > on both the master and slave > - run a delete by query on the master > java -Ddata=args -jar post.jar "name:DDR" > - on master reload the summary screen for core collection1. The Num Docs > field decreased but the index version and generation are unchanged in the > replication section. The index version and generation are also unchanged in > the replication screen. > - on the slave, wait for replication to kick in or trigger it manually. On > the summary screen for core collection1, the Num DOcs field decreased to > match what's on the master. The index version and generation of the master > remain unchanged but the index version and generation of the slave both > changed. The same goes for the index version and generation of the master > and slave on the replication screen. > The replication handler on the master does report changed index version and > generation: > localhost:8983/solr/collection1/replication?command=indexversion > It is only the admin UI that reporting the older index version and generation > on both the core summary screen and replication screen. > This only happens with delete by query. There is no problem with delete with > id or add. > Both the index version and generation do get updated on subsequent delete by > query but both remain one cycle behind on the master. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4391) Dataimport UI defaults should reflect the documented states
[ https://issues.apache.org/jira/browse/SOLR-4391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388863#comment-15388863 ] Alexandre Rafalovitch commented on SOLR-4391: - Has this been checked against new Admin UI? If not, perhaps close it and open with more specific issue when it comes up. > Dataimport UI defaults should reflect the documented states > --- > > Key: SOLR-4391 > URL: https://issues.apache.org/jira/browse/SOLR-4391 > Project: Solr > Issue Type: Improvement > Components: web gui >Affects Versions: 4.0 >Reporter: Ota Mares >Priority: Minor > > The [wiki|http://wiki.apache.org/solr/DataImportHandler#Commands] states that > some dataimport commands are enabled by default. These states should be > reflected in the web ui because currently it leads to confusions if the > commands are enabled or not. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2086) analysis.jsp should honor maxFieldLength setting
[ https://issues.apache.org/jira/browse/SOLR-2086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388862#comment-15388862 ] Alexandre Rafalovitch commented on SOLR-2086: - Can be closed. No JSP anywhere near old or new UI. > analysis.jsp should honor maxFieldLength setting > > > Key: SOLR-2086 > URL: https://issues.apache.org/jira/browse/SOLR-2086 > Project: Solr > Issue Type: Improvement > Components: web gui >Affects Versions: 1.4.1 >Reporter: Eric Pugh > Attachments: SOLR-2086.patch > > > The analysis.jsp ignores the maxFieldLength setting when analyzing. I passed > in a block of text that was 102524 tokens, and it analyzed all of them, even > though maxFieldLength was 1. The difference in results is pretty drastic > between analysis.jsp and adding a document directly. > Also, the GUI pretty much melts down with lots and lots of tokens as well, so > maxFieldLength helps here as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4460) When you have multiple collections, the cloud radial graph view seems to place them right on top of each other.
[ https://issues.apache.org/jira/browse/SOLR-4460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388860#comment-15388860 ] Alexandre Rafalovitch commented on SOLR-4460: - Is this still relevant to new UI? > When you have multiple collections, the cloud radial graph view seems to > place them right on top of each other. > --- > > Key: SOLR-4460 > URL: https://issues.apache.org/jira/browse/SOLR-4460 > Project: Solr > Issue Type: Bug > Components: web gui >Reporter: Mark Miller >Priority: Minor > Attachments: cloud-radial-onecollection.png, cloud.jpg > > -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4963) Schema Browser does not report stats properly on TrieDateField
[ https://issues.apache.org/jira/browse/SOLR-4963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388856#comment-15388856 ] Alexandre Rafalovitch commented on SOLR-4963: - Can this be closed? Not a bug and against old UI. Specific improvements can go against new UI in a new JIRA. > Schema Browser does not report stats properly on TrieDateField > -- > > Key: SOLR-4963 > URL: https://issues.apache.org/jira/browse/SOLR-4963 > Project: Solr > Issue Type: Bug > Components: web gui >Affects Versions: 4.3 > Environment: Customized single-core Solr from example running in > default Jetty server. > Apple Inc. Java HotSpot(TM) 64-Bit Server VM (1.6.0_51 20.51-b01-456) >Reporter: Thomas Murphy >Priority: Minor > Attachments: luke-date.json, solr-admin-query-facet-date.png, > solr-admin-schema-browser-date.png > > > Loading documents into a schema with a TrieDateField and viewing that field > in the Schema Browser has distinctly wrong information displayed. Encountered > using psudo-randomly generated data. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Assigned] (SOLR-1163) Solr Explorer - A generic GWT client for Solr
[
https://issues.apache.org/jira/browse/SOLR-1163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexandre Rafalovitch reassigned SOLR-1163:
---
Assignee: Alexandre Rafalovitch
> Solr Explorer - A generic GWT client for Solr
> -
>
> Key: SOLR-1163
> URL: https://issues.apache.org/jira/browse/SOLR-1163
> Project: Solr
> Issue Type: New Feature
> Components: web gui
>Affects Versions: 1.3
>Reporter: Uri Boness
>Assignee: Alexandre Rafalovitch
> Attachments: SOLR-1163.zip, SOLR-1163.zip, graphics.zip,
> solr-explorer.patch, solr-explorer.patch
>
>
> The attached patch is a GWT generic client for solr. It is currently
> standalone, meaning that once built, one can open the generated HTML file in
> a browser and communicate with any deployed solr. It is configured with it's
> own configuration file, where one can configure the solr instance/core to
> connect to. Since it's currently standalone and completely client side based,
> it uses JSON with padding (cross-side scripting) to connect to remote solr
> servers. Some of the supported features:
> - Simple query search
> - Sorting - one can dynamically define new sort criterias
> - Search results are rendered very much like Google search results are
> rendered. It is also possible to view all stored field values for every hit.
> - Custom hit rendering - It is possible to show thumbnails (images) per hit
> and also customize a view for a hit based on html templates
> - Faceting - one can dynamically define field and query facets via the UI. it
> is also possible to pre-configure these facets in the configuration file.
> - Highlighting - you can dynamically configure highlighting. it can also be
> pre-configured in the configuration file
> - Spellchecking - you can dynamically configure spell checking. Can also be
> done in the configuration file. Supports collation. It is also possible to
> send "build" and "reload" commands.
> - Data import handler - if used, it is possible to send a "full-import" and
> "status" command ("delta-import" is not implemented yet, but it's easy to add)
> - Console - For development time, there's a small console which can help to
> better understand what's going on behind the scenes. One can use it to:
> ** view the client logs
> ** browse the solr scheme
> ** View a break down of the current search context
> ** View a break down of the query URL that is sent to solr
> ** View the raw JSON response returning from Solr
> This client is actually a platform that can be greatly extended for more
> things. The goal is to have a client where the explorer part is just one view
> of it. Other future views include: Monitoring, Administration, Query Builder,
> DataImportHandler configuration, and more...
> To get a better view of what's currently possible. We've set up a public
> version of this client at: http://search.jteam.nl/explorer. This client is
> configured with one solr instance where crawled YouTube movies where indexed.
> You can also check out a screencast for this deployed client:
> http://search.jteam.nl/help
> The patch created a new folder in the contrib. directory. Since the patch
> doesn't contain binaries, an additional zip file is provides that needs to be
> extract to add all the required graphics. This module is maven2 based and is
> configured in such a way that all GWT related tools/libraries are
> automatically downloaded when the modules is compiled. One of the artifacts
> of the build is a war file which can be deployed in any servlet container.
> NOTE: this client works best on WebKit based browsers (for performance
> reason) but also works on firefox and ie 7+. That said, it should be taken
> into account that it is still under development.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-master-Linux (32bit/jdk1.8.0_92) - Build # 17327 - Unstable!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/17327/ Java: 32bit/jdk1.8.0_92 -client -XX:+UseConcMarkSweepGC 1 tests failed. FAILED: org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test Error Message: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:46861/c8n_1x3_lf_shard1_replica1] Stack Trace: org.apache.solr.client.solrj.SolrServerException: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:46861/c8n_1x3_lf_shard1_replica1] at __randomizedtesting.SeedInfo.seed([699CE0C7CB6E87B5:E1C8DF1D6592EA4D]:0) at org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:753) at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1151) at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1040) at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:976) at org.apache.solr.cloud.AbstractFullDistribZkTestBase.sendDocsWithRetry(AbstractFullDistribZkTestBase.java:753) at org.apache.solr.cloud.AbstractFullDistribZkTestBase.sendDocsWithRetry(AbstractFullDistribZkTestBase.java:741) at org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.testRf3WithLeaderFailover(LeaderFailoverAfterPartitionTest.java:178) at org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test(LeaderFailoverAfterPartitionTest.java:57) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871) at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907) at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921) at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985) at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at
[jira] [Commented] (SOLR-1163) Solr Explorer - A generic GWT client for Solr
[
https://issues.apache.org/jira/browse/SOLR-1163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388844#comment-15388844
]
Alexandre Rafalovitch commented on SOLR-1163:
-
Can this be closed? Great idea, but all discussions are completely irrelevant
now if this was restarted.
> Solr Explorer - A generic GWT client for Solr
> -
>
> Key: SOLR-1163
> URL: https://issues.apache.org/jira/browse/SOLR-1163
> Project: Solr
> Issue Type: New Feature
> Components: web gui
>Affects Versions: 1.3
>Reporter: Uri Boness
> Attachments: SOLR-1163.zip, SOLR-1163.zip, graphics.zip,
> solr-explorer.patch, solr-explorer.patch
>
>
> The attached patch is a GWT generic client for solr. It is currently
> standalone, meaning that once built, one can open the generated HTML file in
> a browser and communicate with any deployed solr. It is configured with it's
> own configuration file, where one can configure the solr instance/core to
> connect to. Since it's currently standalone and completely client side based,
> it uses JSON with padding (cross-side scripting) to connect to remote solr
> servers. Some of the supported features:
> - Simple query search
> - Sorting - one can dynamically define new sort criterias
> - Search results are rendered very much like Google search results are
> rendered. It is also possible to view all stored field values for every hit.
> - Custom hit rendering - It is possible to show thumbnails (images) per hit
> and also customize a view for a hit based on html templates
> - Faceting - one can dynamically define field and query facets via the UI. it
> is also possible to pre-configure these facets in the configuration file.
> - Highlighting - you can dynamically configure highlighting. it can also be
> pre-configured in the configuration file
> - Spellchecking - you can dynamically configure spell checking. Can also be
> done in the configuration file. Supports collation. It is also possible to
> send "build" and "reload" commands.
> - Data import handler - if used, it is possible to send a "full-import" and
> "status" command ("delta-import" is not implemented yet, but it's easy to add)
> - Console - For development time, there's a small console which can help to
> better understand what's going on behind the scenes. One can use it to:
> ** view the client logs
> ** browse the solr scheme
> ** View a break down of the current search context
> ** View a break down of the query URL that is sent to solr
> ** View the raw JSON response returning from Solr
> This client is actually a platform that can be greatly extended for more
> things. The goal is to have a client where the explorer part is just one view
> of it. Other future views include: Monitoring, Administration, Query Builder,
> DataImportHandler configuration, and more...
> To get a better view of what's currently possible. We've set up a public
> version of this client at: http://search.jteam.nl/explorer. This client is
> configured with one solr instance where crawled YouTube movies where indexed.
> You can also check out a screencast for this deployed client:
> http://search.jteam.nl/help
> The patch created a new folder in the contrib. directory. Since the patch
> doesn't contain binaries, an additional zip file is provides that needs to be
> extract to add all the required graphics. This module is maven2 based and is
> configured in such a way that all GWT related tools/libraries are
> automatically downloaded when the modules is compiled. One of the artifacts
> of the build is a war file which can be deployed in any servlet container.
> NOTE: this client works best on WebKit based browsers (for performance
> reason) but also works on firefox and ie 7+. That said, it should be taken
> into account that it is still under development.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (SOLR-4372) Search text box in auto-complete/chooser extends outside of the dropdown pane on IE9 & FF 17+
[ https://issues.apache.org/jira/browse/SOLR-4372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shalin Shekhar Mangar resolved SOLR-4372. - Resolution: Won't Fix > Search text box in auto-complete/chooser extends outside of the dropdown pane > on IE9 & FF 17+ > - > > Key: SOLR-4372 > URL: https://issues.apache.org/jira/browse/SOLR-4372 > Project: Solr > Issue Type: Bug > Components: web gui >Affects Versions: 4.1, 4.2 > Environment: IE9 >Reporter: Senthuran Sivananthan >Priority: Minor > Attachments: chooser_ff17+.png, chooser_ie9.png > > > This is an issue across all of the pages. > The textbox in auto-complete/chooser extends outside of the dropdown page on > IE9 and FF17+. > Looks like there's an explicit width of 130px being specified on the textbox: > -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (SOLR-5283) Admin UI issues in IE7
[
https://issues.apache.org/jira/browse/SOLR-5283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shalin Shekhar Mangar resolved SOLR-5283.
-
Resolution: Won't Fix
> Admin UI issues in IE7
> --
>
> Key: SOLR-5283
> URL: https://issues.apache.org/jira/browse/SOLR-5283
> Project: Solr
> Issue Type: Bug
> Components: web gui
>Affects Versions: 4.4
> Environment: IE Version 7.0.5730.11 64-bit edition.
>Reporter: Erik Hatcher
>Priority: Minor
>
> A customer of ours reported:
> {code}
> IE Version 7.0.5730.11 64-bit edition.
> Result:
> Left nav area displays;
> Main area: spinning loading icon displaying the word Loading ...
> Script errors on page:
> Line: 8
> Char: 3
> Error: 'CSSStyleDeclaration' is undefined
> Code: 0
> URL: http://:/solr/js/lib/d3.js
> Line: 17
> Char: 5
> Error: Unexpected call to method or property access.
> Code: 0
> URL : http://:/solr/js/require.js
> {code}
> I've tried replicating this in a Windows virtual machine, but only have IE10
> and have not seen this issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (SOLR-4628) Solr 4.2 Admin UI
[ https://issues.apache.org/jira/browse/SOLR-4628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shalin Shekhar Mangar resolved SOLR-4628. - Resolution: Won't Fix > Solr 4.2 Admin UI > - > > Key: SOLR-4628 > URL: https://issues.apache.org/jira/browse/SOLR-4628 > Project: Solr > Issue Type: Bug > Components: web gui >Affects Versions: 4.2 > Environment: Intel PC + Win7 64 + Tomcat 6 + 19-inch 4:3 screen >Reporter: cozybreeze > Labels: 4.2, admin, gui, solr > > The new Admin web GUI has too-compact column for normal-width screen, e.g. in > the dashboard, if the directory is a long path, it will suppress the display > and replace the trailing part with '...' and there is no way to see the full > path. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5033) Started Time is Incorrect
[ https://issues.apache.org/jira/browse/SOLR-5033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388839#comment-15388839 ] Alexandre Rafalovitch commented on SOLR-5033: - Can this be closed? This was never re-verified. > Started Time is Incorrect > - > > Key: SOLR-5033 > URL: https://issues.apache.org/jira/browse/SOLR-5033 > Project: Solr > Issue Type: Bug > Components: web gui >Affects Versions: 4.2 >Reporter: Mohammed ATMANE >Priority: Minor > > In dataimport page, I have : > Indexing since 8m 30s > Requests: 0 (0/s), Fetched: 0 (0/s), Skipped: 0, Processed: 0 (0/s) > Started: 6 minutes ago > Started Time must be greater than Indexing Time. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (SOLR-3781) Admin UI does not work when wiring Solr into a larger web application
[
https://issues.apache.org/jira/browse/SOLR-3781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shalin Shekhar Mangar resolved SOLR-3781.
-
Resolution: Won't Fix
Fix Version/s: (was: 6.0)
(was: 4.9)
> Admin UI does not work when wiring Solr into a larger web application
> -
>
> Key: SOLR-3781
> URL: https://issues.apache.org/jira/browse/SOLR-3781
> Project: Solr
> Issue Type: Bug
> Components: web gui
>Affects Versions: 4.0-BETA
> Environment: win7 jetty-distribution-7.6.5.v20120716
> startup param:
> -Djetty.port=8084 -DzkRun -Dbootstrap_conf=true
>Reporter: shenjc
>Priority: Minor
> Labels: patch
> Attachments: LoadAdminUiServlet.java.chabot,
> LoadAdminUiServlet.patch, LoadAdminUiServlet_take2.patch, web.xml,
> web.xml.chabot
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> if i am wiring Solr into a larger web application which controls the web
> context root, you will probably want to mount Solr under a path prefix
> (app.war with /app/solr mounted into it, for example).
> For example:
> RootApp.war /
> myApp.war---/myApp
> prefixPath---xxx
> jsdir--js
> js filemain.js
> admin file-admin.html
> org.apache.solr.servlet.LoadAdminUiServlet
> line:49 InputStream in =
> getServletContext().getResourceAsStream("/admin.html");
> can't find admin/html because it's in the prefixPath directory
> org.apache.solr.cloud.ZkController
> line:149-150
> this.nodeName = this.hostName + ':' + this.localHostPort + '_' +
> this.localHostContext;
> this.baseURL = this.localHost + ":" + this.localHostPort + "/" +
> this.localHostContext;
> it can't match this condition
> baseURL need to be http://xx:xx/myApp/myPrefixPath
> eg. http://xx:xx/myApp/xxx
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4372) Search text box in auto-complete/chooser extends outside of the dropdown pane on IE9 & FF 17+
[ https://issues.apache.org/jira/browse/SOLR-4372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388837#comment-15388837 ] Alexandre Rafalovitch commented on SOLR-4372: - Can this be closed? This is old browsers against old admin UI. > Search text box in auto-complete/chooser extends outside of the dropdown pane > on IE9 & FF 17+ > - > > Key: SOLR-4372 > URL: https://issues.apache.org/jira/browse/SOLR-4372 > Project: Solr > Issue Type: Bug > Components: web gui >Affects Versions: 4.1, 4.2 > Environment: IE9 >Reporter: Senthuran Sivananthan >Priority: Minor > Attachments: chooser_ff17+.png, chooser_ie9.png > > > This is an issue across all of the pages. > The textbox in auto-complete/chooser extends outside of the dropdown page on > IE9 and FF17+. > Looks like there's an explicit width of 130px being specified on the textbox: > -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5283) Admin UI issues in IE7
[
https://issues.apache.org/jira/browse/SOLR-5283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388831#comment-15388831
]
Alexandre Rafalovitch commented on SOLR-5283:
-
Can this be closed? IE7 is well dead (I hope)
> Admin UI issues in IE7
> --
>
> Key: SOLR-5283
> URL: https://issues.apache.org/jira/browse/SOLR-5283
> Project: Solr
> Issue Type: Bug
> Components: web gui
>Affects Versions: 4.4
> Environment: IE Version 7.0.5730.11 64-bit edition.
>Reporter: Erik Hatcher
>Priority: Minor
>
> A customer of ours reported:
> {code}
> IE Version 7.0.5730.11 64-bit edition.
> Result:
> Left nav area displays;
> Main area: spinning loading icon displaying the word Loading ...
> Script errors on page:
> Line: 8
> Char: 3
> Error: 'CSSStyleDeclaration' is undefined
> Code: 0
> URL: http://:/solr/js/lib/d3.js
> Line: 17
> Char: 5
> Error: Unexpected call to method or property access.
> Code: 0
> URL : http://:/solr/js/require.js
> {code}
> I've tried replicating this in a Windows virtual machine, but only have IE10
> and have not seen this issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4628) Solr 4.2 Admin UI
[ https://issues.apache.org/jira/browse/SOLR-4628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388827#comment-15388827 ] Alexandre Rafalovitch commented on SOLR-4628: - This is ancient against old Admin UI. Can it be closed? > Solr 4.2 Admin UI > - > > Key: SOLR-4628 > URL: https://issues.apache.org/jira/browse/SOLR-4628 > Project: Solr > Issue Type: Bug > Components: web gui >Affects Versions: 4.2 > Environment: Intel PC + Win7 64 + Tomcat 6 + 19-inch 4:3 screen >Reporter: cozybreeze > Labels: 4.2, admin, gui, solr > > The new Admin web GUI has too-compact column for normal-width screen, e.g. in > the dashboard, if the directory is a long path, it will suppress the display > and replace the trailing part with '...' and there is no way to see the full > path. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4721) Admin UI - Error Handling on initial load
[
https://issues.apache.org/jira/browse/SOLR-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388822#comment-15388822
]
Alexandre Rafalovitch commented on SOLR-4721:
-
Is this safe to close? Open a new one for new Admin UI if the issue happens
there.
> Admin UI - Error Handling on initial load
> -
>
> Key: SOLR-4721
> URL: https://issues.apache.org/jira/browse/SOLR-4721
> Project: Solr
> Issue Type: Improvement
> Components: web gui
>Reporter: Stefan Matheis (steffkes)
>Assignee: Stefan Matheis (steffkes)
> Fix For: 4.9, 6.0
>
>
> Seen this this Week on IRC, at least once. The UI was not completely loaded
> (only displayed the initial "Loading ..." forever) because {{/admin/cores}}
> doesn't respond with valid information.
> don't remember of the actual error in this case, but we should handle this
> scenario better than we do right now.
> We'll see what information we can get from the request and how we can display
> it. at least it should block the whole UI somehow that make it clear that
> something goes completely wrong - there is no way around, /admin/cores is the
> very first request we make to gather initial informations ..
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-Tests-6.x - Build # 341 - Still Unstable
Build: https://builds.apache.org/job/Lucene-Solr-Tests-6.x/341/ 1 tests failed. FAILED: org.apache.solr.cloud.TestHdfsCloudBackupRestore.test Error Message: Error from server at http://127.0.0.1:38990/solr: The backup directory already exists: hdfs://localhost:59991/backup/mytestbackup Stack Trace: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:38990/solr: The backup directory already exists: hdfs://localhost:59991/backup/mytestbackup at __randomizedtesting.SeedInfo.seed([B0705C4E8E343756:3824639420C85AAE]:0) at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:590) at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259) at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248) at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:403) at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:356) at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1270) at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1040) at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:976) at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149) at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:166) at org.apache.solr.cloud.AbstractCloudBackupRestoreTestCase.testBackupAndRestore(AbstractCloudBackupRestoreTestCase.java:206) at org.apache.solr.cloud.AbstractCloudBackupRestoreTestCase.test(AbstractCloudBackupRestoreTestCase.java:126) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871) at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907) at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at
[jira] [Commented] (SOLR-4466) DIH "Started" status says "less than a minute ago" when it's been much longer
[ https://issues.apache.org/jira/browse/SOLR-4466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388773#comment-15388773 ] Alexandre Rafalovitch commented on SOLR-4466: - Nobody confirmed this and we have a new UI now. Can this one be closed? > DIH "Started" status says "less than a minute ago" when it's been much longer > - > > Key: SOLR-4466 > URL: https://issues.apache.org/jira/browse/SOLR-4466 > Project: Solr > Issue Type: Bug > Components: web gui >Affects Versions: 4.1 >Reporter: Shawn Heisey > Fix For: 4.9, 6.0 > > > Sometimes (exact circumstances not known) the "Started" output on the DIH > status in the 4.1 UI will always say "less than a minute ago" even though the > actual value is quite some time ago. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-3781) Admin UI does not work when wiring Solr into a larger web application
[
https://issues.apache.org/jira/browse/SOLR-3781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388767#comment-15388767
]
Alexandre Rafalovitch commented on SOLR-3781:
-
Safe to close? No longer relevant.
> Admin UI does not work when wiring Solr into a larger web application
> -
>
> Key: SOLR-3781
> URL: https://issues.apache.org/jira/browse/SOLR-3781
> Project: Solr
> Issue Type: Bug
> Components: web gui
>Affects Versions: 4.0-BETA
> Environment: win7 jetty-distribution-7.6.5.v20120716
> startup param:
> -Djetty.port=8084 -DzkRun -Dbootstrap_conf=true
>Reporter: shenjc
>Priority: Minor
> Labels: patch
> Fix For: 4.9, 6.0
>
> Attachments: LoadAdminUiServlet.java.chabot,
> LoadAdminUiServlet.patch, LoadAdminUiServlet_take2.patch, web.xml,
> web.xml.chabot
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> if i am wiring Solr into a larger web application which controls the web
> context root, you will probably want to mount Solr under a path prefix
> (app.war with /app/solr mounted into it, for example).
> For example:
> RootApp.war /
> myApp.war---/myApp
> prefixPath---xxx
> jsdir--js
> js filemain.js
> admin file-admin.html
> org.apache.solr.servlet.LoadAdminUiServlet
> line:49 InputStream in =
> getServletContext().getResourceAsStream("/admin.html");
> can't find admin/html because it's in the prefixPath directory
> org.apache.solr.cloud.ZkController
> line:149-150
> this.nodeName = this.hostName + ':' + this.localHostPort + '_' +
> this.localHostContext;
> this.baseURL = this.localHost + ":" + this.localHostPort + "/" +
> this.localHostContext;
> it can't match this condition
> baseURL need to be http://xx:xx/myApp/myPrefixPath
> eg. http://xx:xx/myApp/xxx
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4268) Admin UI - button to unload transient core without removing from solr.xml
[ https://issues.apache.org/jira/browse/SOLR-4268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388762#comment-15388762 ] Alexandre Rafalovitch commented on SOLR-4268: - This seems super-legacy, safe to close? > Admin UI - button to unload transient core without removing from solr.xml > - > > Key: SOLR-4268 > URL: https://issues.apache.org/jira/browse/SOLR-4268 > Project: Solr > Issue Type: Improvement > Components: web gui >Affects Versions: 4.0 >Reporter: Shawn Heisey > Fix For: 4.9, 6.0 > > > The core "unload" button in the UI currently will completely remove a core > from solr.xml. With the implentation of transient cores, there should be a > way to ask Solr to unload a core without removing it entirely. > This leads into a discussion about terminology. UNLOAD isn't a good > single-word description for what it does. A case could be made for having > REMOVE and DELETE actions for CoreAdmin, with confirmation prompts if you > click on those buttons in the UI. DELETE could simply be an option on REMOVE > - which I think you can actually currently do with UNLOAD. > Another idea, not sure if it needs its own issue or is part of this one: If a > core is mentioned in solr.xml but not actually loaded, it would be very cool > if it were listed, but with a different background color to indicate the > non-loaded state. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-3969) Admin dashboard -- include cache/buffer memory utilization
[ https://issues.apache.org/jira/browse/SOLR-3969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388760#comment-15388760 ] Alexandre Rafalovitch commented on SOLR-3969: - Was this ever found to be doable on the backend? > Admin dashboard -- include cache/buffer memory utilization > -- > > Key: SOLR-3969 > URL: https://issues.apache.org/jira/browse/SOLR-3969 > Project: Solr > Issue Type: Improvement > Components: web gui >Affects Versions: 4.0 > Environment: Linux bigindy5 2.6.32-279.9.1.el6.centos.plus.x86_64 #1 > SMP Wed Sep 26 03:52:55 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux > java version "1.7.0_07" > Java(TM) SE Runtime Environment (build 1.7.0_07-b10) > Java HotSpot(TM) 64-Bit Server VM (build 23.3-b01, mixed mode) >Reporter: Shawn Heisey >Priority: Minor > Fix For: 4.9, 6.0 > > > The admin dashboard includes memory utilization, but there is no way on that > screen to see how much memory is being used for cache and buffers -- two > additional numbers shown by top. > The most visually appealing way to display this would be to break the > Physical Memory graph into multiple colors, but if that's not practical, more > graphs could be added. I'm not even sure it's possible, but if it is, it > would make a great addition. Hopefully there would also be a cross-platform > way of doing it so that the major platforms could all be supported. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-Tests-master - Build # 1282 - Still Unstable
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/1282/ 2 tests failed. FAILED: junit.framework.TestSuite.org.apache.solr.cloud.CleanupOldIndexTest Error Message: Suite timeout exceeded (>= 720 msec). Stack Trace: java.lang.Exception: Suite timeout exceeded (>= 720 msec). at __randomizedtesting.SeedInfo.seed([71E23E1618C042B2]:0) FAILED: org.apache.solr.security.BasicAuthIntegrationTest.testBasics Error Message: IOException occured when talking to server at: http://127.0.0.1:43708/solr/testSolrCloudCollection_shard1_replica2 Stack Trace: org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: IOException occured when talking to server at: http://127.0.0.1:43708/solr/testSolrCloudCollection_shard1_replica2 at __randomizedtesting.SeedInfo.seed([71E23E1618C042B2:4C3A903A202E1CC2]:0) at org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:739) at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1151) at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1040) at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:976) at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219) at org.apache.solr.security.BasicAuthIntegrationTest.doExtraTests(BasicAuthIntegrationTest.java:193) at org.apache.solr.cloud.TestMiniSolrCloudClusterBase.testCollectionCreateSearchDelete(TestMiniSolrCloudClusterBase.java:196) at org.apache.solr.cloud.TestMiniSolrCloudClusterBase.testBasics(TestMiniSolrCloudClusterBase.java:79) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871) at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907) at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at
[JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+127) - Build # 17325 - Unstable!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/17325/ Java: 32bit/jdk-9-ea+127 -server -XX:+UseG1GC 2 tests failed. FAILED: junit.framework.TestSuite.org.apache.solr.handler.component.SearchHandlerTest Error Message: 1 thread leaked from SUITE scope at org.apache.solr.handler.component.SearchHandlerTest: 1) Thread[id=1840, name=OverseerHdfsCoreFailoverThread-96281734833635339-127.0.0.1:34161_solr-n_04, state=TIMED_WAITING, group=Overseer Hdfs SolrCore Failover Thread.] at java.lang.Thread.sleep(java.base@9-ea/Native Method) at org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:137) at java.lang.Thread.run(java.base@9-ea/Thread.java:843) Stack Trace: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.handler.component.SearchHandlerTest: 1) Thread[id=1840, name=OverseerHdfsCoreFailoverThread-96281734833635339-127.0.0.1:34161_solr-n_04, state=TIMED_WAITING, group=Overseer Hdfs SolrCore Failover Thread.] at java.lang.Thread.sleep(java.base@9-ea/Native Method) at org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:137) at java.lang.Thread.run(java.base@9-ea/Thread.java:843) at __randomizedtesting.SeedInfo.seed([6D3B397704BABB20]:0) FAILED: junit.framework.TestSuite.org.apache.solr.handler.component.SearchHandlerTest Error Message: There are still zombie threads that couldn't be terminated:1) Thread[id=1840, name=OverseerHdfsCoreFailoverThread-96281734833635339-127.0.0.1:34161_solr-n_04, state=RUNNABLE, group=Overseer Hdfs SolrCore Failover Thread.] at java.lang.Thread.sleep(java.base@9-ea/Native Method) at org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:137) at java.lang.Thread.run(java.base@9-ea/Thread.java:843) Stack Trace: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated: 1) Thread[id=1840, name=OverseerHdfsCoreFailoverThread-96281734833635339-127.0.0.1:34161_solr-n_04, state=RUNNABLE, group=Overseer Hdfs SolrCore Failover Thread.] at java.lang.Thread.sleep(java.base@9-ea/Native Method) at org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:137) at java.lang.Thread.run(java.base@9-ea/Thread.java:843) at __randomizedtesting.SeedInfo.seed([6D3B397704BABB20]:0) Build Log: [...truncated 10764 lines...] [junit4] Suite: org.apache.solr.handler.component.SearchHandlerTest [junit4] 2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.handler.component.SearchHandlerTest_6D3B397704BABB20-001/init-core-data-001 [junit4] 2> 191736 INFO (SUITE-SearchHandlerTest-seed#[6D3B397704BABB20]-worker) [] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN) [junit4] 2> 191736 INFO (SUITE-SearchHandlerTest-seed#[6D3B397704BABB20]-worker) [] o.a.s.SolrTestCaseJ4 initCore [junit4] 2> 191736 INFO (SUITE-SearchHandlerTest-seed#[6D3B397704BABB20]-worker) [] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1' [junit4] 2> 191736 INFO (SUITE-SearchHandlerTest-seed#[6D3B397704BABB20]-worker) [] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx) [junit4] 2> 191736 INFO (SUITE-SearchHandlerTest-seed#[6D3B397704BABB20]-worker) [] o.a.s.c.SolrResourceLoader using system property solr.solr.home: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr [junit4] 2> 191736 INFO (SUITE-SearchHandlerTest-seed#[6D3B397704BABB20]-worker) [] o.a.s.c.SolrResourceLoader Adding 'file:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader [junit4] 2> 191736 INFO (SUITE-SearchHandlerTest-seed#[6D3B397704BABB20]-worker) [] o.a.s.c.SolrResourceLoader Adding 'file:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1/lib/README' to classloader [junit4] 2> 191758 INFO (SUITE-SearchHandlerTest-seed#[6D3B397704BABB20]-worker) [] o.a.s.c.SolrConfig current version of requestparams : -1 [junit4] 2> 191764 INFO (SUITE-SearchHandlerTest-seed#[6D3B397704BABB20]-worker) [] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0 [junit4] 2> 191781 INFO (SUITE-SearchHandlerTest-seed#[6D3B397704BABB20]-worker) [] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml [junit4] 2> 191785 INFO
[jira] [Commented] (SOLR-7756) ExactStatsCache and LRUStatsCache will throw an NPE when a term is not present on a shard
[
https://issues.apache.org/jira/browse/SOLR-7756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388602#comment-15388602
]
Steve Rowe commented on SOLR-7756:
--
SOLR-9328 has a reproducible test failure for
{{TestDistribIDF.testMultiCollectionQuery()}}, which fails on the 5.3.0
release's source, which is when the test was introduced by this issue.
> ExactStatsCache and LRUStatsCache will throw an NPE when a term is not
> present on a shard
> -
>
> Key: SOLR-7756
> URL: https://issues.apache.org/jira/browse/SOLR-7756
> Project: Solr
> Issue Type: Bug
>Affects Versions: 5.0, 5.0.1, 5.1, 5.2, 5.2.1, 5.2.2
>Reporter: Varun Thacker
>Assignee: Varun Thacker
> Fix For: 5.3
>
> Attachments: SOLR-7756.patch, SOLR-7756.patch, SOLR-7756.patch,
> SOLR-7756.patch, SOLR-7756.patch, SOLR-7756.patch
>
>
> If a term doesn't exist on a shard {{ExactStatsCache#getPerShardTermStats}}
> throws an NullPointerException.
> Attaching a test and a patch shortly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9328) TestDistribIDF.testMultiCollectionQuery() failure
[
https://issues.apache.org/jira/browse/SOLR-9328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388600#comment-15388600
]
Steve Rowe commented on SOLR-9328:
--
I checked out the {{releases/lucene-solr/5.3.0}} tag (the first release with
this test, introduced by SOLR-7756) and the test fails in the same way there,
so this problem existed when the test was introduced.
> TestDistribIDF.testMultiCollectionQuery() failure
> -
>
> Key: SOLR-9328
> URL: https://issues.apache.org/jira/browse/SOLR-9328
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Affects Versions: 5.3, 6.2
>Reporter: Steve Rowe
>
> Nightly branch_6x failure, reproduces for me, from
> [https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/125/]:
> {noformat}
> Checking out Revision 0ec1c2509770267e30477f49a890a9d2333cf40d
> (refs/remotes/origin/branch_6x)
> [...]
>[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestDistribIDF
> -Dtests.method=testMultiCollectionQuery -Dtests.seed=1F45A3F87F67AC9C
> -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true
> -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt
> -Dtests.locale=en-ZA -Dtests.timezone=Europe/Tallinn -Dtests.asserts=true
> -Dtests.file.encoding=UTF-8
>[junit4] FAILURE 14.1s J0 | TestDistribIDF.testMultiCollectionQuery <<<
>[junit4]> Throwable #1: java.lang.AssertionError: Doc1 score=0.9808292
> Doc2 score=0.9808292 expected:<1> but was:<0>
>[junit4]> at
> __randomizedtesting.SeedInfo.seed([1F45A3F87F67AC9C:E3664C9A311A9E0]:0)
>[junit4]> at
> org.apache.solr.search.stats.TestDistribIDF.testMultiCollectionQuery(TestDistribIDF.java:183)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-9328) TestDistribIDF.testMultiCollectionQuery() failure
[
https://issues.apache.org/jira/browse/SOLR-9328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Rowe updated SOLR-9328:
-
Affects Version/s: 6.2
5.3
> TestDistribIDF.testMultiCollectionQuery() failure
> -
>
> Key: SOLR-9328
> URL: https://issues.apache.org/jira/browse/SOLR-9328
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Affects Versions: 5.3, 6.2
>Reporter: Steve Rowe
>
> Nightly branch_6x failure, reproduces for me, from
> [https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/125/]:
> {noformat}
> Checking out Revision 0ec1c2509770267e30477f49a890a9d2333cf40d
> (refs/remotes/origin/branch_6x)
> [...]
>[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestDistribIDF
> -Dtests.method=testMultiCollectionQuery -Dtests.seed=1F45A3F87F67AC9C
> -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true
> -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt
> -Dtests.locale=en-ZA -Dtests.timezone=Europe/Tallinn -Dtests.asserts=true
> -Dtests.file.encoding=UTF-8
>[junit4] FAILURE 14.1s J0 | TestDistribIDF.testMultiCollectionQuery <<<
>[junit4]> Throwable #1: java.lang.AssertionError: Doc1 score=0.9808292
> Doc2 score=0.9808292 expected:<1> but was:<0>
>[junit4]> at
> __randomizedtesting.SeedInfo.seed([1F45A3F87F67AC9C:E3664C9A311A9E0]:0)
>[junit4]> at
> org.apache.solr.search.stats.TestDistribIDF.testMultiCollectionQuery(TestDistribIDF.java:183)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-9328) TestDistribIDF.testMultiCollectionQuery() failure
Steve Rowe created SOLR-9328:
Summary: TestDistribIDF.testMultiCollectionQuery() failure
Key: SOLR-9328
URL: https://issues.apache.org/jira/browse/SOLR-9328
Project: Solr
Issue Type: Bug
Security Level: Public (Default Security Level. Issues are Public)
Reporter: Steve Rowe
Nightly branch_6x failure, reproduces for me, from
[https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/125/]:
{noformat}
Checking out Revision 0ec1c2509770267e30477f49a890a9d2333cf40d
(refs/remotes/origin/branch_6x)
[...]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestDistribIDF
-Dtests.method=testMultiCollectionQuery -Dtests.seed=1F45A3F87F67AC9C
-Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true
-Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt
-Dtests.locale=en-ZA -Dtests.timezone=Europe/Tallinn -Dtests.asserts=true
-Dtests.file.encoding=UTF-8
[junit4] FAILURE 14.1s J0 | TestDistribIDF.testMultiCollectionQuery <<<
[junit4]> Throwable #1: java.lang.AssertionError: Doc1 score=0.9808292
Doc2 score=0.9808292 expected:<1> but was:<0>
[junit4]>at
__randomizedtesting.SeedInfo.seed([1F45A3F87F67AC9C:E3664C9A311A9E0]:0)
[junit4]>at
org.apache.solr.search.stats.TestDistribIDF.testMultiCollectionQuery(TestDistribIDF.java:183)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts
[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388541#comment-15388541 ] Erick Erickson commented on SOLR-7280: -- Well, I'll still test it for fun, if _you're_worried I should be more se. But it sounds like you're getting more comfortable with the approach. Anyway, a couple of things: bq: If you simply do that and walk away and come back in the morning, it will work These aren't Junit tests, but scripts because of similar concerns I had. They: 1> bring up and down real solr JVMs via shell scripts, so the default 3 minute timeout is in place That said, making it longer (say 20 minutes?) would emphasize the issueand... 2> I have a monitor process that records how long it took for all the replicas to come up so I can see any anomalies. 3> brings JVMs up and down in different orders to avoid happening to have all the leaders on the first node that comes up. I've been nervous that the way I'm testing certainly isn't foolproof. bq: registering in ZK got spun off into it's own thread Hmm, interesting since the symptom I'm seeing is being unable to spawn native threads and a large spike in threads during startup (steady-state 1,200 threads, startup started crapping out around 2,600). upping the Xmx or Xss (or both) doesn't matter and bumping the ulimit didn't either > Load cores in sorted order and tweak coreLoadThread counts to improve cluster > stability on restarts > --- > > Key: SOLR-7280 > URL: https://issues.apache.org/jira/browse/SOLR-7280 > Project: Solr > Issue Type: Bug > Components: SolrCloud >Reporter: Shalin Shekhar Mangar >Assignee: Noble Paul > Fix For: 6.2, 5.5.3 > > Attachments: SOLR-7280-5x.patch, SOLR-7280-5x.patch, > SOLR-7280-5x.patch, SOLR-7280-test.patch, SOLR-7280.patch, SOLR-7280.patch > > > In SOLR-7191, Damien mentioned that by loading solr cores in a sorted order > and tweaking some of the coreLoadThread counts, he was able to improve the > stability of a cluster with thousands of collections. We should explore some > of these changes and fold them into Solr. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts
[
https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388538#comment-15388538
]
Mike Drob commented on SOLR-7280:
-
If anybody wants to apply that, they should also add a call to
{{resetFactory()}} in the {{@AfterClass}} that I missed.
> Load cores in sorted order and tweak coreLoadThread counts to improve cluster
> stability on restarts
> ---
>
> Key: SOLR-7280
> URL: https://issues.apache.org/jira/browse/SOLR-7280
> Project: Solr
> Issue Type: Bug
> Components: SolrCloud
>Reporter: Shalin Shekhar Mangar
>Assignee: Noble Paul
> Fix For: 6.2, 5.5.3
>
> Attachments: SOLR-7280-5x.patch, SOLR-7280-5x.patch,
> SOLR-7280-5x.patch, SOLR-7280-test.patch, SOLR-7280.patch, SOLR-7280.patch
>
>
> In SOLR-7191, Damien mentioned that by loading solr cores in a sorted order
> and tweaking some of the coreLoadThread counts, he was able to improve the
> stability of a cluster with thousands of collections. We should explore some
> of these changes and fold them into Solr.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts
[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388529#comment-15388529 ] Mike Drob edited comment on SOLR-7280 at 7/21/16 10:23 PM: --- This is the test that Mark was talking about. (attached) was (Author: mdrob): This is the test that Mark was talking about. > Load cores in sorted order and tweak coreLoadThread counts to improve cluster > stability on restarts > --- > > Key: SOLR-7280 > URL: https://issues.apache.org/jira/browse/SOLR-7280 > Project: Solr > Issue Type: Bug > Components: SolrCloud >Reporter: Shalin Shekhar Mangar >Assignee: Noble Paul > Fix For: 6.2, 5.5.3 > > Attachments: SOLR-7280-5x.patch, SOLR-7280-5x.patch, > SOLR-7280-5x.patch, SOLR-7280-test.patch, SOLR-7280.patch, SOLR-7280.patch > > > In SOLR-7191, Damien mentioned that by loading solr cores in a sorted order > and tweaking some of the coreLoadThread counts, he was able to improve the > stability of a cluster with thousands of collections. We should explore some > of these changes and fold them into Solr. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts
[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Drob updated SOLR-7280: Attachment: SOLR-7280-test.patch This is the test that Mark was talking about. > Load cores in sorted order and tweak coreLoadThread counts to improve cluster > stability on restarts > --- > > Key: SOLR-7280 > URL: https://issues.apache.org/jira/browse/SOLR-7280 > Project: Solr > Issue Type: Bug > Components: SolrCloud >Reporter: Shalin Shekhar Mangar >Assignee: Noble Paul > Fix For: 6.2, 5.5.3 > > Attachments: SOLR-7280-5x.patch, SOLR-7280-5x.patch, > SOLR-7280-5x.patch, SOLR-7280-test.patch, SOLR-7280.patch, SOLR-7280.patch > > > In SOLR-7191, Damien mentioned that by loading solr cores in a sorted order > and tweaking some of the coreLoadThread counts, he was able to improve the > stability of a cluster with thousands of collections. We should explore some > of these changes and fold them into Solr. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts
[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388456#comment-15388456 ] Mark Miller commented on SOLR-7280: --- Okay, I think I got it. At some point after the bug I'm referring to was resolved, registering in ZK got spun off into it's own thread and no longer holds up the core load thread. You can see this in ZkContainer. That effectively creates the situation I suggest above: bq. If it indeed remains an issue, another thing we might try is loading cores with a limited number of threads, but registering with ZK with many more, instead of doing both in the same thread as we are now. So we are okay on this issue. [~mdrob], it would be great to get some form of that test committed. > Load cores in sorted order and tweak coreLoadThread counts to improve cluster > stability on restarts > --- > > Key: SOLR-7280 > URL: https://issues.apache.org/jira/browse/SOLR-7280 > Project: Solr > Issue Type: Bug > Components: SolrCloud >Reporter: Shalin Shekhar Mangar >Assignee: Noble Paul > Fix For: 6.2, 5.5.3 > > Attachments: SOLR-7280-5x.patch, SOLR-7280-5x.patch, > SOLR-7280-5x.patch, SOLR-7280.patch, SOLR-7280.patch > > > In SOLR-7191, Damien mentioned that by loading solr cores in a sorted order > and tweaking some of the coreLoadThread counts, he was able to improve the > stability of a cluster with thousands of collections. We should explore some > of these changes and fold them into Solr. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-7381) Add new RangeField
[
https://issues.apache.org/jira/browse/LUCENE-7381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388455#comment-15388455
]
Steve Rowe commented on LUCENE-7381:
Couple reproducing nightly tests on branch_6x, from
[https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/125/]:
{noformat}
[junit4] Suite: org.apache.lucene.search.TestDoubleRangeFieldQueries
[junit4] 2> NOTE: download the large Jenkins line-docs file by running
'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test
-Dtestcase=TestDoubleRangeFieldQueries -Dtests.method=testMultiValued
-Dtests.seed=AE24BFDDECD216AC -Dtests.multiplier=2 -Dtests.nightly=true
-Dtests.slow=true
-Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt
-Dtests.locale=hu -Dtests.timezone=Europe/San_Marino -Dtests.asserts=true
-Dtests.file.encoding=US-ASCII
[junit4] FAILURE 4.20s J0 | TestDoubleRangeFieldQueries.testMultiValued <<<
[junit4]> Throwable #1: java.lang.AssertionError: wrong hit (first of
possibly more):
[junit4]> FAIL: id=204 should not match but did
[junit4]> queryBox=Box(Infinity TO Infinity)
[junit4]> box=Box(Infinity TO Infinity)
[junit4]> queryType=CONTAINS
[junit4]> deleted?=false
[junit4]>at
__randomizedtesting.SeedInfo.seed([AE24BFDDECD216AC:7A04DBEF221056E4]:0)
[junit4]>at
org.apache.lucene.search.BaseRangeFieldQueryTestCase.verify(BaseRangeFieldQueryTestCase.java:278)
[junit4]>at
org.apache.lucene.search.BaseRangeFieldQueryTestCase.doTestRandom(BaseRangeFieldQueryTestCase.java:154)
[junit4]>at
org.apache.lucene.search.BaseRangeFieldQueryTestCase.testMultiValued(BaseRangeFieldQueryTestCase.java:73)
[junit4]>at java.lang.Thread.run(Thread.java:745)
[junit4] 2> NOTE: download the large Jenkins line-docs file by running
'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test
-Dtestcase=TestDoubleRangeFieldQueries -Dtests.method=testRandomBig
-Dtests.seed=AE24BFDDECD216AC -Dtests.multiplier=2 -Dtests.nightly=true
-Dtests.slow=true
-Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt
-Dtests.locale=hu -Dtests.timezone=Europe/San_Marino -Dtests.asserts=true
-Dtests.file.encoding=US-ASCII
[junit4] FAILURE 35.1s J0 | TestDoubleRangeFieldQueries.testRandomBig <<<
[junit4]> Throwable #1: java.lang.AssertionError: wrong hit (first of
possibly more):
[junit4]> FAIL: id=49 should not match but did
[junit4]> queryBox=Box(Infinity TO Infinity)
[junit4]> box=Box(Infinity TO Infinity)
[junit4]> queryType=CONTAINS
[junit4]> deleted?=false
[junit4]>at
__randomizedtesting.SeedInfo.seed([AE24BFDDECD216AC:2973C2527D8B6A2C]:0)
[junit4]>at
org.apache.lucene.search.BaseRangeFieldQueryTestCase.verify(BaseRangeFieldQueryTestCase.java:278)
[junit4]>at
org.apache.lucene.search.BaseRangeFieldQueryTestCase.doTestRandom(BaseRangeFieldQueryTestCase.java:154)
[junit4]>at
org.apache.lucene.search.BaseRangeFieldQueryTestCase.testRandomBig(BaseRangeFieldQueryTestCase.java:69)
[junit4]>at java.lang.Thread.run(Thread.java:745)
[junit4] 2> NOTE: leaving temporary files on disk at:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/sandbox/test/J0/temp/lucene.search.TestDoubleRangeFieldQueries_AE24BFDDECD216AC-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene62):
{id=PostingsFormat(name=Memory doPackFST= false)},
docValues:{id=DocValuesFormat(name=Memory)}, maxPointsInLeafNode=1844,
maxMBSortInHeap=6.78445457997637, sim=ClassicSimilarity, locale=hu,
timezone=Europe/San_Marino
[junit4] 2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation
1.8.0_74 (64-bit)/cpus=4,threads=1,free=179129064,total=341311488
[junit4] 2> NOTE: All tests run in this JVM: [FuzzyLikeThisQueryTest,
TestDocValuesTermsQuery, TestTermAutomatonQuery, TestLatLonPoint,
TestBigIntegerPoint, TestDoubleRangeField, TestHalfFloatPoint,
TestInetAddressPoint, TestLatLonDocValuesField, TestPayloadSpanUtil,
TestDoubleRangeFieldQueries]
[junit4] Completed [18/20 (1!)] on J0 in 42.60s, 5 tests, 2 failures <<<
FAILURES!
{noformat}
> Add new RangeField
> --
>
> Key: LUCENE-7381
> URL: https://issues.apache.org/jira/browse/LUCENE-7381
> Project: Lucene - Core
> Issue Type: New Feature
>Reporter: Nicholas Knize
> Attachments: LUCENE-7381.patch, LUCENE-7381.patch, LUCENE-7381.patch,
> LUCENE-7381.patch, LUCENE-7381.patch, LUCENE-7381.patch
>
>
> I've been tinkering with a new Point-based {{RangeField}} for indexing
> numeric ranges that could be useful for a number of applications.
> For example,
[jira] [Comment Edited] (LUCENE-7381) Add new RangeField
[
https://issues.apache.org/jira/browse/LUCENE-7381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388455#comment-15388455
]
Steve Rowe edited comment on LUCENE-7381 at 7/21/16 9:23 PM:
-
Couple reproducing nightly test failures on branch_6x, from
[https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/125/]:
{noformat}
[junit4] Suite: org.apache.lucene.search.TestDoubleRangeFieldQueries
[junit4] 2> NOTE: download the large Jenkins line-docs file by running
'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test
-Dtestcase=TestDoubleRangeFieldQueries -Dtests.method=testMultiValued
-Dtests.seed=AE24BFDDECD216AC -Dtests.multiplier=2 -Dtests.nightly=true
-Dtests.slow=true
-Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt
-Dtests.locale=hu -Dtests.timezone=Europe/San_Marino -Dtests.asserts=true
-Dtests.file.encoding=US-ASCII
[junit4] FAILURE 4.20s J0 | TestDoubleRangeFieldQueries.testMultiValued <<<
[junit4]> Throwable #1: java.lang.AssertionError: wrong hit (first of
possibly more):
[junit4]> FAIL: id=204 should not match but did
[junit4]> queryBox=Box(Infinity TO Infinity)
[junit4]> box=Box(Infinity TO Infinity)
[junit4]> queryType=CONTAINS
[junit4]> deleted?=false
[junit4]>at
__randomizedtesting.SeedInfo.seed([AE24BFDDECD216AC:7A04DBEF221056E4]:0)
[junit4]>at
org.apache.lucene.search.BaseRangeFieldQueryTestCase.verify(BaseRangeFieldQueryTestCase.java:278)
[junit4]>at
org.apache.lucene.search.BaseRangeFieldQueryTestCase.doTestRandom(BaseRangeFieldQueryTestCase.java:154)
[junit4]>at
org.apache.lucene.search.BaseRangeFieldQueryTestCase.testMultiValued(BaseRangeFieldQueryTestCase.java:73)
[junit4]>at java.lang.Thread.run(Thread.java:745)
[junit4] 2> NOTE: download the large Jenkins line-docs file by running
'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test
-Dtestcase=TestDoubleRangeFieldQueries -Dtests.method=testRandomBig
-Dtests.seed=AE24BFDDECD216AC -Dtests.multiplier=2 -Dtests.nightly=true
-Dtests.slow=true
-Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt
-Dtests.locale=hu -Dtests.timezone=Europe/San_Marino -Dtests.asserts=true
-Dtests.file.encoding=US-ASCII
[junit4] FAILURE 35.1s J0 | TestDoubleRangeFieldQueries.testRandomBig <<<
[junit4]> Throwable #1: java.lang.AssertionError: wrong hit (first of
possibly more):
[junit4]> FAIL: id=49 should not match but did
[junit4]> queryBox=Box(Infinity TO Infinity)
[junit4]> box=Box(Infinity TO Infinity)
[junit4]> queryType=CONTAINS
[junit4]> deleted?=false
[junit4]>at
__randomizedtesting.SeedInfo.seed([AE24BFDDECD216AC:2973C2527D8B6A2C]:0)
[junit4]>at
org.apache.lucene.search.BaseRangeFieldQueryTestCase.verify(BaseRangeFieldQueryTestCase.java:278)
[junit4]>at
org.apache.lucene.search.BaseRangeFieldQueryTestCase.doTestRandom(BaseRangeFieldQueryTestCase.java:154)
[junit4]>at
org.apache.lucene.search.BaseRangeFieldQueryTestCase.testRandomBig(BaseRangeFieldQueryTestCase.java:69)
[junit4]>at java.lang.Thread.run(Thread.java:745)
[junit4] 2> NOTE: leaving temporary files on disk at:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/sandbox/test/J0/temp/lucene.search.TestDoubleRangeFieldQueries_AE24BFDDECD216AC-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene62):
{id=PostingsFormat(name=Memory doPackFST= false)},
docValues:{id=DocValuesFormat(name=Memory)}, maxPointsInLeafNode=1844,
maxMBSortInHeap=6.78445457997637, sim=ClassicSimilarity, locale=hu,
timezone=Europe/San_Marino
[junit4] 2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation
1.8.0_74 (64-bit)/cpus=4,threads=1,free=179129064,total=341311488
[junit4] 2> NOTE: All tests run in this JVM: [FuzzyLikeThisQueryTest,
TestDocValuesTermsQuery, TestTermAutomatonQuery, TestLatLonPoint,
TestBigIntegerPoint, TestDoubleRangeField, TestHalfFloatPoint,
TestInetAddressPoint, TestLatLonDocValuesField, TestPayloadSpanUtil,
TestDoubleRangeFieldQueries]
[junit4] Completed [18/20 (1!)] on J0 in 42.60s, 5 tests, 2 failures <<<
FAILURES!
{noformat}
was (Author: steve_rowe):
Couple reproducing nightly tests on branch_6x, from
[https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/125/]:
{noformat}
[junit4] Suite: org.apache.lucene.search.TestDoubleRangeFieldQueries
[junit4] 2> NOTE: download the large Jenkins line-docs file by running
'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test
-Dtestcase=TestDoubleRangeFieldQueries -Dtests.method=testMultiValued
[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts
[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388442#comment-15388442 ] Mark Miller commented on SOLR-7280: --- [~mdrob] made a nice little test for this and the initial results make it look something was fishy before this or after it. Need to dig into it a little more. > Load cores in sorted order and tweak coreLoadThread counts to improve cluster > stability on restarts > --- > > Key: SOLR-7280 > URL: https://issues.apache.org/jira/browse/SOLR-7280 > Project: Solr > Issue Type: Bug > Components: SolrCloud >Reporter: Shalin Shekhar Mangar >Assignee: Noble Paul > Fix For: 6.2, 5.5.3 > > Attachments: SOLR-7280-5x.patch, SOLR-7280-5x.patch, > SOLR-7280-5x.patch, SOLR-7280.patch, SOLR-7280.patch > > > In SOLR-7191, Damien mentioned that by loading solr cores in a sorted order > and tweaking some of the coreLoadThread counts, he was able to improve the > stability of a cluster with thousands of collections. We should explore some > of these changes and fold them into Solr. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts
[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388376#comment-15388376 ] Mark Miller commented on SOLR-7280: --- bq. Plus I'll vary the order of bring the JVMs up and loop all night that way. If you simply do that and walk away and come back in the morning, it will work. The bug is that starting your shard will go from as fast as possible to 3 minutes plus just because you created too many replicas. And not only will it take 3 minutes, but all the replicas will not participate in the election as is currently designed. That's a bug, and one we have already fixed and one I'm not willing to allow back :) > Load cores in sorted order and tweak coreLoadThread counts to improve cluster > stability on restarts > --- > > Key: SOLR-7280 > URL: https://issues.apache.org/jira/browse/SOLR-7280 > Project: Solr > Issue Type: Bug > Components: SolrCloud >Reporter: Shalin Shekhar Mangar >Assignee: Noble Paul > Fix For: 6.2, 5.5.3 > > Attachments: SOLR-7280-5x.patch, SOLR-7280-5x.patch, > SOLR-7280-5x.patch, SOLR-7280.patch, SOLR-7280.patch > > > In SOLR-7191, Damien mentioned that by loading solr cores in a sorted order > and tweaking some of the coreLoadThread counts, he was able to improve the > stability of a cluster with thousands of collections. We should explore some > of these changes and fold them into Solr. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts
[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388349#comment-15388349 ] Mark Miller commented on SOLR-7280: --- If it indeed remains an issue, another thing we might try is loading cores with a limited number of threads, but registering with ZK with many more, instead of doing both in the same thread as we are now. > Load cores in sorted order and tweak coreLoadThread counts to improve cluster > stability on restarts > --- > > Key: SOLR-7280 > URL: https://issues.apache.org/jira/browse/SOLR-7280 > Project: Solr > Issue Type: Bug > Components: SolrCloud >Reporter: Shalin Shekhar Mangar >Assignee: Noble Paul > Fix For: 6.2, 5.5.3 > > Attachments: SOLR-7280-5x.patch, SOLR-7280-5x.patch, > SOLR-7280-5x.patch, SOLR-7280.patch, SOLR-7280.patch > > > In SOLR-7191, Damien mentioned that by loading solr cores in a sorted order > and tweaking some of the coreLoadThread counts, he was able to improve the > stability of a cluster with thousands of collections. We should explore some > of these changes and fold them into Solr. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 125 - Failure
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/125/ 6 tests failed. FAILED: org.apache.lucene.index.TestDuelingCodecsAtNight.testBigEquals Error Message: Java heap space Stack Trace: java.lang.OutOfMemoryError: Java heap space at __randomizedtesting.SeedInfo.seed([62829676FE994519:8186CDF35582815E]:0) at org.apache.lucene.util.fst.BytesStore.writeBytes(BytesStore.java:112) at org.apache.lucene.util.fst.ByteSequenceOutputs.write(ByteSequenceOutputs.java:120) at org.apache.lucene.util.fst.ByteSequenceOutputs.write(ByteSequenceOutputs.java:35) at org.apache.lucene.util.fst.Outputs.writeFinalOutput(Outputs.java:60) at org.apache.lucene.util.fst.FST.pack(FST.java:1714) at org.apache.lucene.util.fst.Builder.finish(Builder.java:500) at org.apache.lucene.codecs.memory.MemoryPostingsFormat$TermsWriter.finish(MemoryPostingsFormat.java:267) at org.apache.lucene.codecs.memory.MemoryPostingsFormat$MemoryFieldsConsumer.write(MemoryPostingsFormat.java:401) at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsWriter.write(PerFieldPostingsFormat.java:198) at org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:105) at org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:216) at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:101) at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4312) at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3889) at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40) at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2052) at org.apache.lucene.index.IndexWriter.doAfterSegmentFlushed(IndexWriter.java:4953) at org.apache.lucene.index.DocumentsWriter$MergePendingEvent.process(DocumentsWriter.java:742) at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4991) at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4982) at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1565) at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1307) at org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:171) at org.apache.lucene.index.TestDuelingCodecs.createRandomIndex(TestDuelingCodecs.java:139) at org.apache.lucene.index.TestDuelingCodecsAtNight.testBigEquals(TestDuelingCodecsAtNight.java:34) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871) at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907) FAILED: org.apache.lucene.search.TestDoubleRangeFieldQueries.testMultiValued Error Message: wrong hit (first of possibly more): FAIL: id=204 should not match but did queryBox=Box(Infinity TO Infinity) box=Box(Infinity TO Infinity) queryType=CONTAINS deleted?=false Stack Trace: java.lang.AssertionError: wrong hit (first of possibly more): FAIL: id=204 should not match but did queryBox=Box(Infinity TO Infinity) box=Box(Infinity TO Infinity) queryType=CONTAINS deleted?=false at __randomizedtesting.SeedInfo.seed([AE24BFDDECD216AC:7A04DBEF221056E4]:0) at org.junit.Assert.fail(Assert.java:93) at org.apache.lucene.search.BaseRangeFieldQueryTestCase.verify(BaseRangeFieldQueryTestCase.java:278) at org.apache.lucene.search.BaseRangeFieldQueryTestCase.doTestRandom(BaseRangeFieldQueryTestCase.java:154) at org.apache.lucene.search.BaseRangeFieldQueryTestCase.testMultiValued(BaseRangeFieldQueryTestCase.java:73) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871) at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907) at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921) at
[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts
[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388323#comment-15388323 ] Mark Miller commented on SOLR-7280: --- I have no problem if it works. Perhaps some other unrelated changes have influenced how the load threads interact with leader election. I just don't see the problem explained or addressed and there is a previous JIRA issue with this exact problem. The only way I can see it would still not be a problem is if waiting for a leader doesn't hold up core loading. I don't remember seeing that change though. You may not be able to test this simply with tests either - many tests lower the leaderVoteWait to very low to speed up tests. In production it should be like 3 minutes by default. > Load cores in sorted order and tweak coreLoadThread counts to improve cluster > stability on restarts > --- > > Key: SOLR-7280 > URL: https://issues.apache.org/jira/browse/SOLR-7280 > Project: Solr > Issue Type: Bug > Components: SolrCloud >Reporter: Shalin Shekhar Mangar >Assignee: Noble Paul > Fix For: 6.2, 5.5.3 > > Attachments: SOLR-7280-5x.patch, SOLR-7280-5x.patch, > SOLR-7280-5x.patch, SOLR-7280.patch, SOLR-7280.patch > > > In SOLR-7191, Damien mentioned that by loading solr cores in a sorted order > and tweaking some of the coreLoadThread counts, he was able to improve the > stability of a cluster with thousands of collections. We should explore some > of these changes and fold them into Solr. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-9200) Add Delegation Token Support to Solr
[
https://issues.apache.org/jira/browse/SOLR-9200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gregory Chanan updated SOLR-9200:
-
Attachment: SOLR-9200.patch
sorry attached the wrong patch, should be correct now.
> Add Delegation Token Support to Solr
>
>
> Key: SOLR-9200
> URL: https://issues.apache.org/jira/browse/SOLR-9200
> Project: Solr
> Issue Type: New Feature
> Components: security
>Reporter: Gregory Chanan
>Assignee: Gregory Chanan
> Attachments: SOLR-9200.patch, SOLR-9200.patch, SOLR-9200.patch,
> SOLR-9200.patch
>
>
> SOLR-7468 added support for kerberos authentication via the hadoop
> authentication filter. Hadoop also has support for an authentication filter
> that supports delegation tokens, which allow authenticated users the ability
> to grab/renew/delete a token that can be used to bypass the normal
> authentication path for a time. This is useful in a variety of use cases:
> 1) distributed clients (e.g. MapReduce) where each client may not have access
> to the user's kerberos credentials. Instead, the job runner can grab a
> delegation token and use that during task execution.
> 2) If the load on the kerberos server is too high, delegation tokens can
> avoid hitting the kerberos server after the first request
> 3) If requests/permissions need to be delegated to another user: the more
> privileged user can request a delegation token that can be passed to the less
> privileged user.
> Note to self:
> In
> https://issues.apache.org/jira/browse/SOLR-7468?focusedCommentId=14579636=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14579636
> I made the following comment which I need to investigate further, since I
> don't know if anything changed in this area:
> {quote}3) I'm a little concerned with the "NoContext" code in KerberosPlugin
> moving forward (I understand this is more a generic auth question than
> kerberos specific). For example, in the latest version of the filter we are
> using at Cloudera, we play around with the ServletContext in order to pass
> information around
> (https://github.com/cloudera/lucene-solr/blob/cdh5-4.10.3_5.4.2/solr/core/src/java/org/apache/solr/servlet/SolrHadoopAuthenticationFilter.java#L106).
> Is there any way we can get the actual ServletContext in a plugin?{quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-9200) Add Delegation Token Support to Solr
[
https://issues.apache.org/jira/browse/SOLR-9200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gregory Chanan updated SOLR-9200:
-
Attachment: SOLR-9200.patch
Here's a new version of the patch, taking into account [~ichattopadhyaya]'s
comments. In particular:
1. All security related znodes have been moved under /security
2. The existing ZkACLProviders now derived from SecurityAwareZkACLProvider,
which by default does not allow read only access to znodes under /security for
non-solr users.
3. Moved all kerberos-related test setup from KerberosTestUtil to
KerberosTestServices. This object does "all in one" setup, i.e. dealing with
non-supported locales, kdc setup, Configuration management. It also handles
resetting the Configuration at the end of the test, so other tests in the same
jvm can run successfully.
4. Got rid of extra imports.
> Add Delegation Token Support to Solr
>
>
> Key: SOLR-9200
> URL: https://issues.apache.org/jira/browse/SOLR-9200
> Project: Solr
> Issue Type: New Feature
> Components: security
>Reporter: Gregory Chanan
>Assignee: Gregory Chanan
> Attachments: SOLR-9200.patch, SOLR-9200.patch, SOLR-9200.patch,
> SOLR-9200.patch
>
>
> SOLR-7468 added support for kerberos authentication via the hadoop
> authentication filter. Hadoop also has support for an authentication filter
> that supports delegation tokens, which allow authenticated users the ability
> to grab/renew/delete a token that can be used to bypass the normal
> authentication path for a time. This is useful in a variety of use cases:
> 1) distributed clients (e.g. MapReduce) where each client may not have access
> to the user's kerberos credentials. Instead, the job runner can grab a
> delegation token and use that during task execution.
> 2) If the load on the kerberos server is too high, delegation tokens can
> avoid hitting the kerberos server after the first request
> 3) If requests/permissions need to be delegated to another user: the more
> privileged user can request a delegation token that can be passed to the less
> privileged user.
> Note to self:
> In
> https://issues.apache.org/jira/browse/SOLR-7468?focusedCommentId=14579636=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14579636
> I made the following comment which I need to investigate further, since I
> don't know if anything changed in this area:
> {quote}3) I'm a little concerned with the "NoContext" code in KerberosPlugin
> moving forward (I understand this is more a generic auth question than
> kerberos specific). For example, in the latest version of the filter we are
> using at Cloudera, we play around with the ServletContext in order to pass
> information around
> (https://github.com/cloudera/lucene-solr/blob/cdh5-4.10.3_5.4.2/solr/core/src/java/org/apache/solr/servlet/SolrHadoopAuthenticationFilter.java#L106).
> Is there any way we can get the actual ServletContext in a plugin?{quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts
[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388314#comment-15388314 ] Erick Erickson commented on SOLR-7280: -- Not in my testing, but I'll give it another whirl tonight just to be sure. I'm thinking of, say, 3 collections spread over 4 JVMS each with 3 shards each with 150 replicas and 3 coreLoadThreads Plus I'll vary the order of bring the JVMs up and loop all night that way. I've already go the infrastructure in place > Load cores in sorted order and tweak coreLoadThread counts to improve cluster > stability on restarts > --- > > Key: SOLR-7280 > URL: https://issues.apache.org/jira/browse/SOLR-7280 > Project: Solr > Issue Type: Bug > Components: SolrCloud >Reporter: Shalin Shekhar Mangar >Assignee: Noble Paul > Fix For: 6.2, 5.5.3 > > Attachments: SOLR-7280-5x.patch, SOLR-7280-5x.patch, > SOLR-7280-5x.patch, SOLR-7280.patch, SOLR-7280.patch > > > In SOLR-7191, Damien mentioned that by loading solr cores in a sorted order > and tweaking some of the coreLoadThread counts, he was able to improve the > stability of a cluster with thousands of collections. We should explore some > of these changes and fold them into Solr. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts
[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388312#comment-15388312 ] Mark Miller commented on SOLR-7280: --- Yeah, looks like I have the same issue as I mentioned. I've got to veto this commit unless that issue is addressed nicely or (the ugly fix) you just make it fail if someone tries to create a shard with more replica than load threads. > Load cores in sorted order and tweak coreLoadThread counts to improve cluster > stability on restarts > --- > > Key: SOLR-7280 > URL: https://issues.apache.org/jira/browse/SOLR-7280 > Project: Solr > Issue Type: Bug > Components: SolrCloud >Reporter: Shalin Shekhar Mangar >Assignee: Noble Paul > Fix For: 6.2, 5.5.3 > > Attachments: SOLR-7280-5x.patch, SOLR-7280-5x.patch, > SOLR-7280-5x.patch, SOLR-7280.patch, SOLR-7280.patch > > > In SOLR-7191, Damien mentioned that by loading solr cores in a sorted order > and tweaking some of the coreLoadThread counts, he was able to improve the > stability of a cluster with thousands of collections. We should explore some > of these changes and fold them into Solr. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts
[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388304#comment-15388304 ] Mark Miller commented on SOLR-7280: --- Did this just reintroduce the old bug if you do > 8 replicas or it tried to address it somehow? > Load cores in sorted order and tweak coreLoadThread counts to improve cluster > stability on restarts > --- > > Key: SOLR-7280 > URL: https://issues.apache.org/jira/browse/SOLR-7280 > Project: Solr > Issue Type: Bug > Components: SolrCloud >Reporter: Shalin Shekhar Mangar >Assignee: Noble Paul > Fix For: 6.2, 5.5.3 > > Attachments: SOLR-7280-5x.patch, SOLR-7280-5x.patch, > SOLR-7280-5x.patch, SOLR-7280.patch, SOLR-7280.patch > > > In SOLR-7191, Damien mentioned that by loading solr cores in a sorted order > and tweaking some of the coreLoadThread counts, he was able to improve the > stability of a cluster with thousands of collections. We should explore some > of these changes and fold them into Solr. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-6.x-Linux (64bit/jdk1.8.0_92) - Build # 1228 - Unstable!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Linux/1228/ Java: 64bit/jdk1.8.0_92 -XX:-UseCompressedOops -XX:+UseG1GC 1 tests failed. FAILED: org.apache.solr.cloud.CloudExitableDirectoryReaderTest.test Error Message: No live SolrServers available to handle this request Stack Trace: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request at __randomizedtesting.SeedInfo.seed([4669A532EDAEEE12:CE3D9AE8435283EA]:0) at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:381) at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1270) at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1040) at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:976) at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149) at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:942) at org.apache.solr.cloud.CloudExitableDirectoryReaderTest.assertPartialResults(CloudExitableDirectoryReaderTest.java:106) at org.apache.solr.cloud.CloudExitableDirectoryReaderTest.doTimeoutTests(CloudExitableDirectoryReaderTest.java:78) at org.apache.solr.cloud.CloudExitableDirectoryReaderTest.test(CloudExitableDirectoryReaderTest.java:56) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871) at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907) at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) at
[jira] [Commented] (SOLR-9269) Ability to create/delete/list snapshots for a solr core
[
https://issues.apache.org/jira/browse/SOLR-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388119#comment-15388119
]
ASF GitHub Bot commented on SOLR-9269:
--
Github user hgadre commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/52#discussion_r71753653
--- Diff:
solr/core/src/test/org/apache/solr/handler/TestSolrCoreSnapshots.java ---
@@ -0,0 +1,446 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.handler;
+
+import static org.apache.solr.common.cloud.ZkStateReader.BASE_URL_PROP;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.lang.invoke.MethodHandles;
+import java.net.URL;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import org.apache.commons.io.IOUtils;
+import org.apache.lucene.index.DirectoryReader;
+import org.apache.lucene.index.IndexCommit;
+import org.apache.lucene.index.IndexNotFoundException;
+import org.apache.lucene.store.SimpleFSDirectory;
+import org.apache.lucene.util.TestUtil;
+import org.apache.lucene.util.LuceneTestCase.Slow;
+import org.apache.solr.SolrTestCaseJ4;
+import org.apache.solr.client.solrj.SolrClient;
+import org.apache.solr.client.solrj.impl.CloudSolrClient;
+import org.apache.solr.client.solrj.request.CollectionAdminRequest;
+import
org.apache.solr.client.solrj.request.CoreAdminRequest.CreateSnapshot;
+import
org.apache.solr.client.solrj.request.CoreAdminRequest.DeleteSnapshot;
+import org.apache.solr.client.solrj.request.CoreAdminRequest.ListSnapshots;
+import org.apache.solr.cloud.SolrCloudTestCase;
+import org.apache.solr.common.SolrInputDocument;
+import org.apache.solr.common.cloud.DocCollection;
+import org.apache.solr.common.cloud.Replica;
+import org.apache.solr.common.cloud.Slice;
+import org.apache.solr.common.cloud.ZkStateReader;
+import org.apache.solr.common.params.CoreAdminParams.CoreAdminAction;
+import org.apache.solr.common.util.NamedList;
+import org.apache.solr.core.snapshots.SolrSnapshotManager;
+import
org.apache.solr.core.snapshots.SolrSnapshotMetaDataManager.SnapshotMetaData;
+import org.junit.AfterClass;
+import org.junit.BeforeClass;
+import org.junit.Test;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+@SolrTestCaseJ4.SuppressSSL // Currently unknown why SSL does not work
with this test
+@Slow
+public class TestSolrCoreSnapshots extends SolrCloudTestCase {
+ private static final Logger log =
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
+ private static long docsSeed; // see indexDocs()
+
+ @BeforeClass
+ public static void setupClass() throws Exception {
+useFactory("solr.StandardDirectoryFactory");
+configureCluster(1)// nodes
+.addConfig("conf1",
TEST_PATH().resolve("configsets").resolve("cloud-minimal").resolve("conf"))
+.configure();
+
+docsSeed = random().nextLong();
+ }
+
+ @AfterClass
+ public static void teardownClass() throws Exception {
+System.clearProperty("test.build.data");
+System.clearProperty("test.cache.data");
+ }
+
+ @Test
+ public void testBackupRestore() throws Exception {
+CloudSolrClient solrClient = cluster.getSolrClient();
+String collectionName = "SolrCoreSnapshots";
+CollectionAdminRequest.Create create =
CollectionAdminRequest.createCollection(collectionName, "conf1", 1, 1);
+create.process(solrClient);
+
+String location = createTempDir().toFile().getAbsolutePath();
+int nDocs = BackupRestoreUtils.indexDocs(cluster.getSolrClient(),
collectionName, docsSeed);
+
+
[GitHub] lucene-solr pull request #52: [SOLR-9269] Ability to create/delete/list snap...
Github user hgadre commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/52#discussion_r71753653
--- Diff:
solr/core/src/test/org/apache/solr/handler/TestSolrCoreSnapshots.java ---
@@ -0,0 +1,446 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.handler;
+
+import static org.apache.solr.common.cloud.ZkStateReader.BASE_URL_PROP;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.lang.invoke.MethodHandles;
+import java.net.URL;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import org.apache.commons.io.IOUtils;
+import org.apache.lucene.index.DirectoryReader;
+import org.apache.lucene.index.IndexCommit;
+import org.apache.lucene.index.IndexNotFoundException;
+import org.apache.lucene.store.SimpleFSDirectory;
+import org.apache.lucene.util.TestUtil;
+import org.apache.lucene.util.LuceneTestCase.Slow;
+import org.apache.solr.SolrTestCaseJ4;
+import org.apache.solr.client.solrj.SolrClient;
+import org.apache.solr.client.solrj.impl.CloudSolrClient;
+import org.apache.solr.client.solrj.request.CollectionAdminRequest;
+import
org.apache.solr.client.solrj.request.CoreAdminRequest.CreateSnapshot;
+import
org.apache.solr.client.solrj.request.CoreAdminRequest.DeleteSnapshot;
+import org.apache.solr.client.solrj.request.CoreAdminRequest.ListSnapshots;
+import org.apache.solr.cloud.SolrCloudTestCase;
+import org.apache.solr.common.SolrInputDocument;
+import org.apache.solr.common.cloud.DocCollection;
+import org.apache.solr.common.cloud.Replica;
+import org.apache.solr.common.cloud.Slice;
+import org.apache.solr.common.cloud.ZkStateReader;
+import org.apache.solr.common.params.CoreAdminParams.CoreAdminAction;
+import org.apache.solr.common.util.NamedList;
+import org.apache.solr.core.snapshots.SolrSnapshotManager;
+import
org.apache.solr.core.snapshots.SolrSnapshotMetaDataManager.SnapshotMetaData;
+import org.junit.AfterClass;
+import org.junit.BeforeClass;
+import org.junit.Test;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+@SolrTestCaseJ4.SuppressSSL // Currently unknown why SSL does not work
with this test
+@Slow
+public class TestSolrCoreSnapshots extends SolrCloudTestCase {
+ private static final Logger log =
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
+ private static long docsSeed; // see indexDocs()
+
+ @BeforeClass
+ public static void setupClass() throws Exception {
+useFactory("solr.StandardDirectoryFactory");
+configureCluster(1)// nodes
+.addConfig("conf1",
TEST_PATH().resolve("configsets").resolve("cloud-minimal").resolve("conf"))
+.configure();
+
+docsSeed = random().nextLong();
+ }
+
+ @AfterClass
+ public static void teardownClass() throws Exception {
+System.clearProperty("test.build.data");
+System.clearProperty("test.cache.data");
+ }
+
+ @Test
+ public void testBackupRestore() throws Exception {
+CloudSolrClient solrClient = cluster.getSolrClient();
+String collectionName = "SolrCoreSnapshots";
+CollectionAdminRequest.Create create =
CollectionAdminRequest.createCollection(collectionName, "conf1", 1, 1);
+create.process(solrClient);
+
+String location = createTempDir().toFile().getAbsolutePath();
+int nDocs = BackupRestoreUtils.indexDocs(cluster.getSolrClient(),
collectionName, docsSeed);
+
+DocCollection collectionState =
solrClient.getZkStateReader().getClusterState().getCollection(collectionName);
+assertEquals(1, collectionState.getActiveSlices().size());
+Slice shard =
[jira] [Commented] (SOLR-9269) Ability to create/delete/list snapshots for a solr core
[
https://issues.apache.org/jira/browse/SOLR-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388084#comment-15388084
]
ASF GitHub Bot commented on SOLR-9269:
--
Github user hgadre commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/52#discussion_r71750235
--- Diff:
solr/core/src/java/org/apache/solr/core/snapshots/SolrSnapshotMetaDataManager.java
---
@@ -0,0 +1,419 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.core.snapshots;
+
+import java.io.IOException;
+import java.lang.invoke.MethodHandles;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Optional;
+import java.util.stream.Collectors;
+
+import org.apache.lucene.codecs.CodecUtil;
+import org.apache.lucene.index.DirectoryReader;
+import org.apache.lucene.index.IndexCommit;
+import org.apache.lucene.index.IndexDeletionPolicy;
+import org.apache.lucene.index.IndexWriterConfig.OpenMode;
+import org.apache.lucene.store.Directory;
+import org.apache.lucene.store.IOContext;
+import org.apache.lucene.store.IndexInput;
+import org.apache.lucene.store.IndexOutput;
+import org.apache.lucene.util.IOUtils;
+import org.apache.solr.common.SolrException;
+import org.apache.solr.common.SolrException.ErrorCode;
+import org.apache.solr.core.DirectoryFactory;
+import org.apache.solr.core.IndexDeletionPolicyWrapper;
+import org.apache.solr.core.SolrCore;
+import org.apache.solr.core.DirectoryFactory.DirContext;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This class is responsible to manage the persistent snapshots meta-data
for the Solr indexes. The
+ * persistent snapshots are implemented by relying on Lucene {@linkplain
IndexDeletionPolicy}
+ * abstraction to configure a specific {@linkplain IndexCommit} to be
retained. The
+ * {@linkplain IndexDeletionPolicyWrapper} in Solr uses this class to
create/delete the Solr index
+ * snapshots.
+ */
+public class SolrSnapshotMetaDataManager {
+ private static final Logger log =
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
+ public static final String SNAPSHOT_METADATA_DIR = "snapshot_metadata";
+
+ /**
+ * A class defining the meta-data for a specific snapshot.
+ */
+ public static class SnapshotMetaData {
+private String name;
+private String indexDirPath;
+private long generationNumber;
+
+public SnapshotMetaData(String name, String indexDirPath, long
generationNumber) {
+ super();
+ this.name = name;
+ this.indexDirPath = indexDirPath;
+ this.generationNumber = generationNumber;
+}
+
+public String getName() {
+ return name;
+}
+
+public String getIndexDirPath() {
+ return indexDirPath;
+}
+
+public long getGenerationNumber() {
+ return generationNumber;
+}
+
+@Override
+public String toString() {
+ StringBuilder builder = new StringBuilder();
+ builder.append("SnapshotMetaData[name=");
+ builder.append(name);
+ builder.append(", indexDirPath=");
+ builder.append(indexDirPath);
+ builder.append(", generation=");
+ builder.append(generationNumber);
+ builder.append("]");
+ return builder.toString();
+}
+ }
+
+ /** Prefix used for the save file. */
+ public static final String SNAPSHOTS_PREFIX = "snapshots_";
+ private static final int VERSION_START = 0;
+ private static final int VERSION_CURRENT = VERSION_START;
+ private static
[jira] [Commented] (SOLR-9269) Ability to create/delete/list snapshots for a solr core
[
https://issues.apache.org/jira/browse/SOLR-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388085#comment-15388085
]
ASF GitHub Bot commented on SOLR-9269:
--
Github user hgadre commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/52#discussion_r71750261
--- Diff:
solr/core/src/java/org/apache/solr/core/snapshots/SolrSnapshotMetaDataManager.java
---
@@ -0,0 +1,419 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.core.snapshots;
+
+import java.io.IOException;
+import java.lang.invoke.MethodHandles;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Optional;
+import java.util.stream.Collectors;
+
+import org.apache.lucene.codecs.CodecUtil;
+import org.apache.lucene.index.DirectoryReader;
+import org.apache.lucene.index.IndexCommit;
+import org.apache.lucene.index.IndexDeletionPolicy;
+import org.apache.lucene.index.IndexWriterConfig.OpenMode;
+import org.apache.lucene.store.Directory;
+import org.apache.lucene.store.IOContext;
+import org.apache.lucene.store.IndexInput;
+import org.apache.lucene.store.IndexOutput;
+import org.apache.lucene.util.IOUtils;
+import org.apache.solr.common.SolrException;
+import org.apache.solr.common.SolrException.ErrorCode;
+import org.apache.solr.core.DirectoryFactory;
+import org.apache.solr.core.IndexDeletionPolicyWrapper;
+import org.apache.solr.core.SolrCore;
+import org.apache.solr.core.DirectoryFactory.DirContext;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This class is responsible to manage the persistent snapshots meta-data
for the Solr indexes. The
+ * persistent snapshots are implemented by relying on Lucene {@linkplain
IndexDeletionPolicy}
+ * abstraction to configure a specific {@linkplain IndexCommit} to be
retained. The
+ * {@linkplain IndexDeletionPolicyWrapper} in Solr uses this class to
create/delete the Solr index
+ * snapshots.
+ */
+public class SolrSnapshotMetaDataManager {
+ private static final Logger log =
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
+ public static final String SNAPSHOT_METADATA_DIR = "snapshot_metadata";
+
+ /**
+ * A class defining the meta-data for a specific snapshot.
+ */
+ public static class SnapshotMetaData {
+private String name;
+private String indexDirPath;
+private long generationNumber;
+
+public SnapshotMetaData(String name, String indexDirPath, long
generationNumber) {
+ super();
+ this.name = name;
+ this.indexDirPath = indexDirPath;
+ this.generationNumber = generationNumber;
+}
+
+public String getName() {
+ return name;
+}
+
+public String getIndexDirPath() {
+ return indexDirPath;
+}
+
+public long getGenerationNumber() {
+ return generationNumber;
+}
+
+@Override
+public String toString() {
+ StringBuilder builder = new StringBuilder();
+ builder.append("SnapshotMetaData[name=");
+ builder.append(name);
+ builder.append(", indexDirPath=");
+ builder.append(indexDirPath);
+ builder.append(", generation=");
+ builder.append(generationNumber);
+ builder.append("]");
+ return builder.toString();
+}
+ }
+
+ /** Prefix used for the save file. */
+ public static final String SNAPSHOTS_PREFIX = "snapshots_";
+ private static final int VERSION_START = 0;
+ private static final int VERSION_CURRENT = VERSION_START;
+ private static
[GitHub] lucene-solr pull request #52: [SOLR-9269] Ability to create/delete/list snap...
Github user hgadre commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/52#discussion_r71750261
--- Diff:
solr/core/src/java/org/apache/solr/core/snapshots/SolrSnapshotMetaDataManager.java
---
@@ -0,0 +1,419 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.core.snapshots;
+
+import java.io.IOException;
+import java.lang.invoke.MethodHandles;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Optional;
+import java.util.stream.Collectors;
+
+import org.apache.lucene.codecs.CodecUtil;
+import org.apache.lucene.index.DirectoryReader;
+import org.apache.lucene.index.IndexCommit;
+import org.apache.lucene.index.IndexDeletionPolicy;
+import org.apache.lucene.index.IndexWriterConfig.OpenMode;
+import org.apache.lucene.store.Directory;
+import org.apache.lucene.store.IOContext;
+import org.apache.lucene.store.IndexInput;
+import org.apache.lucene.store.IndexOutput;
+import org.apache.lucene.util.IOUtils;
+import org.apache.solr.common.SolrException;
+import org.apache.solr.common.SolrException.ErrorCode;
+import org.apache.solr.core.DirectoryFactory;
+import org.apache.solr.core.IndexDeletionPolicyWrapper;
+import org.apache.solr.core.SolrCore;
+import org.apache.solr.core.DirectoryFactory.DirContext;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This class is responsible to manage the persistent snapshots meta-data
for the Solr indexes. The
+ * persistent snapshots are implemented by relying on Lucene {@linkplain
IndexDeletionPolicy}
+ * abstraction to configure a specific {@linkplain IndexCommit} to be
retained. The
+ * {@linkplain IndexDeletionPolicyWrapper} in Solr uses this class to
create/delete the Solr index
+ * snapshots.
+ */
+public class SolrSnapshotMetaDataManager {
+ private static final Logger log =
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
+ public static final String SNAPSHOT_METADATA_DIR = "snapshot_metadata";
+
+ /**
+ * A class defining the meta-data for a specific snapshot.
+ */
+ public static class SnapshotMetaData {
+private String name;
+private String indexDirPath;
+private long generationNumber;
+
+public SnapshotMetaData(String name, String indexDirPath, long
generationNumber) {
+ super();
+ this.name = name;
+ this.indexDirPath = indexDirPath;
+ this.generationNumber = generationNumber;
+}
+
+public String getName() {
+ return name;
+}
+
+public String getIndexDirPath() {
+ return indexDirPath;
+}
+
+public long getGenerationNumber() {
+ return generationNumber;
+}
+
+@Override
+public String toString() {
+ StringBuilder builder = new StringBuilder();
+ builder.append("SnapshotMetaData[name=");
+ builder.append(name);
+ builder.append(", indexDirPath=");
+ builder.append(indexDirPath);
+ builder.append(", generation=");
+ builder.append(generationNumber);
+ builder.append("]");
+ return builder.toString();
+}
+ }
+
+ /** Prefix used for the save file. */
+ public static final String SNAPSHOTS_PREFIX = "snapshots_";
+ private static final int VERSION_START = 0;
+ private static final int VERSION_CURRENT = VERSION_START;
+ private static final String CODEC_NAME = "solr-snapshots";
+
+ // The index writer which maintains the snapshots metadata
+ private long nextWriteGen;
+
+ private final Directory dir;
+
+ /** Used to map snapshot name to
[GitHub] lucene-solr pull request #52: [SOLR-9269] Ability to create/delete/list snap...
Github user hgadre commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/52#discussion_r71750235
--- Diff:
solr/core/src/java/org/apache/solr/core/snapshots/SolrSnapshotMetaDataManager.java
---
@@ -0,0 +1,419 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.core.snapshots;
+
+import java.io.IOException;
+import java.lang.invoke.MethodHandles;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Optional;
+import java.util.stream.Collectors;
+
+import org.apache.lucene.codecs.CodecUtil;
+import org.apache.lucene.index.DirectoryReader;
+import org.apache.lucene.index.IndexCommit;
+import org.apache.lucene.index.IndexDeletionPolicy;
+import org.apache.lucene.index.IndexWriterConfig.OpenMode;
+import org.apache.lucene.store.Directory;
+import org.apache.lucene.store.IOContext;
+import org.apache.lucene.store.IndexInput;
+import org.apache.lucene.store.IndexOutput;
+import org.apache.lucene.util.IOUtils;
+import org.apache.solr.common.SolrException;
+import org.apache.solr.common.SolrException.ErrorCode;
+import org.apache.solr.core.DirectoryFactory;
+import org.apache.solr.core.IndexDeletionPolicyWrapper;
+import org.apache.solr.core.SolrCore;
+import org.apache.solr.core.DirectoryFactory.DirContext;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This class is responsible to manage the persistent snapshots meta-data
for the Solr indexes. The
+ * persistent snapshots are implemented by relying on Lucene {@linkplain
IndexDeletionPolicy}
+ * abstraction to configure a specific {@linkplain IndexCommit} to be
retained. The
+ * {@linkplain IndexDeletionPolicyWrapper} in Solr uses this class to
create/delete the Solr index
+ * snapshots.
+ */
+public class SolrSnapshotMetaDataManager {
+ private static final Logger log =
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
+ public static final String SNAPSHOT_METADATA_DIR = "snapshot_metadata";
+
+ /**
+ * A class defining the meta-data for a specific snapshot.
+ */
+ public static class SnapshotMetaData {
+private String name;
+private String indexDirPath;
+private long generationNumber;
+
+public SnapshotMetaData(String name, String indexDirPath, long
generationNumber) {
+ super();
+ this.name = name;
+ this.indexDirPath = indexDirPath;
+ this.generationNumber = generationNumber;
+}
+
+public String getName() {
+ return name;
+}
+
+public String getIndexDirPath() {
+ return indexDirPath;
+}
+
+public long getGenerationNumber() {
+ return generationNumber;
+}
+
+@Override
+public String toString() {
+ StringBuilder builder = new StringBuilder();
+ builder.append("SnapshotMetaData[name=");
+ builder.append(name);
+ builder.append(", indexDirPath=");
+ builder.append(indexDirPath);
+ builder.append(", generation=");
+ builder.append(generationNumber);
+ builder.append("]");
+ return builder.toString();
+}
+ }
+
+ /** Prefix used for the save file. */
+ public static final String SNAPSHOTS_PREFIX = "snapshots_";
+ private static final int VERSION_START = 0;
+ private static final int VERSION_CURRENT = VERSION_START;
+ private static final String CODEC_NAME = "solr-snapshots";
+
+ // The index writer which maintains the snapshots metadata
+ private long nextWriteGen;
+
+ private final Directory dir;
+
+ /** Used to map snapshot name to
[jira] [Commented] (SOLR-9269) Ability to create/delete/list snapshots for a solr core
[
https://issues.apache.org/jira/browse/SOLR-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388073#comment-15388073
]
ASF GitHub Bot commented on SOLR-9269:
--
Github user hgadre commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/52#discussion_r71748418
--- Diff:
solr/core/src/java/org/apache/solr/core/snapshots/SolrSnapshotMetaDataManager.java
---
@@ -0,0 +1,419 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.core.snapshots;
+
+import java.io.IOException;
+import java.lang.invoke.MethodHandles;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Optional;
+import java.util.stream.Collectors;
+
+import org.apache.lucene.codecs.CodecUtil;
+import org.apache.lucene.index.DirectoryReader;
+import org.apache.lucene.index.IndexCommit;
+import org.apache.lucene.index.IndexDeletionPolicy;
+import org.apache.lucene.index.IndexWriterConfig.OpenMode;
+import org.apache.lucene.store.Directory;
+import org.apache.lucene.store.IOContext;
+import org.apache.lucene.store.IndexInput;
+import org.apache.lucene.store.IndexOutput;
+import org.apache.lucene.util.IOUtils;
+import org.apache.solr.common.SolrException;
+import org.apache.solr.common.SolrException.ErrorCode;
+import org.apache.solr.core.DirectoryFactory;
+import org.apache.solr.core.IndexDeletionPolicyWrapper;
+import org.apache.solr.core.SolrCore;
+import org.apache.solr.core.DirectoryFactory.DirContext;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This class is responsible to manage the persistent snapshots meta-data
for the Solr indexes. The
+ * persistent snapshots are implemented by relying on Lucene {@linkplain
IndexDeletionPolicy}
+ * abstraction to configure a specific {@linkplain IndexCommit} to be
retained. The
+ * {@linkplain IndexDeletionPolicyWrapper} in Solr uses this class to
create/delete the Solr index
+ * snapshots.
+ */
+public class SolrSnapshotMetaDataManager {
+ private static final Logger log =
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
+ public static final String SNAPSHOT_METADATA_DIR = "snapshot_metadata";
+
+ /**
+ * A class defining the meta-data for a specific snapshot.
+ */
+ public static class SnapshotMetaData {
+private String name;
+private String indexDirPath;
+private long generationNumber;
+
+public SnapshotMetaData(String name, String indexDirPath, long
generationNumber) {
+ super();
+ this.name = name;
+ this.indexDirPath = indexDirPath;
+ this.generationNumber = generationNumber;
+}
+
+public String getName() {
+ return name;
+}
+
+public String getIndexDirPath() {
+ return indexDirPath;
+}
+
+public long getGenerationNumber() {
+ return generationNumber;
+}
+
+@Override
+public String toString() {
+ StringBuilder builder = new StringBuilder();
+ builder.append("SnapshotMetaData[name=");
+ builder.append(name);
+ builder.append(", indexDirPath=");
+ builder.append(indexDirPath);
+ builder.append(", generation=");
+ builder.append(generationNumber);
+ builder.append("]");
+ return builder.toString();
+}
+ }
+
+ /** Prefix used for the save file. */
+ public static final String SNAPSHOTS_PREFIX = "snapshots_";
+ private static final int VERSION_START = 0;
+ private static final int VERSION_CURRENT = VERSION_START;
+ private static
[GitHub] lucene-solr pull request #52: [SOLR-9269] Ability to create/delete/list snap...
Github user hgadre commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/52#discussion_r71748418
--- Diff:
solr/core/src/java/org/apache/solr/core/snapshots/SolrSnapshotMetaDataManager.java
---
@@ -0,0 +1,419 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.core.snapshots;
+
+import java.io.IOException;
+import java.lang.invoke.MethodHandles;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Optional;
+import java.util.stream.Collectors;
+
+import org.apache.lucene.codecs.CodecUtil;
+import org.apache.lucene.index.DirectoryReader;
+import org.apache.lucene.index.IndexCommit;
+import org.apache.lucene.index.IndexDeletionPolicy;
+import org.apache.lucene.index.IndexWriterConfig.OpenMode;
+import org.apache.lucene.store.Directory;
+import org.apache.lucene.store.IOContext;
+import org.apache.lucene.store.IndexInput;
+import org.apache.lucene.store.IndexOutput;
+import org.apache.lucene.util.IOUtils;
+import org.apache.solr.common.SolrException;
+import org.apache.solr.common.SolrException.ErrorCode;
+import org.apache.solr.core.DirectoryFactory;
+import org.apache.solr.core.IndexDeletionPolicyWrapper;
+import org.apache.solr.core.SolrCore;
+import org.apache.solr.core.DirectoryFactory.DirContext;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This class is responsible to manage the persistent snapshots meta-data
for the Solr indexes. The
+ * persistent snapshots are implemented by relying on Lucene {@linkplain
IndexDeletionPolicy}
+ * abstraction to configure a specific {@linkplain IndexCommit} to be
retained. The
+ * {@linkplain IndexDeletionPolicyWrapper} in Solr uses this class to
create/delete the Solr index
+ * snapshots.
+ */
+public class SolrSnapshotMetaDataManager {
+ private static final Logger log =
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
+ public static final String SNAPSHOT_METADATA_DIR = "snapshot_metadata";
+
+ /**
+ * A class defining the meta-data for a specific snapshot.
+ */
+ public static class SnapshotMetaData {
+private String name;
+private String indexDirPath;
+private long generationNumber;
+
+public SnapshotMetaData(String name, String indexDirPath, long
generationNumber) {
+ super();
+ this.name = name;
+ this.indexDirPath = indexDirPath;
+ this.generationNumber = generationNumber;
+}
+
+public String getName() {
+ return name;
+}
+
+public String getIndexDirPath() {
+ return indexDirPath;
+}
+
+public long getGenerationNumber() {
+ return generationNumber;
+}
+
+@Override
+public String toString() {
+ StringBuilder builder = new StringBuilder();
+ builder.append("SnapshotMetaData[name=");
+ builder.append(name);
+ builder.append(", indexDirPath=");
+ builder.append(indexDirPath);
+ builder.append(", generation=");
+ builder.append(generationNumber);
+ builder.append("]");
+ return builder.toString();
+}
+ }
+
+ /** Prefix used for the save file. */
+ public static final String SNAPSHOTS_PREFIX = "snapshots_";
+ private static final int VERSION_START = 0;
+ private static final int VERSION_CURRENT = VERSION_START;
+ private static final String CODEC_NAME = "solr-snapshots";
+
+ // The index writer which maintains the snapshots metadata
+ private long nextWriteGen;
+
+ private final Directory dir;
+
+ /** Used to map snapshot name to
[jira] [Commented] (SOLR-9269) Ability to create/delete/list snapshots for a solr core
[
https://issues.apache.org/jira/browse/SOLR-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388071#comment-15388071
]
ASF GitHub Bot commented on SOLR-9269:
--
Github user hgadre commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/52#discussion_r71748142
--- Diff:
solr/core/src/java/org/apache/solr/handler/admin/CoreAdminOperation.java ---
@@ -919,7 +928,93 @@ public void call(CallInfo callInfo) throws Exception {
}
}
}
+ },
+ CREATESNAPSHOT_OP(CREATESNAPSHOT) {
+@Override
+public void call(CallInfo callInfo) throws Exception {
+ CoreContainer cc = callInfo.handler.getCoreContainer();
+ final SolrParams params = callInfo.req.getParams();
+
+ String commitName =
params.required().get(CoreAdminParams.COMMIT_NAME);
+ String cname = params.required().get(CoreAdminParams.CORE);
+ try (SolrCore core = cc.getCore(cname)) {
+if (core == null) {
+ throw new SolrException(ErrorCode.SERVER_ERROR, "Unable to
locate core " + cname);
+}
+if (core.getDeletionPolicy().getLatestCommit() == null) {
+ throw new SolrException(ErrorCode.SERVER_ERROR, "Could not find
latest commit. Please ensure to execute a hard commit");
+}
+
+String indexDirPath = core.getIndexDir();
+IndexCommit ic = core.getDeletionPolicy().getLatestCommit();
+SolrSnapshotMetaDataManager mgr =
core.getSnapshotMetaDataManager();
+mgr.snapshot(commitName, indexDirPath, ic.getGeneration());
+ }
+}
+ },
+ DELETESNAPSHOT_OP(DELETESNAPSHOT) {
+@Override
+public void call(CallInfo callInfo) throws Exception {
+ CoreContainer cc = callInfo.handler.getCoreContainer();
+ final SolrParams params = callInfo.req.getParams();
+
+ String commitName =
params.required().get(CoreAdminParams.COMMIT_NAME);
+ String cname = params.required().get(CoreAdminParams.CORE);
+ try (SolrCore core = cc.getCore(cname)) {
+if (core == null) {
+ throw new SolrException(ErrorCode.SERVER_ERROR, "Unable to
locate core " + cname);
+}
+
+SolrSnapshotMetaDataManager mgr =
core.getSnapshotMetaDataManager();
+Optional metadata = mgr.release(commitName);
+if (metadata.isPresent()) {
--- End diff --
OK I am using BAD_REQUEST for parameter related errors. I don't think we
should throw error if the commit is not found since this is in-line with the
user expectation (which is Solr should not have a snapshot with the specified
name).
> Ability to create/delete/list snapshots for a solr core
> ---
>
> Key: SOLR-9269
> URL: https://issues.apache.org/jira/browse/SOLR-9269
> Project: Solr
> Issue Type: Sub-task
> Components: SolrCloud
>Reporter: Hrishikesh Gadre
>Assignee: David Smiley
> Attachments: SOLR-9269.patch
>
>
> Support snapshot create/delete/list functionality @ the Solr core level.
> Please refer to parent JIRA for more details.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] lucene-solr pull request #52: [SOLR-9269] Ability to create/delete/list snap...
Github user hgadre commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/52#discussion_r71748142
--- Diff:
solr/core/src/java/org/apache/solr/handler/admin/CoreAdminOperation.java ---
@@ -919,7 +928,93 @@ public void call(CallInfo callInfo) throws Exception {
}
}
}
+ },
+ CREATESNAPSHOT_OP(CREATESNAPSHOT) {
+@Override
+public void call(CallInfo callInfo) throws Exception {
+ CoreContainer cc = callInfo.handler.getCoreContainer();
+ final SolrParams params = callInfo.req.getParams();
+
+ String commitName =
params.required().get(CoreAdminParams.COMMIT_NAME);
+ String cname = params.required().get(CoreAdminParams.CORE);
+ try (SolrCore core = cc.getCore(cname)) {
+if (core == null) {
+ throw new SolrException(ErrorCode.SERVER_ERROR, "Unable to
locate core " + cname);
+}
+if (core.getDeletionPolicy().getLatestCommit() == null) {
+ throw new SolrException(ErrorCode.SERVER_ERROR, "Could not find
latest commit. Please ensure to execute a hard commit");
+}
+
+String indexDirPath = core.getIndexDir();
+IndexCommit ic = core.getDeletionPolicy().getLatestCommit();
+SolrSnapshotMetaDataManager mgr =
core.getSnapshotMetaDataManager();
+mgr.snapshot(commitName, indexDirPath, ic.getGeneration());
+ }
+}
+ },
+ DELETESNAPSHOT_OP(DELETESNAPSHOT) {
+@Override
+public void call(CallInfo callInfo) throws Exception {
+ CoreContainer cc = callInfo.handler.getCoreContainer();
+ final SolrParams params = callInfo.req.getParams();
+
+ String commitName =
params.required().get(CoreAdminParams.COMMIT_NAME);
+ String cname = params.required().get(CoreAdminParams.CORE);
+ try (SolrCore core = cc.getCore(cname)) {
+if (core == null) {
+ throw new SolrException(ErrorCode.SERVER_ERROR, "Unable to
locate core " + cname);
+}
+
+SolrSnapshotMetaDataManager mgr =
core.getSnapshotMetaDataManager();
+Optional metadata = mgr.release(commitName);
+if (metadata.isPresent()) {
--- End diff --
OK I am using BAD_REQUEST for parameter related errors. I don't think we
should throw error if the commit is not found since this is in-line with the
user expectation (which is Solr should not have a snapshot with the specified
name).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] lucene-solr pull request #52: [SOLR-9269] Ability to create/delete/list snap...
Github user hgadre commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/52#discussion_r71747583
--- Diff:
solr/core/src/java/org/apache/solr/handler/admin/CoreAdminOperation.java ---
@@ -919,7 +928,93 @@ public void call(CallInfo callInfo) throws Exception {
}
}
}
+ },
+ CREATESNAPSHOT_OP(CREATESNAPSHOT) {
+@Override
+public void call(CallInfo callInfo) throws Exception {
+ CoreContainer cc = callInfo.handler.getCoreContainer();
+ final SolrParams params = callInfo.req.getParams();
+
+ String commitName =
params.required().get(CoreAdminParams.COMMIT_NAME);
+ String cname = params.required().get(CoreAdminParams.CORE);
+ try (SolrCore core = cc.getCore(cname)) {
+if (core == null) {
+ throw new SolrException(ErrorCode.SERVER_ERROR, "Unable to
locate core " + cname);
+}
+if (core.getDeletionPolicy().getLatestCommit() == null) {
--- End diff --
I see that ReplicationHandler is implemented differently. If the
latestCommit is null, it uses the commit opened by the index searcher. Now I am
using the same logic here, assuming the index searcher will always return
non-null index commit.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9269) Ability to create/delete/list snapshots for a solr core
[
https://issues.apache.org/jira/browse/SOLR-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388066#comment-15388066
]
ASF GitHub Bot commented on SOLR-9269:
--
Github user hgadre commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/52#discussion_r71747583
--- Diff:
solr/core/src/java/org/apache/solr/handler/admin/CoreAdminOperation.java ---
@@ -919,7 +928,93 @@ public void call(CallInfo callInfo) throws Exception {
}
}
}
+ },
+ CREATESNAPSHOT_OP(CREATESNAPSHOT) {
+@Override
+public void call(CallInfo callInfo) throws Exception {
+ CoreContainer cc = callInfo.handler.getCoreContainer();
+ final SolrParams params = callInfo.req.getParams();
+
+ String commitName =
params.required().get(CoreAdminParams.COMMIT_NAME);
+ String cname = params.required().get(CoreAdminParams.CORE);
+ try (SolrCore core = cc.getCore(cname)) {
+if (core == null) {
+ throw new SolrException(ErrorCode.SERVER_ERROR, "Unable to
locate core " + cname);
+}
+if (core.getDeletionPolicy().getLatestCommit() == null) {
--- End diff --
I see that ReplicationHandler is implemented differently. If the
latestCommit is null, it uses the commit opened by the index searcher. Now I am
using the same logic here, assuming the index searcher will always return
non-null index commit.
> Ability to create/delete/list snapshots for a solr core
> ---
>
> Key: SOLR-9269
> URL: https://issues.apache.org/jira/browse/SOLR-9269
> Project: Solr
> Issue Type: Sub-task
> Components: SolrCloud
>Reporter: Hrishikesh Gadre
>Assignee: David Smiley
> Attachments: SOLR-9269.patch
>
>
> Support snapshot create/delete/list functionality @ the Solr core level.
> Please refer to parent JIRA for more details.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9269) Ability to create/delete/list snapshots for a solr core
[
https://issues.apache.org/jira/browse/SOLR-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388057#comment-15388057
]
ASF GitHub Bot commented on SOLR-9269:
--
Github user hgadre commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/52#discussion_r71746146
--- Diff:
solr/core/src/java/org/apache/solr/handler/ReplicationHandler.java ---
@@ -512,11 +513,24 @@ private void doSnapShoot(SolrParams params,
SolrQueryResponse rsp,
numberToKeep = Integer.MAX_VALUE;
}
- IndexDeletionPolicyWrapper delPolicy = core.getDeletionPolicy();
- IndexCommit indexCommit = delPolicy.getLatestCommit();
+ IndexCommit indexCommit = null;
+ String commitName = params.get(CoreAdminParams.COMMIT_NAME);
+ if (commitName != null) {
+SolrSnapshotMetaDataManager snapshotMgr =
core.getSnapshotMetaDataManager();
+Optional commit =
snapshotMgr.getIndexCommitByName(commitName);
+if(commit.isPresent()) {
+ indexCommit = commit.get();
+} else {
+ throw new SolrException(ErrorCode.SERVER_ERROR, "Unable to find
an index commit with name " + commitName +
--- End diff --
Done.
> Ability to create/delete/list snapshots for a solr core
> ---
>
> Key: SOLR-9269
> URL: https://issues.apache.org/jira/browse/SOLR-9269
> Project: Solr
> Issue Type: Sub-task
> Components: SolrCloud
>Reporter: Hrishikesh Gadre
>Assignee: David Smiley
> Attachments: SOLR-9269.patch
>
>
> Support snapshot create/delete/list functionality @ the Solr core level.
> Please refer to parent JIRA for more details.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] lucene-solr pull request #52: [SOLR-9269] Ability to create/delete/list snap...
Github user hgadre commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/52#discussion_r71746146
--- Diff:
solr/core/src/java/org/apache/solr/handler/ReplicationHandler.java ---
@@ -512,11 +513,24 @@ private void doSnapShoot(SolrParams params,
SolrQueryResponse rsp,
numberToKeep = Integer.MAX_VALUE;
}
- IndexDeletionPolicyWrapper delPolicy = core.getDeletionPolicy();
- IndexCommit indexCommit = delPolicy.getLatestCommit();
+ IndexCommit indexCommit = null;
+ String commitName = params.get(CoreAdminParams.COMMIT_NAME);
+ if (commitName != null) {
+SolrSnapshotMetaDataManager snapshotMgr =
core.getSnapshotMetaDataManager();
+Optional commit =
snapshotMgr.getIndexCommitByName(commitName);
+if(commit.isPresent()) {
+ indexCommit = commit.get();
+} else {
+ throw new SolrException(ErrorCode.SERVER_ERROR, "Unable to find
an index commit with name " + commitName +
--- End diff --
Done.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS-EA] Lucene-Solr-master-Linux (64bit/jdk-9-ea+127) - Build # 17322 - Unstable!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/17322/ Java: 64bit/jdk-9-ea+127 -XX:-UseCompressedOops -XX:+UseParallelGC 1 tests failed. FAILED: org.apache.solr.security.BasicAuthIntegrationTest.testBasics Error Message: IOException occured when talking to server at: http://127.0.0.1:39436/solr/testSolrCloudCollection_shard1_replica2 Stack Trace: org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: IOException occured when talking to server at: http://127.0.0.1:39436/solr/testSolrCloudCollection_shard1_replica2 at __randomizedtesting.SeedInfo.seed([18704C9B0A550BEB:25A8E2B732BB559B]:0) at org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:739) at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1151) at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1040) at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:976) at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219) at org.apache.solr.security.BasicAuthIntegrationTest.doExtraTests(BasicAuthIntegrationTest.java:193) at org.apache.solr.cloud.TestMiniSolrCloudClusterBase.testCollectionCreateSearchDelete(TestMiniSolrCloudClusterBase.java:196) at org.apache.solr.cloud.TestMiniSolrCloudClusterBase.testBasics(TestMiniSolrCloudClusterBase.java:79) at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@9-ea/Native Method) at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@9-ea/NativeMethodAccessorImpl.java:62) at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@9-ea/DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(java.base@9-ea/Method.java:533) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871) at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907) at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at
[GitHub] lucene-solr pull request #52: [SOLR-9269] Ability to create/delete/list snap...
Github user hgadre commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/52#discussion_r71745628
--- Diff:
solr/core/src/java/org/apache/solr/core/snapshots/SolrSnapshotMetaDataManager.java
---
@@ -0,0 +1,419 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.core.snapshots;
+
+import java.io.IOException;
+import java.lang.invoke.MethodHandles;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Optional;
+import java.util.stream.Collectors;
+
+import org.apache.lucene.codecs.CodecUtil;
+import org.apache.lucene.index.DirectoryReader;
+import org.apache.lucene.index.IndexCommit;
+import org.apache.lucene.index.IndexDeletionPolicy;
+import org.apache.lucene.index.IndexWriterConfig.OpenMode;
+import org.apache.lucene.store.Directory;
+import org.apache.lucene.store.IOContext;
+import org.apache.lucene.store.IndexInput;
+import org.apache.lucene.store.IndexOutput;
+import org.apache.lucene.util.IOUtils;
+import org.apache.solr.common.SolrException;
+import org.apache.solr.common.SolrException.ErrorCode;
+import org.apache.solr.core.DirectoryFactory;
+import org.apache.solr.core.IndexDeletionPolicyWrapper;
+import org.apache.solr.core.SolrCore;
+import org.apache.solr.core.DirectoryFactory.DirContext;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This class is responsible to manage the persistent snapshots meta-data
for the Solr indexes. The
+ * persistent snapshots are implemented by relying on Lucene {@linkplain
IndexDeletionPolicy}
+ * abstraction to configure a specific {@linkplain IndexCommit} to be
retained. The
+ * {@linkplain IndexDeletionPolicyWrapper} in Solr uses this class to
create/delete the Solr index
+ * snapshots.
+ */
+public class SolrSnapshotMetaDataManager {
+ private static final Logger log =
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
+ public static final String SNAPSHOT_METADATA_DIR = "snapshot_metadata";
+
+ /**
+ * A class defining the meta-data for a specific snapshot.
+ */
+ public static class SnapshotMetaData {
+private String name;
+private String indexDirPath;
+private long generationNumber;
+
+public SnapshotMetaData(String name, String indexDirPath, long
generationNumber) {
+ super();
+ this.name = name;
+ this.indexDirPath = indexDirPath;
+ this.generationNumber = generationNumber;
+}
+
+public String getName() {
+ return name;
+}
+
+public String getIndexDirPath() {
+ return indexDirPath;
+}
+
+public long getGenerationNumber() {
+ return generationNumber;
+}
+
+@Override
+public String toString() {
+ StringBuilder builder = new StringBuilder();
+ builder.append("SnapshotMetaData[name=");
+ builder.append(name);
+ builder.append(", indexDirPath=");
+ builder.append(indexDirPath);
+ builder.append(", generation=");
+ builder.append(generationNumber);
+ builder.append("]");
+ return builder.toString();
+}
+ }
+
+ /** Prefix used for the save file. */
+ public static final String SNAPSHOTS_PREFIX = "snapshots_";
+ private static final int VERSION_START = 0;
+ private static final int VERSION_CURRENT = VERSION_START;
+ private static final String CODEC_NAME = "solr-snapshots";
+
+ // The index writer which maintains the snapshots metadata
+ private long nextWriteGen;
+
+ private final Directory dir;
+
+ /** Used to map snapshot name to
[jira] [Commented] (SOLR-9269) Ability to create/delete/list snapshots for a solr core
[
https://issues.apache.org/jira/browse/SOLR-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388050#comment-15388050
]
ASF GitHub Bot commented on SOLR-9269:
--
Github user hgadre commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/52#discussion_r71745628
--- Diff:
solr/core/src/java/org/apache/solr/core/snapshots/SolrSnapshotMetaDataManager.java
---
@@ -0,0 +1,419 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.core.snapshots;
+
+import java.io.IOException;
+import java.lang.invoke.MethodHandles;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Optional;
+import java.util.stream.Collectors;
+
+import org.apache.lucene.codecs.CodecUtil;
+import org.apache.lucene.index.DirectoryReader;
+import org.apache.lucene.index.IndexCommit;
+import org.apache.lucene.index.IndexDeletionPolicy;
+import org.apache.lucene.index.IndexWriterConfig.OpenMode;
+import org.apache.lucene.store.Directory;
+import org.apache.lucene.store.IOContext;
+import org.apache.lucene.store.IndexInput;
+import org.apache.lucene.store.IndexOutput;
+import org.apache.lucene.util.IOUtils;
+import org.apache.solr.common.SolrException;
+import org.apache.solr.common.SolrException.ErrorCode;
+import org.apache.solr.core.DirectoryFactory;
+import org.apache.solr.core.IndexDeletionPolicyWrapper;
+import org.apache.solr.core.SolrCore;
+import org.apache.solr.core.DirectoryFactory.DirContext;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This class is responsible to manage the persistent snapshots meta-data
for the Solr indexes. The
+ * persistent snapshots are implemented by relying on Lucene {@linkplain
IndexDeletionPolicy}
+ * abstraction to configure a specific {@linkplain IndexCommit} to be
retained. The
+ * {@linkplain IndexDeletionPolicyWrapper} in Solr uses this class to
create/delete the Solr index
+ * snapshots.
+ */
+public class SolrSnapshotMetaDataManager {
+ private static final Logger log =
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
+ public static final String SNAPSHOT_METADATA_DIR = "snapshot_metadata";
+
+ /**
+ * A class defining the meta-data for a specific snapshot.
+ */
+ public static class SnapshotMetaData {
+private String name;
+private String indexDirPath;
+private long generationNumber;
+
+public SnapshotMetaData(String name, String indexDirPath, long
generationNumber) {
+ super();
+ this.name = name;
+ this.indexDirPath = indexDirPath;
+ this.generationNumber = generationNumber;
+}
+
+public String getName() {
+ return name;
+}
+
+public String getIndexDirPath() {
+ return indexDirPath;
+}
+
+public long getGenerationNumber() {
+ return generationNumber;
+}
+
+@Override
+public String toString() {
+ StringBuilder builder = new StringBuilder();
+ builder.append("SnapshotMetaData[name=");
+ builder.append(name);
+ builder.append(", indexDirPath=");
+ builder.append(indexDirPath);
+ builder.append(", generation=");
+ builder.append(generationNumber);
+ builder.append("]");
+ return builder.toString();
+}
+ }
+
+ /** Prefix used for the save file. */
+ public static final String SNAPSHOTS_PREFIX = "snapshots_";
+ private static final int VERSION_START = 0;
+ private static final int VERSION_CURRENT = VERSION_START;
+ private static
[jira] [Commented] (SOLR-9269) Ability to create/delete/list snapshots for a solr core
[
https://issues.apache.org/jira/browse/SOLR-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388048#comment-15388048
]
ASF GitHub Bot commented on SOLR-9269:
--
Github user hgadre commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/52#discussion_r71745546
--- Diff:
solr/core/src/java/org/apache/solr/core/snapshots/SolrSnapshotMetaDataManager.java
---
@@ -0,0 +1,419 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.core.snapshots;
+
+import java.io.IOException;
+import java.lang.invoke.MethodHandles;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Optional;
+import java.util.stream.Collectors;
+
+import org.apache.lucene.codecs.CodecUtil;
+import org.apache.lucene.index.DirectoryReader;
+import org.apache.lucene.index.IndexCommit;
+import org.apache.lucene.index.IndexDeletionPolicy;
+import org.apache.lucene.index.IndexWriterConfig.OpenMode;
+import org.apache.lucene.store.Directory;
+import org.apache.lucene.store.IOContext;
+import org.apache.lucene.store.IndexInput;
+import org.apache.lucene.store.IndexOutput;
+import org.apache.lucene.util.IOUtils;
+import org.apache.solr.common.SolrException;
+import org.apache.solr.common.SolrException.ErrorCode;
+import org.apache.solr.core.DirectoryFactory;
+import org.apache.solr.core.IndexDeletionPolicyWrapper;
+import org.apache.solr.core.SolrCore;
+import org.apache.solr.core.DirectoryFactory.DirContext;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This class is responsible to manage the persistent snapshots meta-data
for the Solr indexes. The
+ * persistent snapshots are implemented by relying on Lucene {@linkplain
IndexDeletionPolicy}
+ * abstraction to configure a specific {@linkplain IndexCommit} to be
retained. The
+ * {@linkplain IndexDeletionPolicyWrapper} in Solr uses this class to
create/delete the Solr index
+ * snapshots.
+ */
+public class SolrSnapshotMetaDataManager {
+ private static final Logger log =
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
+ public static final String SNAPSHOT_METADATA_DIR = "snapshot_metadata";
+
+ /**
+ * A class defining the meta-data for a specific snapshot.
+ */
+ public static class SnapshotMetaData {
+private String name;
+private String indexDirPath;
+private long generationNumber;
+
+public SnapshotMetaData(String name, String indexDirPath, long
generationNumber) {
+ super();
+ this.name = name;
+ this.indexDirPath = indexDirPath;
+ this.generationNumber = generationNumber;
+}
+
+public String getName() {
+ return name;
+}
+
+public String getIndexDirPath() {
+ return indexDirPath;
+}
+
+public long getGenerationNumber() {
+ return generationNumber;
+}
+
+@Override
+public String toString() {
+ StringBuilder builder = new StringBuilder();
+ builder.append("SnapshotMetaData[name=");
+ builder.append(name);
+ builder.append(", indexDirPath=");
+ builder.append(indexDirPath);
+ builder.append(", generation=");
+ builder.append(generationNumber);
+ builder.append("]");
+ return builder.toString();
+}
+ }
+
+ /** Prefix used for the save file. */
+ public static final String SNAPSHOTS_PREFIX = "snapshots_";
+ private static final int VERSION_START = 0;
+ private static final int VERSION_CURRENT = VERSION_START;
+ private static
[GitHub] lucene-solr pull request #52: [SOLR-9269] Ability to create/delete/list snap...
Github user hgadre commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/52#discussion_r71745546
--- Diff:
solr/core/src/java/org/apache/solr/core/snapshots/SolrSnapshotMetaDataManager.java
---
@@ -0,0 +1,419 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.core.snapshots;
+
+import java.io.IOException;
+import java.lang.invoke.MethodHandles;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Optional;
+import java.util.stream.Collectors;
+
+import org.apache.lucene.codecs.CodecUtil;
+import org.apache.lucene.index.DirectoryReader;
+import org.apache.lucene.index.IndexCommit;
+import org.apache.lucene.index.IndexDeletionPolicy;
+import org.apache.lucene.index.IndexWriterConfig.OpenMode;
+import org.apache.lucene.store.Directory;
+import org.apache.lucene.store.IOContext;
+import org.apache.lucene.store.IndexInput;
+import org.apache.lucene.store.IndexOutput;
+import org.apache.lucene.util.IOUtils;
+import org.apache.solr.common.SolrException;
+import org.apache.solr.common.SolrException.ErrorCode;
+import org.apache.solr.core.DirectoryFactory;
+import org.apache.solr.core.IndexDeletionPolicyWrapper;
+import org.apache.solr.core.SolrCore;
+import org.apache.solr.core.DirectoryFactory.DirContext;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This class is responsible to manage the persistent snapshots meta-data
for the Solr indexes. The
+ * persistent snapshots are implemented by relying on Lucene {@linkplain
IndexDeletionPolicy}
+ * abstraction to configure a specific {@linkplain IndexCommit} to be
retained. The
+ * {@linkplain IndexDeletionPolicyWrapper} in Solr uses this class to
create/delete the Solr index
+ * snapshots.
+ */
+public class SolrSnapshotMetaDataManager {
+ private static final Logger log =
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
+ public static final String SNAPSHOT_METADATA_DIR = "snapshot_metadata";
+
+ /**
+ * A class defining the meta-data for a specific snapshot.
+ */
+ public static class SnapshotMetaData {
+private String name;
+private String indexDirPath;
+private long generationNumber;
+
+public SnapshotMetaData(String name, String indexDirPath, long
generationNumber) {
+ super();
+ this.name = name;
+ this.indexDirPath = indexDirPath;
+ this.generationNumber = generationNumber;
+}
+
+public String getName() {
+ return name;
+}
+
+public String getIndexDirPath() {
+ return indexDirPath;
+}
+
+public long getGenerationNumber() {
+ return generationNumber;
+}
+
+@Override
+public String toString() {
+ StringBuilder builder = new StringBuilder();
+ builder.append("SnapshotMetaData[name=");
+ builder.append(name);
+ builder.append(", indexDirPath=");
+ builder.append(indexDirPath);
+ builder.append(", generation=");
+ builder.append(generationNumber);
+ builder.append("]");
+ return builder.toString();
+}
+ }
+
+ /** Prefix used for the save file. */
+ public static final String SNAPSHOTS_PREFIX = "snapshots_";
+ private static final int VERSION_START = 0;
+ private static final int VERSION_CURRENT = VERSION_START;
+ private static final String CODEC_NAME = "solr-snapshots";
+
+ // The index writer which maintains the snapshots metadata
+ private long nextWriteGen;
+
+ private final Directory dir;
+
+ /** Used to map snapshot name to
[jira] [Commented] (SOLR-9269) Ability to create/delete/list snapshots for a solr core
[
https://issues.apache.org/jira/browse/SOLR-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388044#comment-15388044
]
ASF GitHub Bot commented on SOLR-9269:
--
Github user hgadre commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/52#discussion_r71745330
--- Diff:
solr/core/src/java/org/apache/solr/core/snapshots/SolrSnapshotMetaDataManager.java
---
@@ -0,0 +1,419 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.core.snapshots;
+
+import java.io.IOException;
+import java.lang.invoke.MethodHandles;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Optional;
+import java.util.stream.Collectors;
+
+import org.apache.lucene.codecs.CodecUtil;
+import org.apache.lucene.index.DirectoryReader;
+import org.apache.lucene.index.IndexCommit;
+import org.apache.lucene.index.IndexDeletionPolicy;
+import org.apache.lucene.index.IndexWriterConfig.OpenMode;
+import org.apache.lucene.store.Directory;
+import org.apache.lucene.store.IOContext;
+import org.apache.lucene.store.IndexInput;
+import org.apache.lucene.store.IndexOutput;
+import org.apache.lucene.util.IOUtils;
+import org.apache.solr.common.SolrException;
+import org.apache.solr.common.SolrException.ErrorCode;
+import org.apache.solr.core.DirectoryFactory;
+import org.apache.solr.core.IndexDeletionPolicyWrapper;
+import org.apache.solr.core.SolrCore;
+import org.apache.solr.core.DirectoryFactory.DirContext;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This class is responsible to manage the persistent snapshots meta-data
for the Solr indexes. The
+ * persistent snapshots are implemented by relying on Lucene {@linkplain
IndexDeletionPolicy}
+ * abstraction to configure a specific {@linkplain IndexCommit} to be
retained. The
+ * {@linkplain IndexDeletionPolicyWrapper} in Solr uses this class to
create/delete the Solr index
+ * snapshots.
+ */
+public class SolrSnapshotMetaDataManager {
+ private static final Logger log =
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
+ public static final String SNAPSHOT_METADATA_DIR = "snapshot_metadata";
+
+ /**
+ * A class defining the meta-data for a specific snapshot.
+ */
+ public static class SnapshotMetaData {
+private String name;
+private String indexDirPath;
+private long generationNumber;
+
+public SnapshotMetaData(String name, String indexDirPath, long
generationNumber) {
+ super();
+ this.name = name;
+ this.indexDirPath = indexDirPath;
+ this.generationNumber = generationNumber;
+}
+
+public String getName() {
+ return name;
+}
+
+public String getIndexDirPath() {
+ return indexDirPath;
+}
+
+public long getGenerationNumber() {
+ return generationNumber;
+}
+
+@Override
+public String toString() {
+ StringBuilder builder = new StringBuilder();
+ builder.append("SnapshotMetaData[name=");
+ builder.append(name);
+ builder.append(", indexDirPath=");
+ builder.append(indexDirPath);
+ builder.append(", generation=");
+ builder.append(generationNumber);
+ builder.append("]");
+ return builder.toString();
+}
+ }
+
+ /** Prefix used for the save file. */
+ public static final String SNAPSHOTS_PREFIX = "snapshots_";
+ private static final int VERSION_START = 0;
+ private static final int VERSION_CURRENT = VERSION_START;
+ private static
[GitHub] lucene-solr pull request #52: [SOLR-9269] Ability to create/delete/list snap...
Github user hgadre commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/52#discussion_r71745330
--- Diff:
solr/core/src/java/org/apache/solr/core/snapshots/SolrSnapshotMetaDataManager.java
---
@@ -0,0 +1,419 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.core.snapshots;
+
+import java.io.IOException;
+import java.lang.invoke.MethodHandles;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Optional;
+import java.util.stream.Collectors;
+
+import org.apache.lucene.codecs.CodecUtil;
+import org.apache.lucene.index.DirectoryReader;
+import org.apache.lucene.index.IndexCommit;
+import org.apache.lucene.index.IndexDeletionPolicy;
+import org.apache.lucene.index.IndexWriterConfig.OpenMode;
+import org.apache.lucene.store.Directory;
+import org.apache.lucene.store.IOContext;
+import org.apache.lucene.store.IndexInput;
+import org.apache.lucene.store.IndexOutput;
+import org.apache.lucene.util.IOUtils;
+import org.apache.solr.common.SolrException;
+import org.apache.solr.common.SolrException.ErrorCode;
+import org.apache.solr.core.DirectoryFactory;
+import org.apache.solr.core.IndexDeletionPolicyWrapper;
+import org.apache.solr.core.SolrCore;
+import org.apache.solr.core.DirectoryFactory.DirContext;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This class is responsible to manage the persistent snapshots meta-data
for the Solr indexes. The
+ * persistent snapshots are implemented by relying on Lucene {@linkplain
IndexDeletionPolicy}
+ * abstraction to configure a specific {@linkplain IndexCommit} to be
retained. The
+ * {@linkplain IndexDeletionPolicyWrapper} in Solr uses this class to
create/delete the Solr index
+ * snapshots.
+ */
+public class SolrSnapshotMetaDataManager {
+ private static final Logger log =
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
+ public static final String SNAPSHOT_METADATA_DIR = "snapshot_metadata";
+
+ /**
+ * A class defining the meta-data for a specific snapshot.
+ */
+ public static class SnapshotMetaData {
+private String name;
+private String indexDirPath;
+private long generationNumber;
+
+public SnapshotMetaData(String name, String indexDirPath, long
generationNumber) {
+ super();
+ this.name = name;
+ this.indexDirPath = indexDirPath;
+ this.generationNumber = generationNumber;
+}
+
+public String getName() {
+ return name;
+}
+
+public String getIndexDirPath() {
+ return indexDirPath;
+}
+
+public long getGenerationNumber() {
+ return generationNumber;
+}
+
+@Override
+public String toString() {
+ StringBuilder builder = new StringBuilder();
+ builder.append("SnapshotMetaData[name=");
+ builder.append(name);
+ builder.append(", indexDirPath=");
+ builder.append(indexDirPath);
+ builder.append(", generation=");
+ builder.append(generationNumber);
+ builder.append("]");
+ return builder.toString();
+}
+ }
+
+ /** Prefix used for the save file. */
+ public static final String SNAPSHOTS_PREFIX = "snapshots_";
+ private static final int VERSION_START = 0;
+ private static final int VERSION_CURRENT = VERSION_START;
+ private static final String CODEC_NAME = "solr-snapshots";
+
+ // The index writer which maintains the snapshots metadata
+ private long nextWriteGen;
+
+ private final Directory dir;
+
+ /** Used to map snapshot name to
[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 732 - Still Unstable!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/732/ Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC 1 tests failed. FAILED: org.apache.solr.common.cloud.TestCollectionStateWatchers.testWatchesWorkForStateFormat1 Error Message: CollectionStateWatcher not notified of stateformat=1 collection creation Stack Trace: java.lang.AssertionError: CollectionStateWatcher not notified of stateformat=1 collection creation at __randomizedtesting.SeedInfo.seed([CF545883CA66C15F:A893446D3EDEE5FE]:0) at org.junit.Assert.fail(Assert.java:93) at org.junit.Assert.assertTrue(Assert.java:43) at org.apache.solr.common.cloud.TestCollectionStateWatchers.testWatchesWorkForStateFormat1(TestCollectionStateWatchers.java:267) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871) at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907) at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367) at java.lang.Thread.run(Thread.java:745) Build Log: [...truncated 13183 lines...] [junit4] Suite: org.apache.solr.common.cloud.TestCollectionStateWatchers [junit4] 2>
[jira] [Commented] (SOLR-8029) Modernize and standardize Solr APIs
[
https://issues.apache.org/jira/browse/SOLR-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15387985#comment-15387985
]
ASF subversion and git services commented on SOLR-8029:
---
Commit f9f148fb7cf602ddefb25c6edd33887c6a93bcf0 in lucene-solr's branch
refs/heads/apiv2 from [~steve_rowe]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=f9f148f ]
SOLR-8029: make precommit happy: drop unused imports and move license headers
to the top of files
> Modernize and standardize Solr APIs
> ---
>
> Key: SOLR-8029
> URL: https://issues.apache.org/jira/browse/SOLR-8029
> Project: Solr
> Issue Type: Improvement
>Affects Versions: 6.0
>Reporter: Noble Paul
>Assignee: Noble Paul
> Labels: API, EaseOfUse
> Fix For: 6.0
>
> Attachments: SOLR-8029.patch, SOLR-8029.patch, SOLR-8029.patch,
> SOLR-8029.patch
>
>
> Solr APIs have organically evolved and they are sometimes inconsistent with
> each other or not in sync with the widely followed conventions of HTTP
> protocol. Trying to make incremental changes to make them modern is like
> applying band-aid. So, we have done a complete rethink of what the APIs
> should be. The most notable aspects of the API are as follows:
> The new set of APIs will be placed under a new path {{/solr2}}. The legacy
> APIs will continue to work under the {{/solr}} path as they used to and they
> will be eventually deprecated.
> There are 4 types of requests in the new API
> * {{/v2//*}} : Hit a collection directly or manage
> collections/shards/replicas
> * {{/v2//*}} : Hit a core directly or manage cores
> * {{/v2/cluster/*}} : Operations on cluster not pertaining to any collection
> or core. e.g: security, overseer ops etc
> This will be released as part of a major release. Check the link given below
> for the full specification. Your comments are welcome
> [Solr API version 2 Specification | http://bit.ly/1JYsBMQ]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9269) Ability to create/delete/list snapshots for a solr core
[ https://issues.apache.org/jira/browse/SOLR-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15387966#comment-15387966 ] David Smiley commented on SOLR-9269: Done adding comments for now. I really like your tests. > Ability to create/delete/list snapshots for a solr core > --- > > Key: SOLR-9269 > URL: https://issues.apache.org/jira/browse/SOLR-9269 > Project: Solr > Issue Type: Sub-task > Components: SolrCloud >Reporter: Hrishikesh Gadre >Assignee: David Smiley > Attachments: SOLR-9269.patch > > > Support snapshot create/delete/list functionality @ the Solr core level. > Please refer to parent JIRA for more details. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9269) Ability to create/delete/list snapshots for a solr core
[
https://issues.apache.org/jira/browse/SOLR-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15387963#comment-15387963
]
ASF GitHub Bot commented on SOLR-9269:
--
Github user dsmiley commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/52#discussion_r71737044
--- Diff:
solr/core/src/test/org/apache/solr/handler/TestSolrCoreSnapshots.java ---
@@ -0,0 +1,446 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.handler;
+
+import static org.apache.solr.common.cloud.ZkStateReader.BASE_URL_PROP;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.lang.invoke.MethodHandles;
+import java.net.URL;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import org.apache.commons.io.IOUtils;
+import org.apache.lucene.index.DirectoryReader;
+import org.apache.lucene.index.IndexCommit;
+import org.apache.lucene.index.IndexNotFoundException;
+import org.apache.lucene.store.SimpleFSDirectory;
+import org.apache.lucene.util.TestUtil;
+import org.apache.lucene.util.LuceneTestCase.Slow;
+import org.apache.solr.SolrTestCaseJ4;
+import org.apache.solr.client.solrj.SolrClient;
+import org.apache.solr.client.solrj.impl.CloudSolrClient;
+import org.apache.solr.client.solrj.request.CollectionAdminRequest;
+import
org.apache.solr.client.solrj.request.CoreAdminRequest.CreateSnapshot;
+import
org.apache.solr.client.solrj.request.CoreAdminRequest.DeleteSnapshot;
+import org.apache.solr.client.solrj.request.CoreAdminRequest.ListSnapshots;
+import org.apache.solr.cloud.SolrCloudTestCase;
+import org.apache.solr.common.SolrInputDocument;
+import org.apache.solr.common.cloud.DocCollection;
+import org.apache.solr.common.cloud.Replica;
+import org.apache.solr.common.cloud.Slice;
+import org.apache.solr.common.cloud.ZkStateReader;
+import org.apache.solr.common.params.CoreAdminParams.CoreAdminAction;
+import org.apache.solr.common.util.NamedList;
+import org.apache.solr.core.snapshots.SolrSnapshotManager;
+import
org.apache.solr.core.snapshots.SolrSnapshotMetaDataManager.SnapshotMetaData;
+import org.junit.AfterClass;
+import org.junit.BeforeClass;
+import org.junit.Test;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+@SolrTestCaseJ4.SuppressSSL // Currently unknown why SSL does not work
with this test
+@Slow
+public class TestSolrCoreSnapshots extends SolrCloudTestCase {
+ private static final Logger log =
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
+ private static long docsSeed; // see indexDocs()
+
+ @BeforeClass
+ public static void setupClass() throws Exception {
+useFactory("solr.StandardDirectoryFactory");
+configureCluster(1)// nodes
+.addConfig("conf1",
TEST_PATH().resolve("configsets").resolve("cloud-minimal").resolve("conf"))
+.configure();
+
+docsSeed = random().nextLong();
+ }
+
+ @AfterClass
+ public static void teardownClass() throws Exception {
+System.clearProperty("test.build.data");
+System.clearProperty("test.cache.data");
+ }
+
+ @Test
+ public void testBackupRestore() throws Exception {
+CloudSolrClient solrClient = cluster.getSolrClient();
+String collectionName = "SolrCoreSnapshots";
+CollectionAdminRequest.Create create =
CollectionAdminRequest.createCollection(collectionName, "conf1", 1, 1);
+create.process(solrClient);
+
+String location = createTempDir().toFile().getAbsolutePath();
+int nDocs = BackupRestoreUtils.indexDocs(cluster.getSolrClient(),
collectionName, docsSeed);
+
+
[jira] [Commented] (SOLR-9269) Ability to create/delete/list snapshots for a solr core
[
https://issues.apache.org/jira/browse/SOLR-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15387962#comment-15387962
]
ASF GitHub Bot commented on SOLR-9269:
--
Github user dsmiley commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/52#discussion_r71736862
--- Diff:
solr/core/src/java/org/apache/solr/core/snapshots/SolrSnapshotManager.java ---
@@ -0,0 +1,132 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.core.snapshots;
+
+import java.io.IOException;
+import java.lang.invoke.MethodHandles;
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.function.Function;
+import java.util.stream.Collectors;
+
+import org.apache.lucene.index.DirectoryReader;
+import org.apache.lucene.index.IndexCommit;
+import org.apache.lucene.store.Directory;
+import
org.apache.solr.core.snapshots.SolrSnapshotMetaDataManager.SnapshotMetaData;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This class provides functionality required to handle the data files
corresponding to Solr snapshots.
+ */
+public class SolrSnapshotManager {
+ private static final Logger log =
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
+
+ /**
+ * This method deletes index files of the {@linkplain IndexCommit} for
the specified generation number.
+ *
+ * @param dir The index directory storing the snapshot.
+ * @param gen The generation number for the {@linkplain IndexCommit}
+ * @throws IOException in case of I/O errors.
+ */
+ public static synchronized void deleteIndexFiles ( Directory dir,
Collection snapshots, long gen ) throws IOException {
+List commits = DirectoryReader.listCommits(dir);
+Map refCounts = buildRefCounts(snapshots,commits);
+for (IndexCommit ic : commits) {
+ if (ic.getGeneration() == gen) {
+deleteIndexFiles(dir,refCounts, ic);
+break;
+ }
+}
+ }
+
+ /**
+ * This method deletes all files not corresponding to a configured
snapshot in the specified index directory.
+ *
+ * @param dir The index directory to search for.
+ * @throws IOException in case of I/O errors.
+ */
+ public static synchronized void deleteNonSnapshotIndexFiles (Directory
dir, Collection snapshots) throws IOException {
+List commits = DirectoryReader.listCommits(dir);
+Map refCounts = buildRefCounts(snapshots, commits);
+Set snapshotGenNumbers = snapshots.stream()
+
.map(SnapshotMetaData::getGenerationNumber)
+.collect(Collectors.toSet());
+for (IndexCommit ic : commits) {
+ if (!snapshotGenNumbers.contains(ic.getGeneration())) {
+deleteIndexFiles(dir,refCounts, ic);
+ }
+}
+ }
+
+ /**
+ * This method computes reference count for the index files by taking
into consideration
+ * (a) configured snapshots and (b) files sharing between two or more
{@linkplain IndexCommit} instances.
+ *
+ * @param snapshots A collection of user configured snapshots
+ * @param commits A list of {@linkplain IndexCommit} instances
+ * @return A map containing reference count for each index file referred
in one of the {@linkplain IndexCommit} instances.
+ * @throws IOException in case of I/O error.
+ */
+ public static Map buildRefCounts
(Collection snapshots, List commits) throws
IOException {
+Map result = new HashMap<>();
+Map commitsByGen = commits.stream().collect(
+Collectors.toMap(IndexCommit::getGeneration, Function.identity()));
+
+for(SnapshotMetaData md :
[GitHub] lucene-solr pull request #52: [SOLR-9269] Ability to create/delete/list snap...
Github user dsmiley commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/52#discussion_r71737044
--- Diff:
solr/core/src/test/org/apache/solr/handler/TestSolrCoreSnapshots.java ---
@@ -0,0 +1,446 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.handler;
+
+import static org.apache.solr.common.cloud.ZkStateReader.BASE_URL_PROP;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.lang.invoke.MethodHandles;
+import java.net.URL;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import org.apache.commons.io.IOUtils;
+import org.apache.lucene.index.DirectoryReader;
+import org.apache.lucene.index.IndexCommit;
+import org.apache.lucene.index.IndexNotFoundException;
+import org.apache.lucene.store.SimpleFSDirectory;
+import org.apache.lucene.util.TestUtil;
+import org.apache.lucene.util.LuceneTestCase.Slow;
+import org.apache.solr.SolrTestCaseJ4;
+import org.apache.solr.client.solrj.SolrClient;
+import org.apache.solr.client.solrj.impl.CloudSolrClient;
+import org.apache.solr.client.solrj.request.CollectionAdminRequest;
+import
org.apache.solr.client.solrj.request.CoreAdminRequest.CreateSnapshot;
+import
org.apache.solr.client.solrj.request.CoreAdminRequest.DeleteSnapshot;
+import org.apache.solr.client.solrj.request.CoreAdminRequest.ListSnapshots;
+import org.apache.solr.cloud.SolrCloudTestCase;
+import org.apache.solr.common.SolrInputDocument;
+import org.apache.solr.common.cloud.DocCollection;
+import org.apache.solr.common.cloud.Replica;
+import org.apache.solr.common.cloud.Slice;
+import org.apache.solr.common.cloud.ZkStateReader;
+import org.apache.solr.common.params.CoreAdminParams.CoreAdminAction;
+import org.apache.solr.common.util.NamedList;
+import org.apache.solr.core.snapshots.SolrSnapshotManager;
+import
org.apache.solr.core.snapshots.SolrSnapshotMetaDataManager.SnapshotMetaData;
+import org.junit.AfterClass;
+import org.junit.BeforeClass;
+import org.junit.Test;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+@SolrTestCaseJ4.SuppressSSL // Currently unknown why SSL does not work
with this test
+@Slow
+public class TestSolrCoreSnapshots extends SolrCloudTestCase {
+ private static final Logger log =
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
+ private static long docsSeed; // see indexDocs()
+
+ @BeforeClass
+ public static void setupClass() throws Exception {
+useFactory("solr.StandardDirectoryFactory");
+configureCluster(1)// nodes
+.addConfig("conf1",
TEST_PATH().resolve("configsets").resolve("cloud-minimal").resolve("conf"))
+.configure();
+
+docsSeed = random().nextLong();
+ }
+
+ @AfterClass
+ public static void teardownClass() throws Exception {
+System.clearProperty("test.build.data");
+System.clearProperty("test.cache.data");
+ }
+
+ @Test
+ public void testBackupRestore() throws Exception {
+CloudSolrClient solrClient = cluster.getSolrClient();
+String collectionName = "SolrCoreSnapshots";
+CollectionAdminRequest.Create create =
CollectionAdminRequest.createCollection(collectionName, "conf1", 1, 1);
+create.process(solrClient);
+
+String location = createTempDir().toFile().getAbsolutePath();
+int nDocs = BackupRestoreUtils.indexDocs(cluster.getSolrClient(),
collectionName, docsSeed);
+
+DocCollection collectionState =
solrClient.getZkStateReader().getClusterState().getCollection(collectionName);
+assertEquals(1, collectionState.getActiveSlices().size());
+Slice shard =
[GitHub] lucene-solr pull request #52: [SOLR-9269] Ability to create/delete/list snap...
Github user dsmiley commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/52#discussion_r71736862
--- Diff:
solr/core/src/java/org/apache/solr/core/snapshots/SolrSnapshotManager.java ---
@@ -0,0 +1,132 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.core.snapshots;
+
+import java.io.IOException;
+import java.lang.invoke.MethodHandles;
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.function.Function;
+import java.util.stream.Collectors;
+
+import org.apache.lucene.index.DirectoryReader;
+import org.apache.lucene.index.IndexCommit;
+import org.apache.lucene.store.Directory;
+import
org.apache.solr.core.snapshots.SolrSnapshotMetaDataManager.SnapshotMetaData;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This class provides functionality required to handle the data files
corresponding to Solr snapshots.
+ */
+public class SolrSnapshotManager {
+ private static final Logger log =
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
+
+ /**
+ * This method deletes index files of the {@linkplain IndexCommit} for
the specified generation number.
+ *
+ * @param dir The index directory storing the snapshot.
+ * @param gen The generation number for the {@linkplain IndexCommit}
+ * @throws IOException in case of I/O errors.
+ */
+ public static synchronized void deleteIndexFiles ( Directory dir,
Collection snapshots, long gen ) throws IOException {
+List commits = DirectoryReader.listCommits(dir);
+Map refCounts = buildRefCounts(snapshots,commits);
+for (IndexCommit ic : commits) {
+ if (ic.getGeneration() == gen) {
+deleteIndexFiles(dir,refCounts, ic);
+break;
+ }
+}
+ }
+
+ /**
+ * This method deletes all files not corresponding to a configured
snapshot in the specified index directory.
+ *
+ * @param dir The index directory to search for.
+ * @throws IOException in case of I/O errors.
+ */
+ public static synchronized void deleteNonSnapshotIndexFiles (Directory
dir, Collection snapshots) throws IOException {
+List commits = DirectoryReader.listCommits(dir);
+Map refCounts = buildRefCounts(snapshots, commits);
+Set snapshotGenNumbers = snapshots.stream()
+
.map(SnapshotMetaData::getGenerationNumber)
+.collect(Collectors.toSet());
+for (IndexCommit ic : commits) {
+ if (!snapshotGenNumbers.contains(ic.getGeneration())) {
+deleteIndexFiles(dir,refCounts, ic);
+ }
+}
+ }
+
+ /**
+ * This method computes reference count for the index files by taking
into consideration
+ * (a) configured snapshots and (b) files sharing between two or more
{@linkplain IndexCommit} instances.
+ *
+ * @param snapshots A collection of user configured snapshots
+ * @param commits A list of {@linkplain IndexCommit} instances
+ * @return A map containing reference count for each index file referred
in one of the {@linkplain IndexCommit} instances.
+ * @throws IOException in case of I/O error.
+ */
+ public static Map buildRefCounts
(Collection snapshots, List commits) throws
IOException {
+Map result = new HashMap<>();
+Map commitsByGen = commits.stream().collect(
+Collectors.toMap(IndexCommit::getGeneration, Function.identity()));
+
+for(SnapshotMetaData md : snapshots) {
+ IndexCommit ic = commitsByGen.get(md.getGenerationNumber());
+ if (ic != null) {
+Collection fileNames = ic.getFileNames();
+for(String fileName : fileNames) {
+ int
[jira] [Commented] (SOLR-9269) Ability to create/delete/list snapshots for a solr core
[
https://issues.apache.org/jira/browse/SOLR-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15387953#comment-15387953
]
ASF GitHub Bot commented on SOLR-9269:
--
Github user dsmiley commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/52#discussion_r71735109
--- Diff:
solr/core/src/java/org/apache/solr/core/snapshots/SolrSnapshotManager.java ---
@@ -0,0 +1,132 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.core.snapshots;
+
+import java.io.IOException;
+import java.lang.invoke.MethodHandles;
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.function.Function;
+import java.util.stream.Collectors;
+
+import org.apache.lucene.index.DirectoryReader;
+import org.apache.lucene.index.IndexCommit;
+import org.apache.lucene.store.Directory;
+import
org.apache.solr.core.snapshots.SolrSnapshotMetaDataManager.SnapshotMetaData;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This class provides functionality required to handle the data files
corresponding to Solr snapshots.
+ */
+public class SolrSnapshotManager {
+ private static final Logger log =
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
+
+ /**
+ * This method deletes index files of the {@linkplain IndexCommit} for
the specified generation number.
+ *
+ * @param dir The index directory storing the snapshot.
+ * @param gen The generation number for the {@linkplain IndexCommit}
+ * @throws IOException in case of I/O errors.
+ */
+ public static synchronized void deleteIndexFiles ( Directory dir,
Collection snapshots, long gen ) throws IOException {
+List commits = DirectoryReader.listCommits(dir);
+Map refCounts = buildRefCounts(snapshots,commits);
+for (IndexCommit ic : commits) {
+ if (ic.getGeneration() == gen) {
+deleteIndexFiles(dir,refCounts, ic);
+break;
+ }
+}
+ }
+
+ /**
+ * This method deletes all files not corresponding to a configured
snapshot in the specified index directory.
+ *
+ * @param dir The index directory to search for.
+ * @throws IOException in case of I/O errors.
+ */
+ public static synchronized void deleteNonSnapshotIndexFiles (Directory
dir, Collection snapshots) throws IOException {
+List commits = DirectoryReader.listCommits(dir);
+Map refCounts = buildRefCounts(snapshots, commits);
+Set snapshotGenNumbers = snapshots.stream()
+
.map(SnapshotMetaData::getGenerationNumber)
+.collect(Collectors.toSet());
+for (IndexCommit ic : commits) {
+ if (!snapshotGenNumbers.contains(ic.getGeneration())) {
+deleteIndexFiles(dir,refCounts, ic);
+ }
+}
+ }
+
+ /**
+ * This method computes reference count for the index files by taking
into consideration
+ * (a) configured snapshots and (b) files sharing between two or more
{@linkplain IndexCommit} instances.
+ *
+ * @param snapshots A collection of user configured snapshots
+ * @param commits A list of {@linkplain IndexCommit} instances
+ * @return A map containing reference count for each index file referred
in one of the {@linkplain IndexCommit} instances.
+ * @throws IOException in case of I/O error.
+ */
+ public static Map buildRefCounts
(Collection snapshots, List commits) throws
IOException {
--- End diff --
It seems that the callers of this method don't actually need to know the
ref counts. The callers want to know which file names referred to. I think
returning exactly that, and naming it as such, would be much more
[GitHub] lucene-solr pull request #52: [SOLR-9269] Ability to create/delete/list snap...
Github user dsmiley commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/52#discussion_r71735109
--- Diff:
solr/core/src/java/org/apache/solr/core/snapshots/SolrSnapshotManager.java ---
@@ -0,0 +1,132 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.core.snapshots;
+
+import java.io.IOException;
+import java.lang.invoke.MethodHandles;
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.function.Function;
+import java.util.stream.Collectors;
+
+import org.apache.lucene.index.DirectoryReader;
+import org.apache.lucene.index.IndexCommit;
+import org.apache.lucene.store.Directory;
+import
org.apache.solr.core.snapshots.SolrSnapshotMetaDataManager.SnapshotMetaData;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This class provides functionality required to handle the data files
corresponding to Solr snapshots.
+ */
+public class SolrSnapshotManager {
+ private static final Logger log =
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
+
+ /**
+ * This method deletes index files of the {@linkplain IndexCommit} for
the specified generation number.
+ *
+ * @param dir The index directory storing the snapshot.
+ * @param gen The generation number for the {@linkplain IndexCommit}
+ * @throws IOException in case of I/O errors.
+ */
+ public static synchronized void deleteIndexFiles ( Directory dir,
Collection snapshots, long gen ) throws IOException {
+List commits = DirectoryReader.listCommits(dir);
+Map refCounts = buildRefCounts(snapshots,commits);
+for (IndexCommit ic : commits) {
+ if (ic.getGeneration() == gen) {
+deleteIndexFiles(dir,refCounts, ic);
+break;
+ }
+}
+ }
+
+ /**
+ * This method deletes all files not corresponding to a configured
snapshot in the specified index directory.
+ *
+ * @param dir The index directory to search for.
+ * @throws IOException in case of I/O errors.
+ */
+ public static synchronized void deleteNonSnapshotIndexFiles (Directory
dir, Collection snapshots) throws IOException {
+List commits = DirectoryReader.listCommits(dir);
+Map refCounts = buildRefCounts(snapshots, commits);
+Set snapshotGenNumbers = snapshots.stream()
+
.map(SnapshotMetaData::getGenerationNumber)
+.collect(Collectors.toSet());
+for (IndexCommit ic : commits) {
+ if (!snapshotGenNumbers.contains(ic.getGeneration())) {
+deleteIndexFiles(dir,refCounts, ic);
+ }
+}
+ }
+
+ /**
+ * This method computes reference count for the index files by taking
into consideration
+ * (a) configured snapshots and (b) files sharing between two or more
{@linkplain IndexCommit} instances.
+ *
+ * @param snapshots A collection of user configured snapshots
+ * @param commits A list of {@linkplain IndexCommit} instances
+ * @return A map containing reference count for each index file referred
in one of the {@linkplain IndexCommit} instances.
+ * @throws IOException in case of I/O error.
+ */
+ public static Map buildRefCounts
(Collection snapshots, List commits) throws
IOException {
--- End diff --
It seems that the callers of this method don't actually need to know the
ref counts. The callers want to know which file names referred to. I think
returning exactly that, and naming it as such, would be much more helpful.
(could be package-private, BTW)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the
Where is QueryParser?
I'm sorry if this is an ignorant question, but where has the QueryParser namespace gone? I can see that it was removed in 2014 when Itamar made the first moves toward 4.8, but I don't understand why it never reappeared after that. Ben
[jira] [Commented] (SOLR-9269) Ability to create/delete/list snapshots for a solr core
[
https://issues.apache.org/jira/browse/SOLR-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15387920#comment-15387920
]
ASF GitHub Bot commented on SOLR-9269:
--
Github user dsmiley commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/52#discussion_r71730404
--- Diff:
solr/core/src/java/org/apache/solr/core/snapshots/SolrSnapshotMetaDataManager.java
---
@@ -0,0 +1,419 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.core.snapshots;
+
+import java.io.IOException;
+import java.lang.invoke.MethodHandles;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Optional;
+import java.util.stream.Collectors;
+
+import org.apache.lucene.codecs.CodecUtil;
+import org.apache.lucene.index.DirectoryReader;
+import org.apache.lucene.index.IndexCommit;
+import org.apache.lucene.index.IndexDeletionPolicy;
+import org.apache.lucene.index.IndexWriterConfig.OpenMode;
+import org.apache.lucene.store.Directory;
+import org.apache.lucene.store.IOContext;
+import org.apache.lucene.store.IndexInput;
+import org.apache.lucene.store.IndexOutput;
+import org.apache.lucene.util.IOUtils;
+import org.apache.solr.common.SolrException;
+import org.apache.solr.common.SolrException.ErrorCode;
+import org.apache.solr.core.DirectoryFactory;
+import org.apache.solr.core.IndexDeletionPolicyWrapper;
+import org.apache.solr.core.SolrCore;
+import org.apache.solr.core.DirectoryFactory.DirContext;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This class is responsible to manage the persistent snapshots meta-data
for the Solr indexes. The
+ * persistent snapshots are implemented by relying on Lucene {@linkplain
IndexDeletionPolicy}
+ * abstraction to configure a specific {@linkplain IndexCommit} to be
retained. The
+ * {@linkplain IndexDeletionPolicyWrapper} in Solr uses this class to
create/delete the Solr index
+ * snapshots.
+ */
+public class SolrSnapshotMetaDataManager {
+ private static final Logger log =
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
+ public static final String SNAPSHOT_METADATA_DIR = "snapshot_metadata";
+
+ /**
+ * A class defining the meta-data for a specific snapshot.
+ */
+ public static class SnapshotMetaData {
+private String name;
+private String indexDirPath;
+private long generationNumber;
+
+public SnapshotMetaData(String name, String indexDirPath, long
generationNumber) {
+ super();
+ this.name = name;
+ this.indexDirPath = indexDirPath;
+ this.generationNumber = generationNumber;
+}
+
+public String getName() {
+ return name;
+}
+
+public String getIndexDirPath() {
+ return indexDirPath;
+}
+
+public long getGenerationNumber() {
+ return generationNumber;
+}
+
+@Override
+public String toString() {
+ StringBuilder builder = new StringBuilder();
+ builder.append("SnapshotMetaData[name=");
+ builder.append(name);
+ builder.append(", indexDirPath=");
+ builder.append(indexDirPath);
+ builder.append(", generation=");
+ builder.append(generationNumber);
+ builder.append("]");
+ return builder.toString();
+}
+ }
+
+ /** Prefix used for the save file. */
+ public static final String SNAPSHOTS_PREFIX = "snapshots_";
+ private static final int VERSION_START = 0;
+ private static final int VERSION_CURRENT = VERSION_START;
+ private static
[GitHub] lucene-solr pull request #52: [SOLR-9269] Ability to create/delete/list snap...
Github user dsmiley commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/52#discussion_r71730404
--- Diff:
solr/core/src/java/org/apache/solr/core/snapshots/SolrSnapshotMetaDataManager.java
---
@@ -0,0 +1,419 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.core.snapshots;
+
+import java.io.IOException;
+import java.lang.invoke.MethodHandles;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Optional;
+import java.util.stream.Collectors;
+
+import org.apache.lucene.codecs.CodecUtil;
+import org.apache.lucene.index.DirectoryReader;
+import org.apache.lucene.index.IndexCommit;
+import org.apache.lucene.index.IndexDeletionPolicy;
+import org.apache.lucene.index.IndexWriterConfig.OpenMode;
+import org.apache.lucene.store.Directory;
+import org.apache.lucene.store.IOContext;
+import org.apache.lucene.store.IndexInput;
+import org.apache.lucene.store.IndexOutput;
+import org.apache.lucene.util.IOUtils;
+import org.apache.solr.common.SolrException;
+import org.apache.solr.common.SolrException.ErrorCode;
+import org.apache.solr.core.DirectoryFactory;
+import org.apache.solr.core.IndexDeletionPolicyWrapper;
+import org.apache.solr.core.SolrCore;
+import org.apache.solr.core.DirectoryFactory.DirContext;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This class is responsible to manage the persistent snapshots meta-data
for the Solr indexes. The
+ * persistent snapshots are implemented by relying on Lucene {@linkplain
IndexDeletionPolicy}
+ * abstraction to configure a specific {@linkplain IndexCommit} to be
retained. The
+ * {@linkplain IndexDeletionPolicyWrapper} in Solr uses this class to
create/delete the Solr index
+ * snapshots.
+ */
+public class SolrSnapshotMetaDataManager {
+ private static final Logger log =
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
+ public static final String SNAPSHOT_METADATA_DIR = "snapshot_metadata";
+
+ /**
+ * A class defining the meta-data for a specific snapshot.
+ */
+ public static class SnapshotMetaData {
+private String name;
+private String indexDirPath;
+private long generationNumber;
+
+public SnapshotMetaData(String name, String indexDirPath, long
generationNumber) {
+ super();
+ this.name = name;
+ this.indexDirPath = indexDirPath;
+ this.generationNumber = generationNumber;
+}
+
+public String getName() {
+ return name;
+}
+
+public String getIndexDirPath() {
+ return indexDirPath;
+}
+
+public long getGenerationNumber() {
+ return generationNumber;
+}
+
+@Override
+public String toString() {
+ StringBuilder builder = new StringBuilder();
+ builder.append("SnapshotMetaData[name=");
+ builder.append(name);
+ builder.append(", indexDirPath=");
+ builder.append(indexDirPath);
+ builder.append(", generation=");
+ builder.append(generationNumber);
+ builder.append("]");
+ return builder.toString();
+}
+ }
+
+ /** Prefix used for the save file. */
+ public static final String SNAPSHOTS_PREFIX = "snapshots_";
+ private static final int VERSION_START = 0;
+ private static final int VERSION_CURRENT = VERSION_START;
+ private static final String CODEC_NAME = "solr-snapshots";
+
+ // The index writer which maintains the snapshots metadata
+ private long nextWriteGen;
+
+ private final Directory dir;
+
+ /** Used to map snapshot name to
[jira] [Commented] (SOLR-9269) Ability to create/delete/list snapshots for a solr core
[
https://issues.apache.org/jira/browse/SOLR-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15387917#comment-15387917
]
ASF GitHub Bot commented on SOLR-9269:
--
Github user dsmiley commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/52#discussion_r71730215
--- Diff: solr/core/src/java/org/apache/solr/core/SolrCore.java ---
@@ -414,7 +416,19 @@ private IndexDeletionPolicyWrapper
initDeletionPolicy(IndexDeletionPolicyWrapper
} else {
delPolicy = new SolrDeletionPolicy();
}
-return new IndexDeletionPolicyWrapper(delPolicy);
+
+return new IndexDeletionPolicyWrapper(delPolicy, snapshotMgr);
+ }
+
+ private SolrSnapshotMetaDataManager initSnapshotMetaDataManager() {
--- End diff --
What do you think about possibly creating a Directory lazily on first use
of a snapshot? I'm a tad concerned we're weighing Solr down a bit more for
everyone when only a small number of installations in the future will use this.
What's your feeling @markrmiller ?
> Ability to create/delete/list snapshots for a solr core
> ---
>
> Key: SOLR-9269
> URL: https://issues.apache.org/jira/browse/SOLR-9269
> Project: Solr
> Issue Type: Sub-task
> Components: SolrCloud
>Reporter: Hrishikesh Gadre
>Assignee: David Smiley
> Attachments: SOLR-9269.patch
>
>
> Support snapshot create/delete/list functionality @ the Solr core level.
> Please refer to parent JIRA for more details.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] lucene-solr pull request #52: [SOLR-9269] Ability to create/delete/list snap...
Github user dsmiley commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/52#discussion_r71730215
--- Diff: solr/core/src/java/org/apache/solr/core/SolrCore.java ---
@@ -414,7 +416,19 @@ private IndexDeletionPolicyWrapper
initDeletionPolicy(IndexDeletionPolicyWrapper
} else {
delPolicy = new SolrDeletionPolicy();
}
-return new IndexDeletionPolicyWrapper(delPolicy);
+
+return new IndexDeletionPolicyWrapper(delPolicy, snapshotMgr);
+ }
+
+ private SolrSnapshotMetaDataManager initSnapshotMetaDataManager() {
--- End diff --
What do you think about possibly creating a Directory lazily on first use
of a snapshot? I'm a tad concerned we're weighing Solr down a bit more for
everyone when only a small number of installations in the future will use this.
What's your feeling @markrmiller ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9269) Ability to create/delete/list snapshots for a solr core
[
https://issues.apache.org/jira/browse/SOLR-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15387899#comment-15387899
]
ASF GitHub Bot commented on SOLR-9269:
--
Github user dsmiley commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/52#discussion_r71727810
--- Diff:
solr/core/src/java/org/apache/solr/core/snapshots/SolrSnapshotMetaDataManager.java
---
@@ -0,0 +1,419 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.core.snapshots;
+
+import java.io.IOException;
+import java.lang.invoke.MethodHandles;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Optional;
+import java.util.stream.Collectors;
+
+import org.apache.lucene.codecs.CodecUtil;
+import org.apache.lucene.index.DirectoryReader;
+import org.apache.lucene.index.IndexCommit;
+import org.apache.lucene.index.IndexDeletionPolicy;
+import org.apache.lucene.index.IndexWriterConfig.OpenMode;
+import org.apache.lucene.store.Directory;
+import org.apache.lucene.store.IOContext;
+import org.apache.lucene.store.IndexInput;
+import org.apache.lucene.store.IndexOutput;
+import org.apache.lucene.util.IOUtils;
+import org.apache.solr.common.SolrException;
+import org.apache.solr.common.SolrException.ErrorCode;
+import org.apache.solr.core.DirectoryFactory;
+import org.apache.solr.core.IndexDeletionPolicyWrapper;
+import org.apache.solr.core.SolrCore;
+import org.apache.solr.core.DirectoryFactory.DirContext;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This class is responsible to manage the persistent snapshots meta-data
for the Solr indexes. The
+ * persistent snapshots are implemented by relying on Lucene {@linkplain
IndexDeletionPolicy}
+ * abstraction to configure a specific {@linkplain IndexCommit} to be
retained. The
+ * {@linkplain IndexDeletionPolicyWrapper} in Solr uses this class to
create/delete the Solr index
+ * snapshots.
+ */
+public class SolrSnapshotMetaDataManager {
+ private static final Logger log =
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
+ public static final String SNAPSHOT_METADATA_DIR = "snapshot_metadata";
+
+ /**
+ * A class defining the meta-data for a specific snapshot.
+ */
+ public static class SnapshotMetaData {
+private String name;
+private String indexDirPath;
+private long generationNumber;
+
+public SnapshotMetaData(String name, String indexDirPath, long
generationNumber) {
+ super();
+ this.name = name;
+ this.indexDirPath = indexDirPath;
+ this.generationNumber = generationNumber;
+}
+
+public String getName() {
+ return name;
+}
+
+public String getIndexDirPath() {
+ return indexDirPath;
+}
+
+public long getGenerationNumber() {
+ return generationNumber;
+}
+
+@Override
+public String toString() {
+ StringBuilder builder = new StringBuilder();
+ builder.append("SnapshotMetaData[name=");
+ builder.append(name);
+ builder.append(", indexDirPath=");
+ builder.append(indexDirPath);
+ builder.append(", generation=");
+ builder.append(generationNumber);
+ builder.append("]");
+ return builder.toString();
+}
+ }
+
+ /** Prefix used for the save file. */
+ public static final String SNAPSHOTS_PREFIX = "snapshots_";
+ private static final int VERSION_START = 0;
+ private static final int VERSION_CURRENT = VERSION_START;
+ private static
[GitHub] lucene-solr pull request #52: [SOLR-9269] Ability to create/delete/list snap...
Github user dsmiley commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/52#discussion_r71727810
--- Diff:
solr/core/src/java/org/apache/solr/core/snapshots/SolrSnapshotMetaDataManager.java
---
@@ -0,0 +1,419 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.core.snapshots;
+
+import java.io.IOException;
+import java.lang.invoke.MethodHandles;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Optional;
+import java.util.stream.Collectors;
+
+import org.apache.lucene.codecs.CodecUtil;
+import org.apache.lucene.index.DirectoryReader;
+import org.apache.lucene.index.IndexCommit;
+import org.apache.lucene.index.IndexDeletionPolicy;
+import org.apache.lucene.index.IndexWriterConfig.OpenMode;
+import org.apache.lucene.store.Directory;
+import org.apache.lucene.store.IOContext;
+import org.apache.lucene.store.IndexInput;
+import org.apache.lucene.store.IndexOutput;
+import org.apache.lucene.util.IOUtils;
+import org.apache.solr.common.SolrException;
+import org.apache.solr.common.SolrException.ErrorCode;
+import org.apache.solr.core.DirectoryFactory;
+import org.apache.solr.core.IndexDeletionPolicyWrapper;
+import org.apache.solr.core.SolrCore;
+import org.apache.solr.core.DirectoryFactory.DirContext;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This class is responsible to manage the persistent snapshots meta-data
for the Solr indexes. The
+ * persistent snapshots are implemented by relying on Lucene {@linkplain
IndexDeletionPolicy}
+ * abstraction to configure a specific {@linkplain IndexCommit} to be
retained. The
+ * {@linkplain IndexDeletionPolicyWrapper} in Solr uses this class to
create/delete the Solr index
+ * snapshots.
+ */
+public class SolrSnapshotMetaDataManager {
+ private static final Logger log =
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
+ public static final String SNAPSHOT_METADATA_DIR = "snapshot_metadata";
+
+ /**
+ * A class defining the meta-data for a specific snapshot.
+ */
+ public static class SnapshotMetaData {
+private String name;
+private String indexDirPath;
+private long generationNumber;
+
+public SnapshotMetaData(String name, String indexDirPath, long
generationNumber) {
+ super();
+ this.name = name;
+ this.indexDirPath = indexDirPath;
+ this.generationNumber = generationNumber;
+}
+
+public String getName() {
+ return name;
+}
+
+public String getIndexDirPath() {
+ return indexDirPath;
+}
+
+public long getGenerationNumber() {
+ return generationNumber;
+}
+
+@Override
+public String toString() {
+ StringBuilder builder = new StringBuilder();
+ builder.append("SnapshotMetaData[name=");
+ builder.append(name);
+ builder.append(", indexDirPath=");
+ builder.append(indexDirPath);
+ builder.append(", generation=");
+ builder.append(generationNumber);
+ builder.append("]");
+ return builder.toString();
+}
+ }
+
+ /** Prefix used for the save file. */
+ public static final String SNAPSHOTS_PREFIX = "snapshots_";
+ private static final int VERSION_START = 0;
+ private static final int VERSION_CURRENT = VERSION_START;
+ private static final String CODEC_NAME = "solr-snapshots";
+
+ // The index writer which maintains the snapshots metadata
+ private long nextWriteGen;
+
+ private final Directory dir;
+
+ /** Used to map snapshot name to
[jira] [Commented] (SOLR-9269) Ability to create/delete/list snapshots for a solr core
[
https://issues.apache.org/jira/browse/SOLR-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15387894#comment-15387894
]
ASF GitHub Bot commented on SOLR-9269:
--
Github user dsmiley commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/52#discussion_r71727257
--- Diff:
solr/core/src/java/org/apache/solr/core/snapshots/SolrSnapshotMetaDataManager.java
---
@@ -0,0 +1,419 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.core.snapshots;
+
+import java.io.IOException;
+import java.lang.invoke.MethodHandles;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Optional;
+import java.util.stream.Collectors;
+
+import org.apache.lucene.codecs.CodecUtil;
+import org.apache.lucene.index.DirectoryReader;
+import org.apache.lucene.index.IndexCommit;
+import org.apache.lucene.index.IndexDeletionPolicy;
+import org.apache.lucene.index.IndexWriterConfig.OpenMode;
+import org.apache.lucene.store.Directory;
+import org.apache.lucene.store.IOContext;
+import org.apache.lucene.store.IndexInput;
+import org.apache.lucene.store.IndexOutput;
+import org.apache.lucene.util.IOUtils;
+import org.apache.solr.common.SolrException;
+import org.apache.solr.common.SolrException.ErrorCode;
+import org.apache.solr.core.DirectoryFactory;
+import org.apache.solr.core.IndexDeletionPolicyWrapper;
+import org.apache.solr.core.SolrCore;
+import org.apache.solr.core.DirectoryFactory.DirContext;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This class is responsible to manage the persistent snapshots meta-data
for the Solr indexes. The
+ * persistent snapshots are implemented by relying on Lucene {@linkplain
IndexDeletionPolicy}
+ * abstraction to configure a specific {@linkplain IndexCommit} to be
retained. The
+ * {@linkplain IndexDeletionPolicyWrapper} in Solr uses this class to
create/delete the Solr index
+ * snapshots.
+ */
+public class SolrSnapshotMetaDataManager {
+ private static final Logger log =
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
+ public static final String SNAPSHOT_METADATA_DIR = "snapshot_metadata";
+
+ /**
+ * A class defining the meta-data for a specific snapshot.
+ */
+ public static class SnapshotMetaData {
+private String name;
+private String indexDirPath;
+private long generationNumber;
+
+public SnapshotMetaData(String name, String indexDirPath, long
generationNumber) {
+ super();
+ this.name = name;
+ this.indexDirPath = indexDirPath;
+ this.generationNumber = generationNumber;
+}
+
+public String getName() {
+ return name;
+}
+
+public String getIndexDirPath() {
+ return indexDirPath;
+}
+
+public long getGenerationNumber() {
+ return generationNumber;
+}
+
+@Override
+public String toString() {
+ StringBuilder builder = new StringBuilder();
+ builder.append("SnapshotMetaData[name=");
+ builder.append(name);
+ builder.append(", indexDirPath=");
+ builder.append(indexDirPath);
+ builder.append(", generation=");
+ builder.append(generationNumber);
+ builder.append("]");
+ return builder.toString();
+}
+ }
+
+ /** Prefix used for the save file. */
+ public static final String SNAPSHOTS_PREFIX = "snapshots_";
+ private static final int VERSION_START = 0;
+ private static final int VERSION_CURRENT = VERSION_START;
+ private static
[GitHub] lucene-solr pull request #52: [SOLR-9269] Ability to create/delete/list snap...
Github user dsmiley commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/52#discussion_r71727257
--- Diff:
solr/core/src/java/org/apache/solr/core/snapshots/SolrSnapshotMetaDataManager.java
---
@@ -0,0 +1,419 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.core.snapshots;
+
+import java.io.IOException;
+import java.lang.invoke.MethodHandles;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Optional;
+import java.util.stream.Collectors;
+
+import org.apache.lucene.codecs.CodecUtil;
+import org.apache.lucene.index.DirectoryReader;
+import org.apache.lucene.index.IndexCommit;
+import org.apache.lucene.index.IndexDeletionPolicy;
+import org.apache.lucene.index.IndexWriterConfig.OpenMode;
+import org.apache.lucene.store.Directory;
+import org.apache.lucene.store.IOContext;
+import org.apache.lucene.store.IndexInput;
+import org.apache.lucene.store.IndexOutput;
+import org.apache.lucene.util.IOUtils;
+import org.apache.solr.common.SolrException;
+import org.apache.solr.common.SolrException.ErrorCode;
+import org.apache.solr.core.DirectoryFactory;
+import org.apache.solr.core.IndexDeletionPolicyWrapper;
+import org.apache.solr.core.SolrCore;
+import org.apache.solr.core.DirectoryFactory.DirContext;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This class is responsible to manage the persistent snapshots meta-data
for the Solr indexes. The
+ * persistent snapshots are implemented by relying on Lucene {@linkplain
IndexDeletionPolicy}
+ * abstraction to configure a specific {@linkplain IndexCommit} to be
retained. The
+ * {@linkplain IndexDeletionPolicyWrapper} in Solr uses this class to
create/delete the Solr index
+ * snapshots.
+ */
+public class SolrSnapshotMetaDataManager {
+ private static final Logger log =
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
+ public static final String SNAPSHOT_METADATA_DIR = "snapshot_metadata";
+
+ /**
+ * A class defining the meta-data for a specific snapshot.
+ */
+ public static class SnapshotMetaData {
+private String name;
+private String indexDirPath;
+private long generationNumber;
+
+public SnapshotMetaData(String name, String indexDirPath, long
generationNumber) {
+ super();
+ this.name = name;
+ this.indexDirPath = indexDirPath;
+ this.generationNumber = generationNumber;
+}
+
+public String getName() {
+ return name;
+}
+
+public String getIndexDirPath() {
+ return indexDirPath;
+}
+
+public long getGenerationNumber() {
+ return generationNumber;
+}
+
+@Override
+public String toString() {
+ StringBuilder builder = new StringBuilder();
+ builder.append("SnapshotMetaData[name=");
+ builder.append(name);
+ builder.append(", indexDirPath=");
+ builder.append(indexDirPath);
+ builder.append(", generation=");
+ builder.append(generationNumber);
+ builder.append("]");
+ return builder.toString();
+}
+ }
+
+ /** Prefix used for the save file. */
+ public static final String SNAPSHOTS_PREFIX = "snapshots_";
+ private static final int VERSION_START = 0;
+ private static final int VERSION_CURRENT = VERSION_START;
+ private static final String CODEC_NAME = "solr-snapshots";
+
+ // The index writer which maintains the snapshots metadata
+ private long nextWriteGen;
+
+ private final Directory dir;
+
+ /** Used to map snapshot name to
[jira] [Commented] (SOLR-9269) Ability to create/delete/list snapshots for a solr core
[
https://issues.apache.org/jira/browse/SOLR-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15387878#comment-15387878
]
ASF GitHub Bot commented on SOLR-9269:
--
Github user dsmiley commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/52#discussion_r71725843
--- Diff:
solr/core/src/java/org/apache/solr/handler/admin/CoreAdminOperation.java ---
@@ -919,7 +928,93 @@ public void call(CallInfo callInfo) throws Exception {
}
}
}
+ },
+ CREATESNAPSHOT_OP(CREATESNAPSHOT) {
+@Override
+public void call(CallInfo callInfo) throws Exception {
+ CoreContainer cc = callInfo.handler.getCoreContainer();
+ final SolrParams params = callInfo.req.getParams();
+
+ String commitName =
params.required().get(CoreAdminParams.COMMIT_NAME);
+ String cname = params.required().get(CoreAdminParams.CORE);
+ try (SolrCore core = cc.getCore(cname)) {
+if (core == null) {
+ throw new SolrException(ErrorCode.SERVER_ERROR, "Unable to
locate core " + cname);
+}
+if (core.getDeletionPolicy().getLatestCommit() == null) {
+ throw new SolrException(ErrorCode.SERVER_ERROR, "Could not find
latest commit. Please ensure to execute a hard commit");
+}
+
+String indexDirPath = core.getIndexDir();
+IndexCommit ic = core.getDeletionPolicy().getLatestCommit();
+SolrSnapshotMetaDataManager mgr =
core.getSnapshotMetaDataManager();
+mgr.snapshot(commitName, indexDirPath, ic.getGeneration());
+ }
+}
+ },
+ DELETESNAPSHOT_OP(DELETESNAPSHOT) {
+@Override
+public void call(CallInfo callInfo) throws Exception {
+ CoreContainer cc = callInfo.handler.getCoreContainer();
+ final SolrParams params = callInfo.req.getParams();
+
+ String commitName =
params.required().get(CoreAdminParams.COMMIT_NAME);
+ String cname = params.required().get(CoreAdminParams.CORE);
+ try (SolrCore core = cc.getCore(cname)) {
+if (core == null) {
+ throw new SolrException(ErrorCode.SERVER_ERROR, "Unable to
locate core " + cname);
+}
+
+SolrSnapshotMetaDataManager mgr =
core.getSnapshotMetaDataManager();
+Optional metadata = mgr.release(commitName);
+if (metadata.isPresent()) {
--- End diff --
Perhaps add an error if we can't find the commit name? REQUEST_ERROR. And
FYI some other exceptions we throw should also be REQUEST_ERROR if it's based
on a param.
> Ability to create/delete/list snapshots for a solr core
> ---
>
> Key: SOLR-9269
> URL: https://issues.apache.org/jira/browse/SOLR-9269
> Project: Solr
> Issue Type: Sub-task
> Components: SolrCloud
>Reporter: Hrishikesh Gadre
>Assignee: David Smiley
> Attachments: SOLR-9269.patch
>
>
> Support snapshot create/delete/list functionality @ the Solr core level.
> Please refer to parent JIRA for more details.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9269) Ability to create/delete/list snapshots for a solr core
[
https://issues.apache.org/jira/browse/SOLR-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15387879#comment-15387879
]
ASF GitHub Bot commented on SOLR-9269:
--
Github user dsmiley commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/52#discussion_r71725905
--- Diff:
solr/core/src/java/org/apache/solr/handler/admin/CoreAdminOperation.java ---
@@ -919,7 +928,93 @@ public void call(CallInfo callInfo) throws Exception {
}
}
}
+ },
+ CREATESNAPSHOT_OP(CREATESNAPSHOT) {
+@Override
+public void call(CallInfo callInfo) throws Exception {
+ CoreContainer cc = callInfo.handler.getCoreContainer();
+ final SolrParams params = callInfo.req.getParams();
+
+ String commitName =
params.required().get(CoreAdminParams.COMMIT_NAME);
+ String cname = params.required().get(CoreAdminParams.CORE);
+ try (SolrCore core = cc.getCore(cname)) {
+if (core == null) {
+ throw new SolrException(ErrorCode.SERVER_ERROR, "Unable to
locate core " + cname);
+}
+if (core.getDeletionPolicy().getLatestCommit() == null) {
+ throw new SolrException(ErrorCode.SERVER_ERROR, "Could not find
latest commit. Please ensure to execute a hard commit");
+}
+
+String indexDirPath = core.getIndexDir();
+IndexCommit ic = core.getDeletionPolicy().getLatestCommit();
+SolrSnapshotMetaDataManager mgr =
core.getSnapshotMetaDataManager();
+mgr.snapshot(commitName, indexDirPath, ic.getGeneration());
+ }
+}
+ },
+ DELETESNAPSHOT_OP(DELETESNAPSHOT) {
+@Override
+public void call(CallInfo callInfo) throws Exception {
+ CoreContainer cc = callInfo.handler.getCoreContainer();
+ final SolrParams params = callInfo.req.getParams();
+
+ String commitName =
params.required().get(CoreAdminParams.COMMIT_NAME);
+ String cname = params.required().get(CoreAdminParams.CORE);
+ try (SolrCore core = cc.getCore(cname)) {
+if (core == null) {
+ throw new SolrException(ErrorCode.SERVER_ERROR, "Unable to
locate core " + cname);
+}
+
+SolrSnapshotMetaDataManager mgr =
core.getSnapshotMetaDataManager();
+Optional metadata = mgr.release(commitName);
+if (metadata.isPresent()) {
+ long gen = metadata.get().getGenerationNumber();
+ String indexDirPath = metadata.get().getIndexDirPath();
+
+ // If the directory storing the snapshot is not the same as the
*current* core
--- End diff --
This explanation helps; thanks
> Ability to create/delete/list snapshots for a solr core
> ---
>
> Key: SOLR-9269
> URL: https://issues.apache.org/jira/browse/SOLR-9269
> Project: Solr
> Issue Type: Sub-task
> Components: SolrCloud
>Reporter: Hrishikesh Gadre
>Assignee: David Smiley
> Attachments: SOLR-9269.patch
>
>
> Support snapshot create/delete/list functionality @ the Solr core level.
> Please refer to parent JIRA for more details.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] lucene-solr pull request #52: [SOLR-9269] Ability to create/delete/list snap...
Github user dsmiley commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/52#discussion_r71725905
--- Diff:
solr/core/src/java/org/apache/solr/handler/admin/CoreAdminOperation.java ---
@@ -919,7 +928,93 @@ public void call(CallInfo callInfo) throws Exception {
}
}
}
+ },
+ CREATESNAPSHOT_OP(CREATESNAPSHOT) {
+@Override
+public void call(CallInfo callInfo) throws Exception {
+ CoreContainer cc = callInfo.handler.getCoreContainer();
+ final SolrParams params = callInfo.req.getParams();
+
+ String commitName =
params.required().get(CoreAdminParams.COMMIT_NAME);
+ String cname = params.required().get(CoreAdminParams.CORE);
+ try (SolrCore core = cc.getCore(cname)) {
+if (core == null) {
+ throw new SolrException(ErrorCode.SERVER_ERROR, "Unable to
locate core " + cname);
+}
+if (core.getDeletionPolicy().getLatestCommit() == null) {
+ throw new SolrException(ErrorCode.SERVER_ERROR, "Could not find
latest commit. Please ensure to execute a hard commit");
+}
+
+String indexDirPath = core.getIndexDir();
+IndexCommit ic = core.getDeletionPolicy().getLatestCommit();
+SolrSnapshotMetaDataManager mgr =
core.getSnapshotMetaDataManager();
+mgr.snapshot(commitName, indexDirPath, ic.getGeneration());
+ }
+}
+ },
+ DELETESNAPSHOT_OP(DELETESNAPSHOT) {
+@Override
+public void call(CallInfo callInfo) throws Exception {
+ CoreContainer cc = callInfo.handler.getCoreContainer();
+ final SolrParams params = callInfo.req.getParams();
+
+ String commitName =
params.required().get(CoreAdminParams.COMMIT_NAME);
+ String cname = params.required().get(CoreAdminParams.CORE);
+ try (SolrCore core = cc.getCore(cname)) {
+if (core == null) {
+ throw new SolrException(ErrorCode.SERVER_ERROR, "Unable to
locate core " + cname);
+}
+
+SolrSnapshotMetaDataManager mgr =
core.getSnapshotMetaDataManager();
+Optional metadata = mgr.release(commitName);
+if (metadata.isPresent()) {
+ long gen = metadata.get().getGenerationNumber();
+ String indexDirPath = metadata.get().getIndexDirPath();
+
+ // If the directory storing the snapshot is not the same as the
*current* core
--- End diff --
This explanation helps; thanks
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] lucene-solr pull request #52: [SOLR-9269] Ability to create/delete/list snap...
Github user dsmiley commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/52#discussion_r71725843
--- Diff:
solr/core/src/java/org/apache/solr/handler/admin/CoreAdminOperation.java ---
@@ -919,7 +928,93 @@ public void call(CallInfo callInfo) throws Exception {
}
}
}
+ },
+ CREATESNAPSHOT_OP(CREATESNAPSHOT) {
+@Override
+public void call(CallInfo callInfo) throws Exception {
+ CoreContainer cc = callInfo.handler.getCoreContainer();
+ final SolrParams params = callInfo.req.getParams();
+
+ String commitName =
params.required().get(CoreAdminParams.COMMIT_NAME);
+ String cname = params.required().get(CoreAdminParams.CORE);
+ try (SolrCore core = cc.getCore(cname)) {
+if (core == null) {
+ throw new SolrException(ErrorCode.SERVER_ERROR, "Unable to
locate core " + cname);
+}
+if (core.getDeletionPolicy().getLatestCommit() == null) {
+ throw new SolrException(ErrorCode.SERVER_ERROR, "Could not find
latest commit. Please ensure to execute a hard commit");
+}
+
+String indexDirPath = core.getIndexDir();
+IndexCommit ic = core.getDeletionPolicy().getLatestCommit();
+SolrSnapshotMetaDataManager mgr =
core.getSnapshotMetaDataManager();
+mgr.snapshot(commitName, indexDirPath, ic.getGeneration());
+ }
+}
+ },
+ DELETESNAPSHOT_OP(DELETESNAPSHOT) {
+@Override
+public void call(CallInfo callInfo) throws Exception {
+ CoreContainer cc = callInfo.handler.getCoreContainer();
+ final SolrParams params = callInfo.req.getParams();
+
+ String commitName =
params.required().get(CoreAdminParams.COMMIT_NAME);
+ String cname = params.required().get(CoreAdminParams.CORE);
+ try (SolrCore core = cc.getCore(cname)) {
+if (core == null) {
+ throw new SolrException(ErrorCode.SERVER_ERROR, "Unable to
locate core " + cname);
+}
+
+SolrSnapshotMetaDataManager mgr =
core.getSnapshotMetaDataManager();
+Optional metadata = mgr.release(commitName);
+if (metadata.isPresent()) {
--- End diff --
Perhaps add an error if we can't find the commit name? REQUEST_ERROR. And
FYI some other exceptions we throw should also be REQUEST_ERROR if it's based
on a param.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9269) Ability to create/delete/list snapshots for a solr core
[
https://issues.apache.org/jira/browse/SOLR-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15387872#comment-15387872
]
ASF GitHub Bot commented on SOLR-9269:
--
Github user dsmiley commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/52#discussion_r71725010
--- Diff:
solr/core/src/java/org/apache/solr/handler/admin/CoreAdminOperation.java ---
@@ -919,7 +928,93 @@ public void call(CallInfo callInfo) throws Exception {
}
}
}
+ },
+ CREATESNAPSHOT_OP(CREATESNAPSHOT) {
+@Override
+public void call(CallInfo callInfo) throws Exception {
+ CoreContainer cc = callInfo.handler.getCoreContainer();
+ final SolrParams params = callInfo.req.getParams();
+
+ String commitName =
params.required().get(CoreAdminParams.COMMIT_NAME);
+ String cname = params.required().get(CoreAdminParams.CORE);
+ try (SolrCore core = cc.getCore(cname)) {
+if (core == null) {
+ throw new SolrException(ErrorCode.SERVER_ERROR, "Unable to
locate core " + cname);
+}
+if (core.getDeletionPolicy().getLatestCommit() == null) {
--- End diff --
Firstly, this check can go below after we get it on line 949; why get it
twice. Secondly... I wonder if it's null in an initial empty-index case? If
it is, this error message is wrong/confusing... and perhaps we can support that
or no?
> Ability to create/delete/list snapshots for a solr core
> ---
>
> Key: SOLR-9269
> URL: https://issues.apache.org/jira/browse/SOLR-9269
> Project: Solr
> Issue Type: Sub-task
> Components: SolrCloud
>Reporter: Hrishikesh Gadre
>Assignee: David Smiley
> Attachments: SOLR-9269.patch
>
>
> Support snapshot create/delete/list functionality @ the Solr core level.
> Please refer to parent JIRA for more details.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] lucene-solr pull request #52: [SOLR-9269] Ability to create/delete/list snap...
Github user dsmiley commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/52#discussion_r71725010
--- Diff:
solr/core/src/java/org/apache/solr/handler/admin/CoreAdminOperation.java ---
@@ -919,7 +928,93 @@ public void call(CallInfo callInfo) throws Exception {
}
}
}
+ },
+ CREATESNAPSHOT_OP(CREATESNAPSHOT) {
+@Override
+public void call(CallInfo callInfo) throws Exception {
+ CoreContainer cc = callInfo.handler.getCoreContainer();
+ final SolrParams params = callInfo.req.getParams();
+
+ String commitName =
params.required().get(CoreAdminParams.COMMIT_NAME);
+ String cname = params.required().get(CoreAdminParams.CORE);
+ try (SolrCore core = cc.getCore(cname)) {
+if (core == null) {
+ throw new SolrException(ErrorCode.SERVER_ERROR, "Unable to
locate core " + cname);
+}
+if (core.getDeletionPolicy().getLatestCommit() == null) {
--- End diff --
Firstly, this check can go below after we get it on line 949; why get it
twice. Secondly... I wonder if it's null in an initial empty-index case? If
it is, this error message is wrong/confusing... and perhaps we can support that
or no?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
