This is an automated email from the ASF dual-hosted git repository.
janhoy pushed a commit to branch branch_9x
in repository https://gitbox.apache.org/repos/asf/solr.git
The following commit(s) were added to refs/heads/branch_9x by this push:
new 5232976 SOLR-9575 Allow starting with an empty SOLR_HOME (#558)
5232976 is described below
commit 523297699b863bcbe89309b625b27dcf0944e82f
Author: Jan Høydahl <[email protected]>
AuthorDate: Sat Jan 29 16:37:03 2022 +0100
SOLR-9575 Allow starting with an empty SOLR_HOME (#558)
Solr no longer requires solr.xml and zoo.cfg files be present in
$SOLR_HOME. If not found, defaults are used
(cherry picked from commit 0fb94f7b38e7a7b735ce97ad8d19412e5fbd1f93)
---
solr/CHANGES.txt | 4 ++
solr/bin/solr | 5 +-
solr/bin/solr.cmd | 10 +++-
solr/bin/solr.in.cmd | 4 ++
solr/bin/solr.in.sh | 4 ++
.../java/org/apache/solr/cloud/SolrZkServer.java | 43 +++++++++--------
.../java/org/apache/solr/core/SolrXmlConfig.java | 56 ++++++++++++----------
solr/docker/scripts/init-var-solr | 10 ----
solr/solr-ref-guide/src/configuring-solr-xml.adoc | 15 ++++--
.../src/major-changes-in-solr-9.adoc | 4 +-
.../src/solr-control-script-reference.adoc | 2 -
11 files changed, 95 insertions(+), 62 deletions(-)
diff --git a/solr/CHANGES.txt b/solr/CHANGES.txt
index 55c7715..61174f1 100644
--- a/solr/CHANGES.txt
+++ b/solr/CHANGES.txt
@@ -252,6 +252,10 @@ when told to. The admin UI now tells it to. (Nazerke
Seidan, David Smiley)
* SOLR-15755: Parse collection, shard, core, replica info from newSearcher
events in postlogs tool (Joel Bernstein)
+* SOLR-9575: Solr no longer requires a solr.xml to be present in $SOLR_HOME.
If one is not found, Solr will load
+ the one in $SOLR_TIP/server/solr/solr.xml. Solr also does not require a
zoo.cfg in $SOLR_HOME if started with
+ embedded zookeeper. (janhoy)
+
* SOLR-11905: New Admin UI Query screen input boxes for JSON query and facet
DSL (janhoy)
Build
diff --git a/solr/bin/solr b/solr/bin/solr
index 898648e..3a6f0e4 100755
--- a/solr/bin/solr
+++ b/solr/bin/solr
@@ -2069,8 +2069,11 @@ if [ "$SOLR_MODE" == 'solrcloud' ]; then
CLOUD_MODE_OPTS+=('-Dbootstrap_confdir=./solr/collection1/conf'
'-Dcollection.configName=myconf' '-DnumShards=1')
fi
+ if [ "$SOLR_SOLRXML_REQUIRED" == "true" ]; then
+ CLOUD_MODE_OPTS+=("-Dsolr.solrxml.required=true")
+ fi
else
- if [ ! -e "$SOLR_HOME/solr.xml" ]; then
+ if [ ! -e "$SOLR_HOME/solr.xml" ] && [ "$SOLR_SOLRXML_REQUIRED" == "true" ];
then
echo -e "\nSolr home directory $SOLR_HOME must contain a solr.xml file!\n"
exit 1
fi
diff --git a/solr/bin/solr.cmd b/solr/bin/solr.cmd
index af25f9a..e2b0c6c 100755
--- a/solr/bin/solr.cmd
+++ b/solr/bin/solr.cmd
@@ -1163,12 +1163,18 @@ IF "%SOLR_MODE%"=="solrcloud" (
set "CLOUD_MODE_OPTS=!CLOUD_MODE_OPTS! -DcreateZkChroot=%ZK_CREATE_CHROOT%"
)
+ IF "%SOLR_SOLRXML_REQUIRED%"=="true" (
+ set "CLOUD_MODE_OPTS=!CLOUD_MODE_OPTS! -Dsolr.solrxml.required=true"
+ )
+
IF EXIST "%SOLR_HOME%\collection1\core.properties" set
"CLOUD_MODE_OPTS=!CLOUD_MODE_OPTS! -Dbootstrap_confdir=./solr/collection1/conf
-Dcollection.configName=myconf -DnumShards=1"
) ELSE (
set CLOUD_MODE_OPTS=
IF NOT EXIST "%SOLR_HOME%\solr.xml" (
- set "SCRIPT_ERROR=Solr home directory %SOLR_HOME% must contain solr.xml!"
- goto err
+ IF "%SOLR_SOLRXML_REQUIRED%"=="true" (
+ set "SCRIPT_ERROR=Solr home directory %SOLR_HOME% must contain solr.xml!"
+ goto err
+ )
)
)
diff --git a/solr/bin/solr.in.cmd b/solr/bin/solr.in.cmd
index ca411f9..c554be2 100755
--- a/solr/bin/solr.in.cmd
+++ b/solr/bin/solr.in.cmd
@@ -215,6 +215,10 @@ REM Sometimes it may be necessary to place a core or a
backup on a different loc
REM This parameter lets you specify file system path(s) to explicitly allow.
The special value of '*' will allow any path
REM set SOLR_OPTS=%SOLR_OPTS% -Dsolr.allowPaths=D:\,E:\other\path
+REM Before version 9.0, Solr required a copy of solr.xml file in $SOLR_HOME.
Now Solr will use a default file if not found.
+REM To restore the old behaviour, set the variable below to true
+REM set SOLR_SOLRXML_REQUIRED=false
+
REM Some previous versions of Solr use an outdated log4j dependency. If you
are unable to use at least log4j version 2.15.0
REM then enable the following setting to address CVE-2021-44228
REM set SOLR_OPTS=%SOLR_OPTS% -Dlog4j2.formatMsgNoLookups=true
diff --git a/solr/bin/solr.in.sh b/solr/bin/solr.in.sh
index 37e2ff8..aaea9c8 100644
--- a/solr/bin/solr.in.sh
+++ b/solr/bin/solr.in.sh
@@ -259,6 +259,10 @@
#SOLR_HEAP_DUMP=true
#SOLR_HEAP_DUMP_DIR=/var/log/dumps
+# Before version 9.0, Solr required a copy of solr.xml file in $SOLR_HOME. Now
Solr will use a default file if not found.
+# To restore the old behaviour, set the variable below to true
+#SOLR_SOLRXML_REQUIRED=false
+
# Some previous versions of Solr use an outdated log4j dependency. If you are
unable to use at least log4j version 2.15.0
# then enable the following setting to address CVE-2021-44228
# SOLR_OPTS="$SOLR_OPTS -Dlog4j2.formatMsgNoLookups=true"
diff --git a/solr/core/src/java/org/apache/solr/cloud/SolrZkServer.java
b/solr/core/src/java/org/apache/solr/cloud/SolrZkServer.java
index ccb4315..1b1b42a 100644
--- a/solr/core/src/java/org/apache/solr/cloud/SolrZkServer.java
+++ b/solr/core/src/java/org/apache/solr/cloud/SolrZkServer.java
@@ -17,6 +17,7 @@
package org.apache.solr.cloud;
import org.apache.solr.common.SolrException;
+import org.apache.solr.servlet.SolrDispatchFilter;
import org.apache.zookeeper.server.ServerConfig;
import org.apache.zookeeper.server.ZooKeeperServerMain;
import org.apache.zookeeper.server.quorum.QuorumPeer;
@@ -37,8 +38,6 @@ import java.nio.file.Path;
import java.util.Map;
import java.util.Properties;
import java.util.concurrent.atomic.AtomicReference;
-import java.util.regex.Pattern;
-
public class SolrZkServer {
private static final Logger log =
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
@@ -84,9 +83,20 @@ public class SolrZkServer {
zkProps.zkRun = zkRun;
zkProps.solrPort = Integer.toString(solrPort);
}
-
+
+ var zooCfgPath = Path.of(confHome).resolve("zoo.cfg");
+ if (!Files.exists(zooCfgPath)) {
+ log.info("Zookeeper configuration not found in {}, using built-in
default", confHome);
+ String solrInstallDir =
System.getProperty(SolrDispatchFilter.SOLR_INSTALL_DIR_ATTRIBUTE);
+ if (solrInstallDir == null) {
+ throw new SolrException(SolrException.ErrorCode.SERVER_ERROR,
+ "Could not find default zoo.cfg file due to missing " +
SolrDispatchFilter.SOLR_INSTALL_DIR_ATTRIBUTE);
+ }
+ zooCfgPath =
Path.of(solrInstallDir).resolve("server").resolve("solr").resolve("zoo.cfg");
+ }
+
try {
- props = SolrZkServerProps.getProperties(confHome + '/' + "zoo.cfg");
+ props = SolrZkServerProps.getProperties(zooCfgPath);
SolrZkServerProps.injectServers(props, zkRun, zkHost);
if (props.getProperty("clientPort") == null) {
props.setProperty("clientPort", Integer.toString(solrPort + 1000));
@@ -168,34 +178,29 @@ public class SolrZkServer {
// zoo.cfg (which validates that there is a dataDir)
class SolrZkServerProps extends QuorumPeerConfig {
private static final Logger log =
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
- private static final Pattern MISSING_MYID_FILE_PATTERN =
Pattern.compile(".*myid file is missing$");
String solrPort; // port that Solr is listening on
String zkRun;
/**
* Parse a ZooKeeper configuration file
- * @param path the patch of the configuration file
+ * @param configPath the path of the configuration file
* @throws IllegalArgumentException if a config file does not exist at the
given path
* @throws ConfigException error processing configuration
*/
- public static Properties getProperties(String path) throws ConfigException {
- Path configPath = Path.of(path);
+ public static Properties getProperties(Path configPath) throws
ConfigException {
log.info("Reading configuration from: {}", configPath);
- try {
- if (!Files.exists(configPath)) {
- throw new IllegalArgumentException(configPath.toString()
- + " file is missing");
- }
+ if (!Files.exists(configPath)) {
+ throw new IllegalArgumentException(configPath + " file is missing");
+ }
- try (Reader reader = Files.newBufferedReader(configPath)) {
- Properties cfg = new Properties();
- cfg.load(reader);
- return cfg;
- }
+ try (Reader reader = Files.newBufferedReader(configPath)) {
+ Properties cfg = new Properties();
+ cfg.load(reader);
+ return cfg;
} catch (IOException | IllegalArgumentException e) {
- throw new ConfigException("Error processing " + path, e);
+ throw new ConfigException("Error processing " + configPath, e);
}
}
diff --git a/solr/core/src/java/org/apache/solr/core/SolrXmlConfig.java
b/solr/core/src/java/org/apache/solr/core/SolrXmlConfig.java
index f3e5581..48f23be 100644
--- a/solr/core/src/java/org/apache/solr/core/SolrXmlConfig.java
+++ b/solr/core/src/java/org/apache/solr/core/SolrXmlConfig.java
@@ -16,6 +16,26 @@
*/
package org.apache.solr.core;
+import com.google.common.base.Strings;
+import org.apache.commons.io.IOUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.solr.client.solrj.impl.HttpClientUtil;
+import org.apache.solr.common.SolrException;
+import org.apache.solr.common.util.DOMUtil;
+import org.apache.solr.common.util.NamedList;
+import org.apache.solr.common.util.PropertiesUtil;
+import org.apache.solr.common.util.Utils;
+import org.apache.solr.logging.LogWatcherConfig;
+import org.apache.solr.metrics.reporters.SolrJmxReporter;
+import org.apache.solr.servlet.SolrDispatchFilter;
+import org.apache.solr.update.UpdateShardHandlerConfig;
+import org.apache.solr.util.JmxUtil;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.w3c.dom.Node;
+import org.w3c.dom.NodeList;
+import org.xml.sax.InputSource;
+
import javax.management.MBeanServer;
import javax.xml.xpath.XPath;
import javax.xml.xpath.XPathConstants;
@@ -37,25 +57,6 @@ import java.util.Properties;
import java.util.Set;
import java.util.regex.Pattern;
-import com.google.common.base.Strings;
-import org.apache.commons.io.IOUtils;
-import org.apache.commons.lang3.StringUtils;
-import org.apache.solr.client.solrj.impl.HttpClientUtil;
-import org.apache.solr.common.SolrException;
-import org.apache.solr.common.util.DOMUtil;
-import org.apache.solr.common.util.NamedList;
-import org.apache.solr.common.util.PropertiesUtil;
-import org.apache.solr.common.util.Utils;
-import org.apache.solr.logging.LogWatcherConfig;
-import org.apache.solr.metrics.reporters.SolrJmxReporter;
-import org.apache.solr.update.UpdateShardHandlerConfig;
-import org.apache.solr.util.JmxUtil;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-import org.w3c.dom.Node;
-import org.w3c.dom.NodeList;
-import org.xml.sax.InputSource;
-
import static org.apache.solr.common.params.CommonParams.NAME;
@@ -161,14 +162,21 @@ public class SolrXmlConfig {
}
public static NodeConfig fromFile(Path solrHome, Path configFile, Properties
substituteProps) {
-
- log.info("Loading container configuration from {}", configFile);
-
if (!Files.exists(configFile)) {
- throw new SolrException(SolrException.ErrorCode.SERVER_ERROR,
- "solr.xml does not exist in " + configFile.getParent() + " cannot
start Solr");
+ if (Boolean.getBoolean("solr.solrxml.required")) {
+ throw new SolrException(SolrException.ErrorCode.SERVER_ERROR,
+ "solr.xml does not exist in " + configFile.getParent() + " cannot
start Solr");
+ }
+ log.info("solr.xml not found in SOLR_HOME, using built-in default");
+ String solrInstallDir =
System.getProperty(SolrDispatchFilter.SOLR_INSTALL_DIR_ATTRIBUTE);
+ if (solrInstallDir == null) {
+ throw new SolrException(SolrException.ErrorCode.SERVER_ERROR,
+ "Could not find default solr.xml file due to missing " +
SolrDispatchFilter.SOLR_INSTALL_DIR_ATTRIBUTE);
+ }
+ configFile =
Path.of(solrInstallDir).resolve("server").resolve("solr").resolve("solr.xml");
}
+ log.info("Loading solr.xml from {}", configFile);
try (InputStream inputStream = Files.newInputStream(configFile)) {
return fromInputStream(solrHome, inputStream, substituteProps);
} catch (SolrException exc) {
diff --git a/solr/docker/scripts/init-var-solr
b/solr/docker/scripts/init-var-solr
index 64f73a4..3c6e25d 100755
--- a/solr/docker/scripts/init-var-solr
+++ b/solr/docker/scripts/init-var-solr
@@ -56,16 +56,6 @@ if [ ! -d "$DIR/logs" ]; then
mkdir -m0770 "$DIR/logs"
fi
-if [ ! -f "$DIR/data/solr.xml" ]; then
- #echo "Copying solr.xml"
- cp -a /opt/solr/server/solr/solr.xml "$DIR/data/solr.xml"
-fi
-
-if [ ! -f "$DIR/data/zoo.cfg" ]; then
- #echo "Copying zoo.cfg"
- cp -a /opt/solr/server/solr/zoo.cfg "$DIR/data/zoo.cfg"
-fi
-
if [ ! -f "$DIR/log4j2.xml" ]; then
#echo "Copying log4j2.xml"
cp -a /opt/solr/server/resources/log4j2.xml "$DIR/log4j2.xml"
diff --git a/solr/solr-ref-guide/src/configuring-solr-xml.adoc
b/solr/solr-ref-guide/src/configuring-solr-xml.adoc
index 306d834..84246e2 100644
--- a/solr/solr-ref-guide/src/configuring-solr-xml.adoc
+++ b/solr/solr-ref-guide/src/configuring-solr-xml.adoc
@@ -23,8 +23,9 @@ For details on how to configure `core.properties`, see the
section <<core-discov
== Defining solr.xml
-You can find `solr.xml` in your `$SOLR_HOME` directory (usually `server/solr`
or `/var/solr/data`) or optionally in ZooKeeper when using SolrCloud.
-The default `solr.xml` file looks like this:
+You can find `solr.xml` in your `$SOLR_HOME` directory (usually `server/solr`
or `/var/solr/data`) or optionally in ZooKeeper when using SolrCloud. If
`$SOLR_HOME/solr.xml` is not found, Solr will use the default `solr.xml` file.
+
+The default `solr.xml` file is found in `$SOLR_TIP/server/solr/solr.xml` and
looks like this:
[source,xml]
----
@@ -36,16 +37,21 @@ The default `solr.xml` file looks like this:
<str name="allowUrls">${solr.allowUrls:}</str>
<solrcloud>
+
<str name="host">${host:}</str>
<int name="hostPort">${solr.port.advertise:0}</int>
<str name="hostContext">${hostContext:solr}</str>
+
<bool name="genericCoreNodeNames">${genericCoreNodeNames:true}</bool>
+
<int name="zkClientTimeout">${zkClientTimeout:30000}</int>
<int name="distribUpdateSoTimeout">${distribUpdateSoTimeout:600000}</int>
<int
name="distribUpdateConnTimeout">${distribUpdateConnTimeout:60000}</int>
<str
name="zkCredentialsProvider">${zkCredentialsProvider:org.apache.solr.common.cloud.DefaultZkCredentialsProvider}</str>
<str
name="zkACLProvider">${zkACLProvider:org.apache.solr.common.cloud.DefaultZkACLProvider}</str>
<bool
name="distributedClusterStateUpdates">${distributedClusterStateUpdates:false}</bool>
+ <bool
name="distributedCollectionConfigSetExecution">${distributedCollectionConfigSetExecution:false}</bool>
+
</solrcloud>
<shardHandlerFactory name="shardHandlerFactory"
@@ -54,6 +60,10 @@ The default `solr.xml` file looks like this:
<int name="connTimeout">${connTimeout:60000}</int>
</shardHandlerFactory>
+ <metrics enabled="${metricsEnabled:true}">
+ <!--reporter name="jmx_metrics" group="core"
class="org.apache.solr.metrics.reporters.SolrJmxReporter"/-->
+ </metrics>
+
</solr>
----
@@ -543,7 +553,6 @@ If only `dividend` routing is desired, `hash` may be
explicitly set to the empty
The `<metrics>` element in `solr.xml` allows you to customize the metrics
reported by Solr.
You can define system properties that should not be returned, or define custom
suppliers and reporters.
-In a default `solr.xml` you will not see any `<metrics>` configuration.
If you would like to customize the metrics for your installation, see the
section <<metrics-reporting.adoc#metrics-configuration,Metrics Configuration>>.
== Substituting JVM System Properties in solr.xml
diff --git a/solr/solr-ref-guide/src/major-changes-in-solr-9.adoc
b/solr/solr-ref-guide/src/major-changes-in-solr-9.adoc
index 3bb2800..0f03ed0 100644
--- a/solr/solr-ref-guide/src/major-changes-in-solr-9.adoc
+++ b/solr/solr-ref-guide/src/major-changes-in-solr-9.adoc
@@ -159,11 +159,13 @@ wrapped in a `<raw name="field_name">[...]</raw>` element
at the top level of ea
`<arr name="field_name"><raw>[...]</raw></arr>` element for multi-valued
fields). Existing clients that parse field
values serialized in this way will need to be updated accordingly.
+* SOLR-9575: Solr no longer requires a `solr.xml` in `$SOLR_HOME`. If one is
not found, Solr will instead use the default one from
`$SOLR_TIP/server/solr/solr.xml`. You can revert to the pre-9.0 behaviour by
setting environment variable `SOLR_SOLRXML_REQUIRED=true` or system property
`-Dsolr.solrxml.required=true`. Solr also does not require a `zoo.cfg` in
`$SOLR_HOME` if started with embedded zookeeper.
+
=== solr.xml maxBooleanClauses now enforced recursively
Lucene 9.0 has additional safety checks over previous versions that impact how
the `solr.xml` global
`<<configuring-solr-xml#global-maxbooleanclauses,maxBooleanClauses>>` option is
enforced.
-In previous versios of Solr, this option was a hard limit on the number of
clauses in any `BooleanQuery` object - but it was only enforced for the
_direct_ clauses.
+In previous versions of Solr, this option was a hard limit on the number of
clauses in any `BooleanQuery` object - but it was only enforced for the
_direct_ clauses.
Starting with Solr 9, this global limit is now also enforced against the total
number of clauses in a _nested_ query structure.
Users who upgrade from prior versions of Solr may find that some requests
involving complex internal query structures (Example: long query strings using
`edismax` with many `qf` and `pf` fields that include query time synonym
expansion) which worked in the past now hit this limit and fail.
diff --git a/solr/solr-ref-guide/src/solr-control-script-reference.adoc
b/solr/solr-ref-guide/src/solr-control-script-reference.adoc
index d6d7a07..573a398 100644
--- a/solr/solr-ref-guide/src/solr-control-script-reference.adoc
+++ b/solr/solr-ref-guide/src/solr-control-script-reference.adoc
@@ -190,8 +190,6 @@ Sets the `solr.solr.home` system property.
Solr will create core directories under this directory.
This allows you to run multiple Solr instances on the same host while reusing
the same server directory set using the `-d` parameter.
+
-If set, the specified directory should contain a `solr.xml` file, unless
`solr.xml` exists in ZooKeeper.
-+
This parameter is ignored when running examples (`-e`), as the
`solr.solr.home` depends on which example is run.
+
*Example*: `bin/solr start -s newHome`
