Will White created SOLR-16813:
---------------------------------
Summary: PackageLoader not copying manifest.json file to filestore
Key: SOLR-16813
URL: https://issues.apache.org/jira/browse/SOLR-16813
Project: Solr
Issue Type: Bug
Security Level: Public (Default Security Level. Issues are Public)
Components: Package Manager
Affects Versions: 9.2.1, 9.2
Reporter: Will White
When a new Solr node enters the cluster and the package management system
recreates the package from the source node, the filestore location is set up
and the JAR files are copied across correctly, but the {{manifest.json}} file
isn't (and the {{.manifest.json.json}} file isn't generated). This means that
when trying to interact with the package (such as {{bin/solr package
list-installed}}) the command errors out because of the missing expected file.
This can be reproduced by:
* Setup Zookeeper cluster
* Create two solr nodes ("solr1" and "solr2") in cluster
* Install a package on solr1
** {{bin/solr package add-repo data-import-handler
"https://raw.githubusercontent.com/searchscale/dataimporthandler/master/repo"}}
** {{bin/solr package install data-import-handler}}
** {{find /var/solr/data/filestore}} ({{manifest.json}} and
{{.manifest.json.json}} should exist)
** {{bin/solr package list-installed}}
* At this point, the package exists as expected on solr2
* Stop solr1 and solr2
* Restart the cluster, now with an additional solr3 node
* solr3 will have the JAR files and their {{.json}} files, but the
{{manifest.json}} file won't be present
** Running {{bin/solr package list-installed}} will throw an exception
----
The difference appears to be that the
{{org.apache.solr.packagemanager.RepositoryManager::installPackage}} method
contains the following lines:
{code:java}
// org/apache/solr/packagemanager/RepositoryManager.java#182
if (release.manifest == null) {
String manifestJson = PackageUtils.getFileFromJarsAsString(downloaded,
"manifest.json");
if (manifestJson == null) {
throw new SolrException(ErrorCode.NOT_FOUND, "No manifest found for
package: " + packageName + ", version: " + version);
}
release.manifest = getMapper().readValue(manifestJson,
SolrPackage.Manifest.class);
}
String manifestJson = getMapper().writeValueAsString(release.manifest);
// We go on to write this file later
{code}
But in the equivalent space in
{{org.apache.solr.pkg.SolrPackageLoader.Version()}}, the equivalent section to
fetch the JAR files if missing doesn't reference the version.manifest anywhere:
{code:java}
// org/apache/solr/pkg/SolrPackageLoader.java#276
coreContainer.getPackageStoreAPI().validateFiles(version.files, true, s ->
errs.add(s));
if (!errs.isEmpty()) {
...
}
for (String file : version.files) {
paths.add(coreContainer.getPackageStoreAPI().getPackageStore().getRealpath(file));
}
{code}
Something like [this
commit|https://github.com/apache/solr/commit/9fc5253eef573dc5e53edf52d98d4a2e55e00e1e]
solves the problem by manually copying the file across, but I'm not sure if
this is the best approach to take.
I'm happy to put a full PR together, although I'd appreciate it if someone
could confirm whether the {{manifest.json}} file should be transferred from the
source node like this (and can be treated as mandatory), or if I've
misunderstood how this should be working.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
