[
https://issues.apache.org/jira/browse/SOLR-16813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17725017#comment-17725017
]
Will White commented on SOLR-16813:
-----------------------------------
Brief discussion in -IRC- Slack:
https://apachesolr.slack.com/archives/C01GVPZSSK0/p1684680323785689
Tossed a coin on priority, raised as Major because without manually copying the
{{manifest.json}} file across to the new node, all commands to the new package
break (which I've interpreted as a Major issue on the assumption of how the
package manager might be used in a clustered environment). Happy to downgrade
to a Minor if that's a less common use case than I've supposed.
> PackageLoader not copying manifest.json file to filestore
> ----------------------------------------------------------
>
> Key: SOLR-16813
> URL: https://issues.apache.org/jira/browse/SOLR-16813
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Package Manager
> Affects Versions: 9.2, 9.2.1
> Reporter: Will White
> Priority: Major
>
> When a new Solr node enters the cluster and the package management system
> recreates the package from the source node, the filestore location is set up
> and the JAR files are copied across correctly, but the {{manifest.json}} file
> isn't (and the {{.manifest.json.json}} file isn't generated). This means that
> when trying to interact with the package (such as {{bin/solr package
> list-installed}}) the command errors out because of the missing expected file.
> This can be reproduced by:
> * Setup Zookeeper cluster
> * Create two solr nodes ("solr1" and "solr2") in cluster
> * Install a package on solr1
> ** {{bin/solr package add-repo data-import-handler
> "https://raw.githubusercontent.com/searchscale/dataimporthandler/master/repo"}}
> ** {{bin/solr package install data-import-handler}}
> ** {{find /var/solr/data/filestore}} ({{manifest.json}} and
> {{.manifest.json.json}} should exist)
> ** {{bin/solr package list-installed}}
> * At this point, the package exists as expected on solr2
> * Stop solr1 and solr2
> * Restart the cluster, now with an additional solr3 node
> * solr3 will have the JAR files and their {{.json}} files, but the
> {{manifest.json}} file won't be present
> ** Running {{bin/solr package list-installed}} will throw an exception
> ----
> The difference appears to be that the
> {{org.apache.solr.packagemanager.RepositoryManager::installPackage}} method
> contains the following lines:
> {code:java}
> // org/apache/solr/packagemanager/RepositoryManager.java#182
> if (release.manifest == null) {
> String manifestJson = PackageUtils.getFileFromJarsAsString(downloaded,
> "manifest.json");
> if (manifestJson == null) {
> throw new SolrException(ErrorCode.NOT_FOUND, "No manifest found for
> package: " + packageName + ", version: " + version);
> }
> release.manifest = getMapper().readValue(manifestJson,
> SolrPackage.Manifest.class);
> }
> String manifestJson = getMapper().writeValueAsString(release.manifest);
> // We go on to write this file later
> {code}
> But in the equivalent space in
> {{org.apache.solr.pkg.SolrPackageLoader.Version()}}, the equivalent section
> to fetch the JAR files if missing doesn't reference the version.manifest
> anywhere:
> {code:java}
> // org/apache/solr/pkg/SolrPackageLoader.java#276
> coreContainer.getPackageStoreAPI().validateFiles(version.files, true, s ->
> errs.add(s));
> if (!errs.isEmpty()) {
> ...
> }
> for (String file : version.files) {
>
> paths.add(coreContainer.getPackageStoreAPI().getPackageStore().getRealpath(file));
> }
> {code}
> Something like [this
> commit|https://github.com/apache/solr/commit/9fc5253eef573dc5e53edf52d98d4a2e55e00e1e]
> solves the problem by manually copying the file across, but I'm not sure if
> this is the best approach to take.
> I'm happy to put a full PR together, although I'd appreciate it if someone
> could confirm whether the {{manifest.json}} file should be transferred from
> the source node like this (and can be treated as mandatory), or if I've
> misunderstood how this should be working.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
