nick-boss-tech commented on code in PR #5352:
URL: https://github.com/apache/solr/pull/5352#discussion_r4636871553


##########
solr/core/src/test/org/apache/solr/cli/PackageToolTest.java:
##########
@@ -300,6 +307,41 @@ public void testPackageTool() throws Exception {
           "--credentials",
           SecurityJson.USER_PASS
         });
+
+    var packageRequest = new PackageApi.GetPackage("question-answer");
+    packageRequest.setBasicAuthCredentials(SecurityJson.USER, 
SecurityJson.PASS);
+    var versions = packageRequest.process(cluster.getSolrClient()).versions;
+    assertEquals(2, versions.size());
+    for (var version : versions) {
+      List<String> files = new ArrayList<>(version.files);
+      files.add(version.manifest);
+      for (String file : files) {
+        assertTrue(cluster.getZkClient().exists("/packagestore" + file));
+      }
+
+      run(
+          new String[] {
+            "--solr-url",
+            solrUrl,
+            "uninstall",

Review Comment:
   🤖 *AI text below* 🤖
   
   Could we cover an uninstall that gets past package removal but fails during 
file cleanup? [The CLI removes the package version 
first](https://github.com/epugh/solr/blob/35e418738b28d91d5c70722e09ed9b0a476d48b1/solr/core/src/java/org/apache/solr/cli/packagemanager/PackageManager.java#L147-L166),
 then stops at the first failed file DELETE. A retry now can’t look up the 
version to recover its file list. This is especially likely if the new CLI 
reaches an older server, since the CLI no longer removes the ZooKeeper entry 
and that server still rejects registered-file deletes. Could we check 
compatibility before unregistering and make remaining cleanup resumable after a 
partial failure?



##########
solr/solr-ref-guide/modules/configuration-guide/pages/package-manager-internals.adoc:
##########
@@ -90,6 +90,7 @@ The end points are:
 
 * `PUT /api/cluster/filestore/files/{full/path/to/file}` to upload a new file 
and add it to the file store
 * `GET /api/cluster/filestore/files/{full/path/to/file}` to download a file 
already in the filestore
+* `DELETE /api/cluster/filestore/files/{full/path/to/file}` to remove a file 
from ZooKeeper and delete its copies on live nodes. Files used by an installed 
package cannot be deleted.

Review Comment:
   🤖 *AI text below* 🤖
   
   Could the in-use check cover manifests as well as artifacts before this 
endpoint starts removing registered files? 
[`isJarInuse()`](https://github.com/epugh/solr/blob/35e418738b28d91d5c70722e09ed9b0a476d48b1/solr/core/src/java/org/apache/solr/pkg/PackageStore.java#L375-L391)
 only checks `version.files`, but the installer [registers the manifest 
separately](https://github.com/epugh/solr/blob/35e418738b28d91d5c70722e09ed9b0a476d48b1/solr/core/src/java/org/apache/solr/cli/packagemanager/RepositoryManager.java#L209-L223).
 Package management still reads that manifest for installed versions. Please 
keep both kinds of references protected and add a regression assertion for the 
manifest.



##########
solr/core/src/test/org/apache/solr/filestore/TestDistribFileStore.java:
##########
@@ -193,6 +194,25 @@ public void testFileStoreManagement() throws Exception {
 
       expected = 
Collections.singletonMap(":files:/package/mypkg/v1.0/runtimelibs.jar", null);
       checkAllNodesForFile(cluster, "/package/mypkg/v1.0/runtimelibs.jar", 
expected, false);
+
+      final String filePath = "/package/mypkg/v1.0/runtimelibs_v2.jar";
+      final String fileUrl = j.getBaseURLV2() + "/cluster/filestore/files" + 
filePath;
+      assertTrue(cluster.getZkClient().exists(DistribFileStore.ZK_PACKAGESTORE 
+ filePath));
+
+      resp =
+          j.getSolrClient()
+              .getHttpClient()
+              .newRequest(fileUrl + "?localDelete=true")
+              .method("DELETE")
+              .send();
+      assertEquals(400, resp.getStatus());
+      assertTrue(cluster.getZkClient().exists(DistribFileStore.ZK_PACKAGESTORE 
+ filePath));
+
+      resp = 
j.getSolrClient().getHttpClient().newRequest(fileUrl).method("DELETE").send();

Review Comment:
   🤖 *AI text below* 🤖
   
   Could we cover pending file transfers when asserting that cluster DELETE is 
complete? [A fetch verifies the downloaded data and then persists 
it](https://github.com/epugh/solr/blob/35e418738b28d91d5c70722e09ed9b0a476d48b1/solr/core/src/java/org/apache/solr/filestore/DistribFileStore.java#L193-L214)
 without checking that the same registration is still current. It can restore a 
deleted copy even if every delete succeeded. Conversely, a delayed delete 
shouldn’t touch a replacement at the same path. Binding those operations to a 
registration generation and coordinating that check with the local write would 
protect both cases.



##########
solr/core/src/test/org/apache/solr/filestore/TestDistribFileStore.java:
##########
@@ -193,6 +194,25 @@ public void testFileStoreManagement() throws Exception {
 
       expected = 
Collections.singletonMap(":files:/package/mypkg/v1.0/runtimelibs.jar", null);
       checkAllNodesForFile(cluster, "/package/mypkg/v1.0/runtimelibs.jar", 
expected, false);
+
+      final String filePath = "/package/mypkg/v1.0/runtimelibs_v2.jar";
+      final String fileUrl = j.getBaseURLV2() + "/cluster/filestore/files" + 
filePath;
+      assertTrue(cluster.getZkClient().exists(DistribFileStore.ZK_PACKAGESTORE 
+ filePath));
+
+      resp =
+          j.getSolrClient()
+              .getHttpClient()
+              .newRequest(fileUrl + "?localDelete=true")
+              .method("DELETE")
+              .send();
+      assertEquals(400, resp.getStatus());
+      assertTrue(cluster.getZkClient().exists(DistribFileStore.ZK_PACKAGESTORE 
+ filePath));
+
+      resp = 
j.getSolrClient().getHttpClient().newRequest(fileUrl).method("DELETE").send();
+      assertEquals(200, resp.getStatus());
+      
assertFalse(cluster.getZkClient().exists(DistribFileStore.ZK_PACKAGESTORE + 
filePath));

Review Comment:
   🤖 *AI text below* 🤖
   
   Could we make the file-in-use decision atomic with package registration? 
[Delete checks references before its write 
lock](https://github.com/epugh/solr/blob/35e418738b28d91d5c70722e09ed9b0a476d48b1/solr/core/src/java/org/apache/solr/filestore/ClusterFileStore.java#L290-L328),
 and [package registration validates the files before recording the 
version](https://github.com/epugh/solr/blob/35e418738b28d91d5c70722e09ed9b0a476d48b1/solr/core/src/java/org/apache/solr/pkg/PackageStore.java#L256-L291)
 without taking that lock. Now that DELETE can remove registered files, 
overlapping operations can leave an installed version pointing at a missing 
artifact. Rechecking under a lock shared by both operations, or using a 
transaction that checks both states, would preserve that invariant.



##########
solr/core/src/test/org/apache/solr/filestore/TestDistribFileStore.java:
##########
@@ -193,6 +194,25 @@ public void testFileStoreManagement() throws Exception {
 
       expected = 
Collections.singletonMap(":files:/package/mypkg/v1.0/runtimelibs.jar", null);
       checkAllNodesForFile(cluster, "/package/mypkg/v1.0/runtimelibs.jar", 
expected, false);
+
+      final String filePath = "/package/mypkg/v1.0/runtimelibs_v2.jar";
+      final String fileUrl = j.getBaseURLV2() + "/cluster/filestore/files" + 
filePath;
+      assertTrue(cluster.getZkClient().exists(DistribFileStore.ZK_PACKAGESTORE 
+ filePath));
+
+      resp =
+          j.getSolrClient()
+              .getHttpClient()
+              .newRequest(fileUrl + "?localDelete=true")
+              .method("DELETE")
+              .send();
+      assertEquals(400, resp.getStatus());
+      assertTrue(cluster.getZkClient().exists(DistribFileStore.ZK_PACKAGESTORE 
+ filePath));
+
+      resp = 
j.getSolrClient().getHttpClient().newRequest(fileUrl).method("DELETE").send();
+      assertEquals(200, resp.getStatus());

Review Comment:
   🤖 *AI text below* 🤖
   
   Could we cover a remote delete failure here? The success case waits for 
copies to disappear, but [the server ignores the remote 
futures](https://github.com/epugh/solr/blob/35e418738b28d91d5c70722e09ed9b0a476d48b1/solr/core/src/java/org/apache/solr/filestore/DistribFileStore.java#L495-L508).
 If a live node fails its delete, the API can still return 200 and remove the 
ZooKeeper entry while that node retains a copy. If cluster deletion is best 
effort, say so in the guide. If 200 means all live copies are gone, wait for 
the results and surface failures so callers can retry.



##########
solr/core/src/java/org/apache/solr/filestore/DistribFileStore.java:
##########
@@ -486,6 +485,7 @@ public List<FileDetails> list(String path, 
Predicate<String> predicate) {
 
   @Override
   public void delete(String path) {
+    deleteZKFileEntry(path);

Review Comment:
   🤖 *AI text below* 🤖
   
   Could cluster DELETE treat a registered file as present even if no live node 
has its bytes? 
[`doClusterDelete()`](https://github.com/epugh/solr/blob/35e418738b28d91d5c70722e09ed9b0a476d48b1/solr/core/src/java/org/apache/solr/filestore/ClusterFileStore.java#L283-L295)
 currently calls `getType(path, true)` and returns 400 on `NOFILE` before it 
reaches this line. That leaves stale ZooKeeper registrations in place. The old 
package CLI could clear the registration on its own; after this change it 
depends on the endpoint. Deleting a registered file shouldn’t require 
downloading it first.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to