JingsongLi commented on code in PR #11248:
URL: https://github.com/apache/paimon/pull/11248#discussion_r4624145757


##########
paimon-core/src/main/java/org/apache/paimon/utils/FileStorePathFactory.java:
##########
@@ -329,15 +326,22 @@ public IndexPathFactory indexFileFactory(BinaryRow 
partition, int bucket) {
     }
 
     public IndexPathFactory globalIndexFileFactory() {
+        ExternalPathProvider provider =
+                globalIndexExternalPaths == null || 
globalIndexExternalPaths.isEmpty()
+                        ? null
+                        : new 
RoundRobinExternalPathProvider(globalIndexExternalPaths, null);
         return new IndexPathFactory() {
             @Override
-            public Path toPath(String fileName) {
-                return new Path(globalIndexRootDir(), fileName);
+            public synchronized Path newPathFromName(String fileName) {
+                return provider == null
+                        ? new Path(indexPath(), fileName)
+                        : provider.getNextExternalDataPath(fileName);
             }
 
             @Override
             public Path newPath() {
-                return toPath(INDEX_PREFIX + uuid + "-" + 
indexFileCount.getAndIncrement());
+                return newPathFromName(

Review Comment:
   **[P1] Persist the allocated destination path when copying indexes**
   
   When Spark `sys.copy` copies a dynamic-bucket table with two index roots, 
this allocation can select root B for a hash index stored under root A. 
`ListIndexFilesOperator` copies the file to B but passes only the new basename 
to `CopyFilesUtil.toNewIndexFileMeta`, which builds `externalPath` from the 
source file's parent. The copy can commit a reference to `A/<new-name>`, and 
the next write to the target fails when it loads the hash index. I reproduced 
the missing-file error using the current allocator, metadata helper, and 
`HashIndexFile` with two local roots.
   
   Please build the copied `IndexFileMeta` from the actual destination `Path` 
and the target factory's `isExternalPath()` flag. A two-root copy test should 
cover a destination root that differs from the source root.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to