JingsongLi commented on code in PR #11248:
URL: https://github.com/apache/paimon/pull/11248#discussion_r4643410498
##########
paimon-api/src/main/java/org/apache/paimon/CoreOptions.java:
##########
@@ -4188,17 +4189,19 @@ public boolean indexFileInDataFileDir() {
return options.get(INDEX_FILE_IN_DATA_FILE_DIR);
}
- public Path globalIndexExternalPath() {
+ public List<Path> globalIndexExternalPaths() {
String pathString = options.get(GLOBAL_INDEX_EXTERNAL_PATH);
if (pathString == null || pathString.isEmpty()) {
- return null;
+ return Collections.emptyList();
}
- Path path = new Path(pathString);
- String scheme = path.toUri().getScheme();
- if (scheme == null) {
- throw new IllegalArgumentException("scheme should not be null: " +
path);
+ List<Path> paths = new ArrayList<>();
+ for (String value : pathString.split(",")) {
Review Comment:
**[P1] Preserve the existing interpretation of persisted single-root paths**
I wrote a real append table with the previously valid literal URI
`file:///tmp/index,archive`. The baseline Java reader returns the stored row.
This head throws `scheme should not be null: archive` during ordinary snapshot
planning, even without using a global index. The shared parser also reads every
historical schema during DROP, so removing the setting from the latest schema
still leaves the old value in the cleanup path. Please preserve the old
single-URI option and give the root list an unambiguous representation. Add an
upgrade test that reads an existing table and checks cleanup after altering its
options.
##########
docs/docs/multimodal-table/global-index/manage-indexes.mdx:
##########
@@ -475,7 +512,7 @@ These table options affect global index build and read
behavior:
| `scalar-index.search-mode` | `fast` | Search mode for BTree, Bitmap,
Multivalue, FM, and R-tree bbox queries. |
| `vector-index.search-mode` | `fast` | Search mode for vector queries. |
| `full-text-index.search-mode` | `fast` | Search mode for full-text queries. |
-| `global-index.external-path` | Not set | Root directory for global index
files. If not set, files are stored under the table index directory. |
+| `global-index.external-path` | Not set | Comma-separated root directories
for global index files, for example `s3://bucket-a/index,s3://bucket-b/index`.
New files use round-robin selection independently of data-file external path
settings. Each file's actual path is saved in metadata, so changing this option
does not relocate existing files. If not set, files are stored under the table
index directory. PyPaimon index builds support only a single external directory
and reject comma-separated values in table or build options. |
Review Comment:
**[P2] Handle multi-root layouts in PyPaimon deletion-vector writes**
The guard added to the Python index builder leaves ordinary table updates
exposed. I appended to a Data Evolution Parquet table with deletion vectors and
`file:///tmp/root-a,file:///tmp/root-b`, then used PyPaimon's public
`new_update().delete_by_row_id` Python path and committed snapshot 2
successfully. The committed DV's physical path was under
`root-a,file:/tmp/root-b/`, outside both configured directories. Java's
external-root cleanup cannot find it there. The multiple-root handling or early
rejection needs to cover the shared Python index path allocation used by DELETE
and hash bucket writes, before creating any file.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]