u70b3 commented on issue #66497: URL: https://github.com/apache/doris/issues/66497#issuecomment-5301801004
Small v5.1 ยง3.1 clarification (no scope change): with pinned `lance-core 9.1.0-beta.3`, unfiltered `Dataset.describeIndices()` is not safe for Directory `SHOW INDEX`. The pinned JNI converts every returned `IndexDescription` by calling `details()`, while a valid system entry can have a details type with no scalar plugin; that entry can abort the bulk call before Doris can filter it ([pinned JNI source](https://github.com/lance-format/lance/blob/e934cc2ceda2bd5f5aa37a953cc29f71d24bd5c0/java/lance-jni/src/index.rs#L15-L51); [observed PR case](https://github.com/apache/doris/pull/66637#discussion_r3781698221)). For this Directory SHOW slice, the one-snapshot rule therefore also permits one opened `Dataset` to call `listIndexes()`, filter the formal system names `__lance_frag_reuse` and `__lance_mem_wal`, deduplicate repeated physical entries by name, and call `describeIndices(IndexCriteria.hasName(name))` for each remaining exact name on that same `Dataset` ([physical-list source](https://github.com/lance-format/lance/blob/e934cc2ceda2bd5f5aa37a953cc29f71d24bd5c0/java/lance-jni/src/blocking_dataset.rs#L1875-L1901); [logical-grouping source](https://github.com/lance-format/lance/blob/e934cc2ceda2bd5f5aa37a953cc29f71d24bd5c0/rust/lance/src/index.rs#L1270-L1311)). This Dataset index read MUST retain one deadline and separate finite raw/logical bounds, and fail closed unless every lookup returns exactly one non-null description with the requested name and the final logical metadata is well formed and unique. The formal pinned MemWAL name is `__lance_mem_wal` ([constant](https://github.com/la nce-format/lance/blob/e934cc2ceda2bd5f5aa37a953cc29f71d24bd5c0/rust/lance-table/src/system_index/mem_wal.rs#L13)). This clarification does not change REST, physical-inspection, or mutation scope. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
