This is an automated email from the ASF dual-hosted git repository.
zhangstar333 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new fdb747ee65f [doc](lance) add doc about SHOW INDEX for lance catalog
(#4090)
fdb747ee65f is described below
commit fdb747ee65f0dcdf7c315e777d75cc3ab9d54b92
Author: kid <[email protected]>
AuthorDate: Wed Aug 26 18:47:53 2026 +0800
[doc](lance) add doc about SHOW INDEX for lance catalog (#4090)
Document the `SHOW INDEX` support for Lance Filesystem Catalog tables
added in apache/doris#66637.
## What Changed
- `lakehouse/catalogs/lance-catalog.mdx` (EN + zh-CN)
- Feature Overview: metadata access now lists `SHOW INDEX` (Filesystem
Catalogs only); added a `System tables (table$...)` row marked "Not
supported" that points users to `SHOW INDEX` for index metadata,
aligning with the Paimon/Iceberg system-table model.
- New "Inspect Lance Indexes / 查看 Lance 索引" section: syntax and aliases
(`SHOW INDEXES` / `SHOW KEY` / `SHOW KEYS`), the 13-column result
mapping for Lance tables (only `Table`, `Key_name`, `Seq_in_index`,
`Column_name`, `Index_type`, and `Properties` carry values),
nested-field paths with backtick quoting, the `Properties` details
allowlist, output examples taken from the code PR's regression test,
fail-closed semantics, the `SHOW` privilege requirement, and the fixed
`SHOW INDEX is not supported for Lance REST catalogs` error.
- Current Limitations and Recommendations: added a `SHOW INDEX` bullet.
- `sql-manual/sql-statements/table-and-view/index/SHOW-INDEX.md` (EN +
zh-CN)
- Description now covers Lance logical indexes, and notes that tables in
other external catalogs (Iceberg, Paimon, ...) return an empty result,
with links to their system-table docs (including Paimon
`table_indexes`).
- Added a Lance example.
`docs/` (dev) is intentionally untouched: the Lance catalog doc was
removed from dev in #4086 because apache/doris master has no Lance code
(the feature lives on branch-4.1).
---
.../lakehouse/catalogs/lance-catalog.mdx | 39 ++++++++++++++++++++--
.../table-and-view/index/SHOW-INDEX.md | 14 ++++++--
.../lakehouse/catalogs/lance-catalog.mdx | 39 ++++++++++++++++++++--
.../table-and-view/index/SHOW-INDEX.md | 14 ++++++--
4 files changed, 97 insertions(+), 9 deletions(-)
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/lakehouse/catalogs/lance-catalog.mdx
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/lakehouse/catalogs/lance-catalog.mdx
index fb8edaa8fbe..221f7fb4eb7 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/lakehouse/catalogs/lance-catalog.mdx
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/lakehouse/catalogs/lance-catalog.mdx
@@ -2,7 +2,7 @@
{
"title": "Lance Catalog",
"language": "zh-CN",
- "description": "Apache Doris Lance Catalog 使用指南：通过 Filesystem 或 REST
Namespace 访问 Lance 数据集，支持并行读取、谓词下推、S3/Local TVF 以及向量检索。"
+ "description": "Apache Doris Lance Catalog 使用指南：通过 Filesystem 或 REST
Namespace 访问 Lance 数据集，支持并行读取、谓词下推、S3/Local TVF、索引查看以及向量检索。"
}
---
@@ -22,7 +22,8 @@ Lance 是面向分析和 AI 场景的列式数据格式。Doris 可以通过 Lan
|---|---|
| Filesystem Catalog | 支持本地文件系统、`file://` 和 `s3://` Warehouse |
| REST Catalog | 支持 Lance REST Namespace，以及无认证、Bearer Token、API Key 和自定义 HTTP
Header |
-| 元数据访问 | 支持 `SHOW DATABASES`、`SHOW TABLES` 和 `DESC` |
+| 元数据访问 | 支持 `SHOW DATABASES`、`SHOW TABLES`、`DESC` 和 `SHOW INDEX`（仅 Filesystem
Catalog） |
+| 系统表（`table$...`） | 暂不支持；索引元数据请使用 `SHOW INDEX` |
| 数据查询 | 支持列裁剪、并行扫描 Lance Fragment 和当前版本的快照一致性读取 |
| 谓词下推 | 支持将部分标量谓词下推到 Lance 执行 |
| 文件 TVF | 支持通过 `s3()` 和 `local()` 直接查询 Lance 数据集 |
@@ -213,6 +214,39 @@ FROM lance_catalog.default.user_profiles;
普通 Catalog 查询会在规划阶段固定一个 Lance 数据集版本，并按 Fragment 生成扫描任务。因此，同一条查询读取一致的快照，同时可以由多个
Scanner 并行扫描不同 Fragment，不会让每个 Scanner 重复扫描整个数据集。
+## 查看 Lance 索引
+
+对于 Filesystem Catalog 中的表，`SHOW INDEX` 可以查看 Lance 数据集中记录的逻辑标量索引和向量索引。变种语法
`SHOW INDEXES`、`SHOW KEY` 和 `SHOW KEYS` 返回相同结果：
+
+```sql
+SHOW INDEX FROM lance_catalog.default.items;
+```
+
+Doris 在语句执行时读取数据集最新快照中的索引元数据，结果按索引名和列位置排序。Lance 内部维护的系统索引（如
`__lance_frag_reuse` 和 `__lance_mem_wal`）不会展示。没有索引的表返回空结果。
+
+`SHOW INDEX` 返回标准的 13 列结果集。对于 Lance 表，只有以下列有值，其余列恒为空：
+
+| 列 | Lance 表的取值 |
+|---|---|
+| `Table` | 表名。 |
+| `Key_name` | Lance 逻辑索引名。 |
+| `Seq_in_index` | 列在索引中的位置，从 `1` 开始。索引包含多个字段时，每个字段输出一行。 |
+| `Column_name` | 被索引的字段。嵌套字段上的索引显示以 `.` 连接的字段路径，包含字母、数字和 `_`
以外字符的路径段使用反引号引用，例如 `` attributes.`child.with.dot` ``。 |
+| `Index_type` | Lance SDK 报告的索引类型，例如
`BTree`、`IVF_FLAT`、`IVF_SQ`、`IVF_PQ`、`IVF_HNSW_FLAT`、`IVF_HNSW_SQ` 或
`IVF_HNSW_PQ`。 |
+| `Properties` | 包含固定字段集合的索引详情 JSON：顶层为 `metric_type` 和
`target_partition_size`，`compression` 下为 `type`、`num_bits`、`num_sub_vectors` 和
`rotation_type`，`hnsw` 下为 `construction_ef`、`max_connections` 和
`max_level`。键按字典序排列；索引详情不包含以上字段时值为 `{}`。 |
+
+下面的示例展示了一个 `IVF_PQ` 向量索引和一个建立在嵌套字段上的 `BTree` 索引：
+
+```text
+Table Key_name Seq_in_index Column_name
Index_type Properties
+vs_ivf_pq_f32 embedding_ivf_pq_f32 1 embedding
IVF_PQ
{"compression":{"num_bits":4,"num_sub_vectors":4,"type":"pq"},"metric_type":"L2"}
+nested_index nested_label_btree 1 attributes.`child.with.dot`
BTree {}
+```
+
+`SHOW INDEX` 只查看已有索引，不会创建索引。请使用 Lance SDK 或其他 Lance 写入端在 Doris 之外创建 Lance
索引。如果数据集中记录的索引元数据不一致（例如索引引用了未知字段，或两个索引完全重名），语句会直接报错，而不是返回部分元数据。
+
+`SHOW INDEX` 需要用户拥有该表的 `SHOW` 权限。对于 REST Catalog，该语句会被拒绝并返回错误 `SHOW INDEX is
not supported for Lance REST catalogs`。
+
## 类型映射
| Lance / Arrow 类型 | Doris 类型 | 说明 |
@@ -566,6 +600,7 @@ TVF `filter` 中引用的 Prefilter 列与外层 `WHERE` 列不同：前者由 L
- 使用不支持的列类型时，建议显式列出需要读取的列，避免 `SELECT *` 投影到不支持的列。
- 对普通扫描使用 `EXPLAIN` 检查 `lancePushdownPredicate`，确认目标条件是否已下推。
- 向量检索前应在 Lance 中创建与查询方式匹配的索引；小数据集或验证场景可以设置 `"use_index" = "false"` 使用 Flat
Search。
+- 可以使用 `SHOW INDEX` 查看 Filesystem Catalog 表的 Lance
逻辑索引，例如在执行索引向量检索前确认索引名、索引类型和被索引字段；REST Catalog 不支持 `SHOW INDEX`。
- 向量查询需要稳定顺序时，显式使用 `ORDER BY _distance ASC` 并增加唯一 Tie-breaker。
- 需要在向量候选生成前过滤时使用 `vector_search()` 的 `filter`；外层 `WHERE` 只过滤每个搜索 Split
已生成的候选，并在 Doris 全局 TopN 之前执行，应允许其最终结果少于 `top_k`。
- 使用 `EXPLAIN` 检查 `lanceSearchFragments` 和
`lanceSearchIndexSegments`。前者表示固定快照中的可见 Fragment 数量，后者表示 FE 选择的物理 Index Segment
Split 数量；此外还可能存在回退 Fragment Split。
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/sql-manual/sql-statements/table-and-view/index/SHOW-INDEX.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/sql-manual/sql-statements/table-and-view/index/SHOW-INDEX.md
index 568063842db..090ceebaa9c 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/sql-manual/sql-statements/table-and-view/index/SHOW-INDEX.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/sql-manual/sql-statements/table-and-view/index/SHOW-INDEX.md
@@ -2,13 +2,17 @@
{
"title": "SHOW INDEX",
"language": "zh-CN",
- "description": "该语句用于展示一个表中索引的相关信息，目前只支持 inverted index, ann index 索引"
+ "description": "该语句用于展示一个表中索引的相关信息，包括内表的 inverted index、ann index 索引以及
Lance Catalog 表的逻辑索引"
}
---
## 描述
- 该语句用于展示一个表中索引的相关信息，目前只支持 inverted index, ann index 索引
+该语句用于展示一个表中索引的相关信息。对于内表，目前只支持 inverted index、ann index 索引。
+
+对于 Lance Filesystem Catalog 中的表，该语句展示 Lance 数据集中记录的逻辑标量索引和向量索引，且只有
`Table`、`Key_name`、`Seq_in_index`、`Column_name`、`Index_type` 和 `Properties`
列有值。不支持 Lance REST Catalog。详见[查看 Lance
索引](../../../../lakehouse/catalogs/lance-catalog.mdx#查看-lance-索引)。
+
+对于 Iceberg、Paimon 等其他外部 Catalog 中的表，该语句返回空结果。这些格式通过系统表暴露表元数据，参见 [Iceberg
系统表](../../../../lakehouse/catalogs/iceberg-catalog.mdx#系统表)和 [Paimon
系统表](../../../../lakehouse/catalogs/paimon-catalog.mdx#系统表)；Paimon 的索引文件可通过
[`table_indexes`](../../../../lakehouse/catalogs/paimon-catalog.mdx#table_indexes)
系统表查看。
## 语法
@@ -60,9 +64,13 @@ SHOW KEY[ S ] FROM [ <db_name>. ] <table_name> [ FROM
<db_name> ];
## 示例
- 展示指定 table_name 的下索引
-
+
```SQL
SHOW INDEX FROM example_db.table_name;
```
+- 展示 Lance Catalog 表的逻辑索引
+ ```SQL
+ SHOW INDEX FROM lance_catalog.default.items;
+ ```
diff --git a/versioned_docs/version-4.x/lakehouse/catalogs/lance-catalog.mdx
b/versioned_docs/version-4.x/lakehouse/catalogs/lance-catalog.mdx
index 20ff078fd70..73e0163b6b0 100644
--- a/versioned_docs/version-4.x/lakehouse/catalogs/lance-catalog.mdx
+++ b/versioned_docs/version-4.x/lakehouse/catalogs/lance-catalog.mdx
@@ -2,7 +2,7 @@
{
"title": "Lance Catalog",
"language": "en",
- "description": "Apache Doris Lance Catalog guide: access Lance datasets
through Filesystem or REST Namespace catalogs, with parallel reads, predicate
pushdown, S3/Local TVFs, and vector search."
+ "description": "Apache Doris Lance Catalog guide: access Lance datasets
through Filesystem or REST Namespace catalogs, with parallel reads, predicate
pushdown, S3/Local TVFs, index inspection, and vector search."
}
---
@@ -22,7 +22,8 @@ Doris currently provides read-only access to Lance. Creating,
writing, updating,
|---|---|
| Filesystem Catalog | Supports warehouses on a local file system, `file://`,
or `s3://` |
| REST Catalog | Supports Lance REST Namespace with no authentication, Bearer
Token, API Key, or custom HTTP headers |
-| Metadata access | Supports `SHOW DATABASES`, `SHOW TABLES`, and `DESC` |
+| Metadata access | Supports `SHOW DATABASES`, `SHOW TABLES`, `DESC`, and
`SHOW INDEX` (Filesystem Catalogs only) |
+| System tables (`table$...`) | Not supported; use `SHOW INDEX` for index
metadata |
| Data queries | Supports column pruning, parallel Lance Fragment scans, and
snapshot-consistent reads of the current version |
| Predicate pushdown | Supports pushing compatible scalar predicates down to
Lance |
| File TVFs | Supports querying Lance datasets directly through `s3()` and
`local()` |
@@ -213,6 +214,39 @@ FROM lance_catalog.default.user_profiles;
For a regular Catalog query, Doris pins a Lance dataset version during
planning and generates scan tasks by Fragment. A query therefore reads a
consistent snapshot, while multiple Scanners can read different Fragments in
parallel without every Scanner repeatedly scanning the entire dataset.
+## Inspect Lance Indexes
+
+For a table in a Filesystem Catalog, `SHOW INDEX` displays the logical scalar
and vector indexes recorded in the Lance dataset. The variant statements `SHOW
INDEXES`, `SHOW KEY`, and `SHOW KEYS` produce the same result:
+
+```sql
+SHOW INDEX FROM lance_catalog.default.items;
+```
+
+Doris reads the index metadata from the latest dataset snapshot at execution
time and sorts the result by index name and column position. System indexes
that Lance maintains internally, such as `__lance_frag_reuse` and
`__lance_mem_wal`, are not displayed. A table without indexes returns an empty
result.
+
+`SHOW INDEX` returns the standard 13-column result set. For a Lance table,
only the following columns carry values; the other columns are always empty:
+
+| Column | Value for a Lance table |
+|---|---|
+| `Table` | Table name. |
+| `Key_name` | Name of the Lance logical index. |
+| `Seq_in_index` | Position of the column in the index, starting from `1`. An
index on multiple fields produces one row per field. |
+| `Column_name` | Indexed field. An index on a nested field displays the field
path joined with `.`, and a path segment containing characters other than
letters, digits, and `_` is quoted with backticks, for example ``
attributes.`child.with.dot` ``. |
+| `Index_type` | Index type reported by the Lance SDK, such as `BTree`,
`IVF_FLAT`, `IVF_SQ`, `IVF_PQ`, `IVF_HNSW_FLAT`, `IVF_HNSW_SQ`, or
`IVF_HNSW_PQ`. |
+| `Properties` | JSON object with a fixed set of index details: `metric_type`
and `target_partition_size` at the top level, `type`, `num_bits`,
`num_sub_vectors`, and `rotation_type` under `compression`, and
`construction_ef`, `max_connections`, and `max_level` under `hnsw`. Keys are
sorted, and the value is `{}` when the index details contain none of these
fields. |
+
+The following rows illustrate an `IVF_PQ` vector index and a `BTree` index on
a nested field:
+
+```text
+Table Key_name Seq_in_index Column_name
Index_type Properties
+vs_ivf_pq_f32 embedding_ivf_pq_f32 1 embedding
IVF_PQ
{"compression":{"num_bits":4,"num_sub_vectors":4,"type":"pq"},"metric_type":"L2"}
+nested_index nested_label_btree 1 attributes.`child.with.dot`
BTree {}
+```
+
+`SHOW INDEX` only inspects existing indexes; it does not create one. Create
Lance indexes with the Lance SDK or another Lance writer outside Doris. If the
recorded index metadata is inconsistent, for example when an index references
an unknown field or two indexes share an identical name, the statement fails
instead of returning partial metadata.
+
+`SHOW INDEX` requires the `SHOW` privilege on the table. For a REST Catalog,
the statement is rejected with the error `SHOW INDEX is not supported for Lance
REST catalogs`.
+
## Type Mapping
| Lance / Arrow Type | Doris Type | Description |
@@ -566,6 +600,7 @@ Setting `experimental_topn_lazy_materialization_threshold`
to `-1` disables the
- For tables containing unsupported column types, explicitly list the columns
to read instead of projecting unsupported columns through `SELECT *`.
- For regular scans, inspect `lancePushdownPredicate` in `EXPLAIN` to verify
which conditions have been pushed down.
- Create a vector index in Lance that matches the intended query before
running indexed vector search. For small datasets or validation, set
`"use_index" = "false"` to perform Flat Search.
+- Use `SHOW INDEX` on a Filesystem Catalog table to inspect its logical Lance
indexes, for example to verify the index name, type, and indexed fields before
indexed vector search. `SHOW INDEX` is not supported for REST Catalogs.
- For deterministic vector result ordering, explicitly use `ORDER BY _distance
ASC` and add a unique tie-breaker.
- Use the `vector_search()` `filter` parameter when filtering must occur
before vector candidates are generated. An outer `WHERE` filters only the
candidates already generated by each search Split and runs before Doris global
TopN, so allow for a final result with fewer than `top_k` rows.
- Use `EXPLAIN` to inspect `lanceSearchFragments` and
`lanceSearchIndexSegments`. The former is the number of visible Fragments in
the pinned snapshot; the latter is the number of physical Index Segment splits
selected by the FE. Additional fallback Fragment splits may also be present.
diff --git
a/versioned_docs/version-4.x/sql-manual/sql-statements/table-and-view/index/SHOW-INDEX.md
b/versioned_docs/version-4.x/sql-manual/sql-statements/table-and-view/index/SHOW-INDEX.md
index 30ab8b802c4..138e3dfb95f 100644
---
a/versioned_docs/version-4.x/sql-manual/sql-statements/table-and-view/index/SHOW-INDEX.md
+++
b/versioned_docs/version-4.x/sql-manual/sql-statements/table-and-view/index/SHOW-INDEX.md
@@ -2,13 +2,17 @@
{
"title": "SHOW INDEX",
"language": "en",
- "description": "This statement is used to display information about
indexes in a table. Currently, only inverted indexes and ann indexes are
supported."
+ "description": "This statement is used to display information about
indexes in a table, including inverted indexes and ann indexes on internal
tables and logical indexes on Lance Catalog tables."
}
---
## Description
-This statement is used to display information about indexes in a table.
Currently, only inverted indexes and ann indexes are supported.
+This statement is used to display information about indexes in a table. For
internal tables, only inverted indexes and ann indexes are supported.
+
+For a table in a Lance Filesystem Catalog, this statement displays the logical
scalar and vector indexes recorded in the Lance dataset, and only the `Table`,
`Key_name`, `Seq_in_index`, `Column_name`, `Index_type`, and `Properties`
columns carry values. Lance REST Catalogs are not supported. See [Inspect Lance
Indexes](../../../../lakehouse/catalogs/lance-catalog.mdx#inspect-lance-indexes)
for details.
+
+For tables in other external catalogs, such as Iceberg and Paimon, this
statement returns an empty result. Those formats expose table metadata through
system tables instead; see [Iceberg system
tables](../../../../lakehouse/catalogs/iceberg-catalog.mdx#system-tables) and
[Paimon system
tables](../../../../lakehouse/catalogs/paimon-catalog.mdx#system-tables).
Paimon index files can be inspected through the
[`table_indexes`](../../../../lakehouse/catalogs/paimon-catalog.mdx#table_indexes)
[...]
## Syntax
@@ -64,3 +68,9 @@ The user executing this SQL command must have at least the
following privileges:
SHOW INDEX FROM example_db.table_name;
```
+- Display logical indexes for a Lance Catalog table
+
+ ```SQL
+ SHOW INDEX FROM lance_catalog.default.items;
+ ```
+
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
