This is an automated email from the ASF dual-hosted git repository.
zhangstar333 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new 8f67786961e [doc](lance) Document OSS support for the Lance catalog
(#4094)
8f67786961e is described below
commit 8f67786961ece462582246bfab216574aef1c301
Author: FANNG <[email protected]>
AuthorDate: Wed Sep 2 17:55:58 2026 +0800
[doc](lance) Document OSS support for the Lance catalog (#4094)
## Summary
Documents the Alibaba Cloud OSS support added by apache/doris#67157
(issue apache/doris#67131). The Lance catalog page currently tells users
that a warehouse can only be a local path, `file://` or `s3://`, so OSS
looks unsupported.
Requested by @zhangstar333 in apache/doris#67157.
## Changes
Both `versioned_docs/version-4.x/.../lance-catalog.mdx` and the zh-CN
copy:
- `warehouse` now lists `oss://` among the accepted forms.
- New **Use Alibaba Cloud OSS** section next to the existing S3 one,
with a `CREATE CATALOG` example, the qualified
`oss://bucket.oss-<region>.aliyuncs.com/path` form Doris reduces to the
bucket, the bucket requirement, and a note on `oss.session_token`.
- REST catalog section now names the OSS properties usable as defaults,
states that namespace-vended credentials take precedence over the
catalog's own, and shows a catalog created with no access key at all.
- Documents that a namespace may vend the options as either
`oss_endpoint` / `oss_access_key_id` / `oss_secret_access_key` /
`oss_region` / `oss_security_token` or the bare OSS-native `endpoint` /
`access_key_id` / `access_key_secret` / `region` / `security_token`, and
that vending one option under both spellings with different values is an
error.
## Verification
The examples were run against a real OSS bucket on a Doris cluster built
from apache/doris#67157, not written from the code alone:
- the filesystem example **exactly as written** — without
`fs.oss.support`, which the doc does not mention — creates the catalog
and scans correctly;
- the qualified `oss://bucket.<endpoint>/path` warehouse also scans, and
`oss:/path` is rejected at `CREATE CATALOG`;
- the no-credential REST example was exercised against a real Apache
Gravitino 1.3.0 `lance-rest` service, which vends the bare OSS-native
spellings;
- the conflicting-spelling error was reproduced.
## Scope
Every statement on the page was exercised against a live bucket,
including `oss.session_token`: an STS triple obtained through
`AssumeRole` scans the table, and the same temporary key pair
**without** the token fails to initialize the catalog with OSS's own
`InvalidAccessKeyId` / "The Security Token may be lost to specify that
it is a STS Access Id". That negative control is what shows the token is
carried through to the BE rather than the scan succeeding by some other
route.
A path-style addressing note was written and then removed: Alibaba Cloud
OSS uses virtual-hosted addressing and its public endpoints no longer
accept path-style requests, so there was no configuration to confirm it
against.
---
.../lakehouse/catalogs/lance-catalog.mdx | 34 ++++++++++++++++++++--
.../lakehouse/catalogs/lance-catalog.mdx | 34 ++++++++++++++++++++--
2 files changed, 64 insertions(+), 4 deletions(-)
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/lakehouse/catalogs/lance-catalog.mdx
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/lakehouse/catalogs/lance-catalog.mdx
index 7e0cd6849dd..2eb959c6a16 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/lakehouse/catalogs/lance-catalog.mdx
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/lakehouse/catalogs/lance-catalog.mdx
@@ -83,7 +83,7 @@ Filesystem Catalog 直接从 Warehouse 目录发现 Lance Namespace 和表。
| 属性 | 是否必需 | 说明 |
|---|---|---|
-| `warehouse` | 是 | Lance Warehouse 根路径。支持本地绝对路径、`file://` URI 和 `s3://` URI。 |
+| `warehouse` | 是 | Lance Warehouse 根路径。支持本地绝对路径、`file://` URI、`s3://` URI 和
`oss://` URI。 |
#### 使用 S3 兼容对象存储
@@ -104,6 +104,24 @@ CREATE CATALOG lance_catalog PROPERTIES (
访问 AWS S3 时，可以省略 `s3.endpoint`，并按实际环境配置访问密钥、Region 和 Path Style。
+#### 使用阿里云 OSS
+
+```sql
+CREATE CATALOG lance_oss PROPERTIES (
+ "type" = "lance",
+ "lance.catalog.type" = "filesystem",
+ "warehouse" = "oss://my-bucket/lance",
+ "oss.endpoint" = "oss-cn-beijing.aliyuncs.com",
+ "oss.region" = "cn-beijing",
+ "oss.access_key" = "<ak>",
+ "oss.secret_key" = "<sk>"
+);
+```
+
+`warehouse` 也支持带 Endpoint 的形式
`oss://my-bucket.oss-cn-beijing.aliyuncs.com/lance`，Doris 会在打开 Namespace 前将其还原为
Bucket。`warehouse` 必须指定 Bucket，因此 `oss:/lance` 这类根路径会在创建 Catalog 时被拒绝。
+
+使用 STS 临时凭证访问 OSS 时，可以增加 `"oss.session_token" = "<token>"`。
+
#### 使用本地文件系统
```sql
@@ -147,7 +165,19 @@ CREATE CATALOG lance_rest PROPERTIES (
"lance.rest.api-key" = "your-api-key"
```
-如果 REST 服务返回临时存储凭证，Doris 会使用这些凭证访问对应的 Lance 表。也可以在 Catalog 中配置
`s3.endpoint`、`s3.access_key`、`s3.secret_key`、`s3.region` 和
`use_path_style`，作为默认的对象存储访问参数。
+如果 REST 服务返回临时存储凭证，Doris 会使用这些凭证访问对应的 Lance 表。也可以在 Catalog 中配置默认的对象存储访问参数：S3
兼容存储使用 `s3.endpoint`、`s3.access_key`、`s3.secret_key`、`s3.region` 和
`use_path_style`，OSS 使用
`oss.endpoint`、`oss.access_key`、`oss.secret_key`、`oss.region` 和
`oss.session_token`。
+
+对于同一张表，Namespace 下发的凭证优先于 Catalog 中配置的凭证。因此，当 Namespace 能够下发完整凭证时，创建 Catalog
时无需保存任何访问密钥：
+
+```sql
+CREATE CATALOG lance_rest_oss PROPERTIES (
+ "type" = "lance",
+ "lance.catalog.type" = "rest",
+ "lance.rest.uri" = "https://lance.example.com";
+);
+```
+
+对于 OSS 表，Namespace 下发的存储选项可以写成
`oss_endpoint`、`oss_access_key_id`、`oss_secret_access_key`、`oss_region`、`oss_security_token`，也可以写成
OSS 原生的
`endpoint`、`access_key_id`、`access_key_secret`、`region`、`security_token`。Doris
同时接受这两种写法，并将其视为同一个选项。如果同一个选项以两种写法下发且取值不同，则会报错。
:::caution
当前 BE Reader 不支持由 REST Namespace 管理版本的 Lance 表（Managed Versioning）。
diff --git a/versioned_docs/version-4.x/lakehouse/catalogs/lance-catalog.mdx
b/versioned_docs/version-4.x/lakehouse/catalogs/lance-catalog.mdx
index ee259c9a9c7..69d16d8d90f 100644
--- a/versioned_docs/version-4.x/lakehouse/catalogs/lance-catalog.mdx
+++ b/versioned_docs/version-4.x/lakehouse/catalogs/lance-catalog.mdx
@@ -83,7 +83,7 @@ A Filesystem Catalog discovers Lance Namespaces and tables
directly from a wareh
| Property | Required | Description |
|---|---|---|
-| `warehouse` | Yes | Root path of the Lance warehouse. Local absolute paths,
`file://` URIs, and `s3://` URIs are supported. |
+| `warehouse` | Yes | Root path of the Lance warehouse. Local absolute paths,
`file://` URIs, `s3://` URIs, and `oss://` URIs are supported. |
#### Use S3-Compatible Object Storage
@@ -104,6 +104,24 @@ CREATE CATALOG lance_catalog PROPERTIES (
When accessing AWS S3, you can omit `s3.endpoint` and configure credentials,
Region, and Path Style for your environment.
+#### Use Alibaba Cloud OSS
+
+```sql
+CREATE CATALOG lance_oss PROPERTIES (
+ "type" = "lance",
+ "lance.catalog.type" = "filesystem",
+ "warehouse" = "oss://my-bucket/lance",
+ "oss.endpoint" = "oss-cn-beijing.aliyuncs.com",
+ "oss.region" = "cn-beijing",
+ "oss.access_key" = "<ak>",
+ "oss.secret_key" = "<sk>"
+);
+```
+
+`warehouse` also accepts the qualified form
`oss://my-bucket.oss-cn-beijing.aliyuncs.com/lance`, which Doris reduces to the
bucket before opening the Namespace. It must always name a bucket, so a root
such as `oss:/lance` is rejected when the Catalog is created.
+
+To access OSS with a temporary STS credential, add `"oss.session_token" =
"<token>"`.
+
#### Use a Local File System
```sql
@@ -147,7 +165,19 @@ For API Key authentication, replace the authentication
properties with:
"lance.rest.api-key" = "your-api-key"
```
-If the REST service returns temporary storage credentials, Doris uses those
credentials to access the corresponding Lance table. You can also configure
`s3.endpoint`, `s3.access_key`, `s3.secret_key`, `s3.region`, and
`use_path_style` in the Catalog as the default object storage access parameters.
+If the REST service returns temporary storage credentials, Doris uses those
credentials to access the corresponding Lance table. You can also configure
default object storage access parameters in the Catalog: `s3.endpoint`,
`s3.access_key`, `s3.secret_key`, `s3.region`, and `use_path_style` for
S3-compatible storage, or `oss.endpoint`, `oss.access_key`, `oss.secret_key`,
`oss.region`, and `oss.session_token` for OSS.
+
+Credentials vended by the Namespace take precedence over the ones configured
in the Catalog, per table. A Namespace that vends complete credentials
therefore lets you create the Catalog without storing any access key in it:
+
+```sql
+CREATE CATALOG lance_rest_oss PROPERTIES (
+ "type" = "lance",
+ "lance.catalog.type" = "rest",
+ "lance.rest.uri" = "https://lance.example.com";
+);
+```
+
+For an OSS table, the Namespace may spell the vended options either as
`oss_endpoint`, `oss_access_key_id`, `oss_secret_access_key`, `oss_region`, and
`oss_security_token`, or with the bare OSS-native names `endpoint`,
`access_key_id`, `access_key_secret`, `region`, and `security_token`. Doris
accepts both and treats them as the same option. Vending the same option under
both spellings with different values is an error.
:::caution
The current BE Reader does not support Lance tables whose versions are managed
by REST Namespace (Managed Versioning).
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
