This is an automated email from the ASF dual-hosted git repository. chaokunyang pushed a commit to branch update_docs in repository https://gitbox.apache.org/repos/asf/fory-site.git
commit 1c666ad31c23949cd8c2286b19cb601f7bc032e0 Author: chaokunyang <[email protected]> AuthorDate: Tue Aug 4 10:49:05 2026 +0800 docs: consolidate row format overview --- docs/row-format/compact.md | 50 ------------ docs/row-format/index.md | 90 +++++++++++++++++----- docs/row-format/rust.md | 2 +- docs/row-format/standard.md | 57 -------------- .../current/row-format/compact.md | 50 ------------ .../current/row-format/index.md | 83 ++++++++++++++++---- .../current/row-format/rust.md | 2 +- .../current/row-format/standard.md | 51 ------------ 8 files changed, 142 insertions(+), 243 deletions(-) diff --git a/docs/row-format/compact.md b/docs/row-format/compact.md deleted file mode 100644 index dbe2104691..0000000000 --- a/docs/row-format/compact.md +++ /dev/null @@ -1,50 +0,0 @@ ---- -title: Compact Row -sidebar_position: 2 -id: compact -license: | - Licensed to the Apache Software Foundation (ASF) under one or more - contributor license agreements. See the NOTICE file distributed with - this work for additional information regarding copyright ownership. - The ASF licenses this file to You under the Apache License, Version 2.0 - (the "License"); you may not use this file except in compliance with - the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. ---- - -Compact Row is a Java-only row encoding that reduces fixed-slot and null-bitmap overhead. It is not -wire-compatible with Standard Row. - -## Create a compact encoder - -```java -RowEncoder<MyBean> encoder = - Encoders.buildBeanCodec(MyBean.class) - .compactEncoding() - .build() - .get(); - -BinaryRow row = encoder.toRow(value); -MyBean decoded = encoder.fromRow(row); -``` - -Reuse the encoder within one thread. Create separate encoders for concurrent threads. - -## Layout tradeoffs - -- Fixed-size fields use their natural widths instead of eight-byte Standard Row slots. -- Fields are sorted by alignment to reduce padding. -- The null bitmap is omitted when no field is nullable. -- Fixed-size nested structs can be stored inline. - -Choose Compact Row only when every reader is Java and the space reduction justifies the -Java-specific layout. Use [Standard Row](standard.md) for Java/Python/C++/Rust interchange. - -See the [Row Format specification](../specification/row_format_spec.md) for the exact binary layout. diff --git a/docs/row-format/index.md b/docs/row-format/index.md index 1ebcf6d913..9960f200b4 100644 --- a/docs/row-format/index.md +++ b/docs/row-format/index.md @@ -19,27 +19,81 @@ license: | limitations under the License. --- -Row Format stores typed values in a cache-friendly binary layout for random and partial access -without reconstructing a complete object graph. Use it for analytical and -in-memory data processing. +Row Format is a cache-friendly binary format for efficient random access and partial +deserialization. Unlike object graph serialization, it lets readers access individual fields +without reconstructing the complete object. -## Choose a row family +Row Format is intended only for trusted analytical data, including memory-mapped data, selective +field access, and data pipelines. Use [Object Serialization](../object-serialization/index.md) when +the application needs general object graphs, shared or circular references, or complete object +reconstruction as its primary access pattern. -| Family | Runtime support | Compatibility | -| --------------------------- | ----------------------- | -------------------------------- | -| [Standard Row](standard.md) | Java, Python, C++, Rust | Shared Standard Row layout | -| [Compact Row](compact.md) | Java | Java-only, space-oriented layout | +## Choose a Layout -Use Binary Object Serialization when the goal is complete object reconstruction, references, or -general application messaging. Use Row Format when a workload reads selected fields, nested arrays, -or maps directly from encoded data. +| Layout | Runtime support | Compatibility | +| ------------ | ----------------------- | -------------------------------- | +| Standard Row | Java, Python, C++, Rust | Shared Standard Row layout | +| Compact Row | Java | Java-only, space-oriented layout | -## Runtime guides +## Standard Row -- [Java](java.md) -- [Python](python.md) -- [C++](cpp.md) -- [Rust](rust.md) +Standard Row is the interoperable layout for Java, Python, C++, and Rust. -The normative [Row Format specification](../specification/row_format_spec.md) defines Standard and -Compact layouts. +### Features + +- **Zero-copy random access**: Read selected fields directly from encoded data. +- **Partial deserialization**: Reconstruct only the values an application needs. +- **Cross-language compatibility**: Share Standard Row bytes between Java, Python, C++, and Rust. +- **Apache Arrow integration**: Convert rows to Arrow data in Java and Python. + +### Layout + +Standard Row stores fixed-width values inline and variable-width values by offset and size. Rows, +arrays, and maps use a schema to resolve field positions and element types. The normative byte +layout, alignment rules, type table, and endianness are defined by the +[Row Format specification](../specification/row_format_spec.md). + +### Runtime Support + +| Runtime | Standard Row compatibility | Runtime guide | Additional integration | +| ------- | -------------------------- | ------------------- | ------------------------------------------------------ | +| Java | Compatible | [Java](java.md) | Arrow conversion; interface and extension-type mapping | +| Python | Compatible | [Python](python.md) | PyArrow schema and table conversion | +| C++ | Compatible | [C++](cpp.md) | Native row readers and writers | +| Rust | Compatible | [Rust](rust.md) | Borrowed struct, array, and map views | + +Use the runtime guides for installation, schema construction, encoding, random access, partial +reads, and language-specific integrations. + +## Compact Row + +Compact Row is a Java-only row encoding that reduces fixed-slot and null-bitmap overhead. It is not +wire-compatible with Standard Row. + +### Create a Compact Encoder + +```java +RowEncoder<MyBean> encoder = + Encoders.buildBeanCodec(MyBean.class) + .compactEncoding() + .build() + .get(); + +BinaryRow row = encoder.toRow(value); +MyBean decoded = encoder.fromRow(row); +``` + +Reuse the encoder within one thread. Create separate encoders for concurrent threads. + +### Layout Tradeoffs + +- Fixed-size fields use their natural widths instead of eight-byte Standard Row slots. +- Fields are sorted by alignment to reduce padding. +- The null bitmap is omitted when no field is nullable. +- Fixed-size nested structs can be stored inline. + +Choose Compact Row only when every reader is Java and the space reduction justifies the +Java-specific layout. Use Standard Row for Java, Python, C++, and Rust interchange. + +See the [Row Format specification](../specification/row_format_spec.md) for the exact Standard and +Compact binary layouts. diff --git a/docs/row-format/rust.md b/docs/row-format/rust.md index 33efc184ac..e0efd399ce 100644 --- a/docs/row-format/rust.md +++ b/docs/row-format/rust.md @@ -186,5 +186,5 @@ assert!(scores.get(scores.len()).is_err()); ## Related Topics - [Basic Serialization](../object-serialization/rust/core-api.md) - Object graph serialization -- [Standard Row Format](standard.md) - Shared layout for Java, Python, C++, and Rust +- [Standard Row Format](index.md#standard-row) - Shared layout for Java, Python, C++, and Rust - [Row Format Specification](../specification/row_format_spec.md) - Protocol details diff --git a/docs/row-format/standard.md b/docs/row-format/standard.md deleted file mode 100644 index b420f1e9f5..0000000000 --- a/docs/row-format/standard.md +++ /dev/null @@ -1,57 +0,0 @@ ---- -title: Standard Row Format -sidebar_position: 1 -id: standard -license: | - Licensed to the Apache Software Foundation (ASF) under one or more - contributor license agreements. See the NOTICE file distributed with - this work for additional information regarding copyright ownership. - The ASF licenses this file to You under the Apache License, Version 2.0 - (the "License"); you may not use this file except in compliance with - the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. ---- - -Fory Row Format is a cache-friendly binary format for efficient random access -and partial deserialization. Unlike object graph serialization, it lets readers -access individual fields without reconstructing the complete object. - -## Features - -- **Zero-copy random access**: Read selected fields directly from encoded data. -- **Partial deserialization**: Reconstruct only the values an application needs. -- **Cross-language compatibility**: Share Standard Row bytes between Java, - Python, C++, and Rust. -- **Apache Arrow integration**: Convert rows to Arrow data in Java and Python. - -## Format Boundary - -Standard Row stores fixed-width values inline and variable-width values by -offset and size. Rows, arrays, and maps use a schema to resolve field positions -and element types. The normative byte layout, alignment rules, type table, and -endianness are defined by the -[Row Format Specification](../specification/row_format_spec.md). - -Row Format is intended only for trusted analytics data, including memory-mapped data, selective field -access, and data pipelines. Use [Object Serialization](../object-serialization/index.md) -when the application needs general object graphs, shared or circular references, -or complete object reconstruction as its primary access pattern. - -## Implementations - -| Runtime | Standard Row compatibility | Runtime guide | Additional integration | -| ------- | -------------------------- | ------------------- | ------------------------------------------------------ | -| Java | Compatible | [Java](java.md) | Arrow conversion; interface and extension-type mapping | -| Python | Compatible | [Python](python.md) | PyArrow schema and table conversion | -| C++ | Compatible | [C++](cpp.md) | Native row readers and writers | -| Rust | Compatible | [Rust](rust.md) | Borrowed struct, array, and map views | - -Use the runtime guides for installation, schema construction, encoding, random -access, partial reads, and language-specific integrations. diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/row-format/compact.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/row-format/compact.md deleted file mode 100644 index 98bfcffb60..0000000000 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/current/row-format/compact.md +++ /dev/null @@ -1,50 +0,0 @@ ---- -title: Compact Row -sidebar_position: 2 -id: compact -license: | - Licensed to the Apache Software Foundation (ASF) under one or more - contributor license agreements. See the NOTICE file distributed with - this work for additional information regarding copyright ownership. - The ASF licenses this file to You under the Apache License, Version 2.0 - (the "License"); you may not use this file except in compliance with - the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. ---- - -Compact Row 是仅支持 Java 的 Row 编码,可减少固定槽位和 null bitmap 开销。它与 Standard Row -的编码格式不兼容。 - -## 创建 Compact Encoder - -```java -RowEncoder<MyBean> encoder = - Encoders.buildBeanCodec(MyBean.class) - .compactEncoding() - .build() - .get(); - -BinaryRow row = encoder.toRow(value); -MyBean decoded = encoder.fromRow(row); -``` - -在单个线程中复用 encoder。并发线程应分别创建 encoder。 - -## 布局权衡 - -- 固定大小字段使用自然宽度,而不是 Standard Row 的八字节槽位。 -- 字段按对齐要求排序,以减少 padding。 -- 没有可空字段时省略 null bitmap。 -- 固定大小的嵌套 struct 可以内联存储。 - -仅当所有 reader 都使用 Java,且节省的空间足以抵消 Java 特有布局的限制时,才应选择 Compact Row。 -Java/Python/C++/Rust 互操作请使用 [Standard Row](standard.md)。 - -精确二进制布局请参阅 [Row Format 规范](../specification/row_format_spec.md)。 diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/row-format/index.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/row-format/index.md index 542e9cfb4d..ee24702372 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/current/row-format/index.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/row-format/index.md @@ -19,24 +19,77 @@ license: | limitations under the License. --- -Row Format 使用缓存友好的二进制布局存储有类型的值,无需重建完整对象图即可随机或部分访问。 -它适用于分析和内存数据处理。 +Row Format 是一种缓存友好的二进制格式,可实现高效的随机访问和部分反序列化。与对象图 +序列化不同,它允许读取方无需重建完整对象即可访问单个字段。 -## 选择 Row 系列 +Row Format 仅用于可信的分析数据,包括内存映射数据、选择性字段访问和数据管道。如果应用 +需要通用对象图、共享或循环引用,或者以完整对象重建为主要访问方式,请使用 +[对象序列化](../object-serialization/index.md)。 -| 系列 | 运行时支持 | 兼容性 | -| --------------------------- | ----------------------- | --------------------------- | -| [Standard Row](standard.md) | Java、Python、C++、Rust | 共享的 Standard Row 布局 | -| [Compact Row](compact.md) | Java | 仅 Java、面向空间优化的布局 | +## 选择布局 -如果目标是完整对象重建、引用处理或通用应用消息传递,请使用二进制对象序列化。如果工作负载需要 -直接从编码数据中读取选定字段、嵌套数组或 Map,请使用 Row Format。 +| 布局 | 运行时支持 | 兼容性 | +| ------------ | ----------------------- | --------------------------- | +| Standard Row | Java、Python、C++、Rust | 共享的 Standard Row 布局 | +| Compact Row | Java | 仅 Java、面向空间优化的布局 | -## 运行时指南 +## Standard Row -- [Java](java.md) -- [Python](python.md) -- [C++](cpp.md) -- [Rust](rust.md) +Standard Row 是 Java、Python、C++ 和 Rust 之间可互操作的布局。 -规范性 [Row Format 规范](../specification/row_format_spec.md)定义了 Standard 和 Compact 布局。 +### 功能特性 + +- **零拷贝随机访问**:直接从编码数据中读取选定字段。 +- **部分反序列化**:只重建应用所需的值。 +- **跨语言兼容**:在 Java、Python、C++ 和 Rust 之间共享 Standard Row 字节。 +- **Apache Arrow 集成**:在 Java 和 Python 中将 Row 转换为 Arrow 数据。 + +### 布局 + +Standard Row 内联存储固定宽度值,并通过偏移量和大小存储变长值。Row、数组和 Map 使用 +Schema 解析字段位置和元素类型。规范性的字节布局、对齐规则、类型表和字节序由 +[Row Format 规范](../specification/row_format_spec.md)定义。 + +### 运行时支持 + +| 运行时 | Standard Row 兼容性 | 运行时指南 | 其他集成 | +| ------ | ------------------- | ------------------- | ------------------------------ | +| Java | 兼容 | [Java](java.md) | Arrow 转换;接口和扩展类型映射 | +| Python | 兼容 | [Python](python.md) | PyArrow Schema 和 table 转换 | +| C++ | 兼容 | [C++](cpp.md) | 原生 Row 读取器和写入器 | +| Rust | 兼容 | [Rust](rust.md) | 借用式结构体、数组和 Map 视图 | + +安装、Schema 构建、编码、随机访问、部分读取和特定语言集成请参阅各运行时指南。 + +## Compact Row + +Compact Row 是仅支持 Java 的 Row 编码,可减少固定槽位和 null bitmap 开销。它与 Standard Row +的编码格式不兼容。 + +### 创建 Compact Encoder + +```java +RowEncoder<MyBean> encoder = + Encoders.buildBeanCodec(MyBean.class) + .compactEncoding() + .build() + .get(); + +BinaryRow row = encoder.toRow(value); +MyBean decoded = encoder.fromRow(row); +``` + +在单个线程中复用 encoder。并发线程应分别创建 encoder。 + +### 布局权衡 + +- 固定大小字段使用自然宽度,而不是 Standard Row 的八字节槽位。 +- 字段按对齐要求排序,以减少 padding。 +- 没有可空字段时省略 null bitmap。 +- 固定大小的嵌套 struct 可以内联存储。 + +仅当所有 reader 都使用 Java,且节省的空间足以抵消 Java 特有布局的限制时,才应选择 +Compact Row。Java、Python、C++ 和 Rust 互操作请使用 Standard Row。 + +精确的 Standard 和 Compact 二进制布局请参阅 +[Row Format 规范](../specification/row_format_spec.md)。 diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/row-format/rust.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/row-format/rust.md index fbc27f9918..13f9cafe35 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/current/row-format/rust.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/row-format/rust.md @@ -185,5 +185,5 @@ assert!(scores.get(scores.len()).is_err()); ## 相关主题 - [基本序列化](../object-serialization/rust/core-api.md) - 对象图序列化 -- [标准行格式](standard.md) - Java、Python、C++ 和 Rust 的共享布局 +- [标准行格式](index.md#standard-row) - Java、Python、C++ 和 Rust 的共享布局 - [行格式规范](../specification/row_format_spec.md) - 协议详情 diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/row-format/standard.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/row-format/standard.md deleted file mode 100644 index 4ac61321e6..0000000000 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/current/row-format/standard.md +++ /dev/null @@ -1,51 +0,0 @@ ---- -title: Standard Row Format -sidebar_position: 1 -id: standard -license: | - Licensed to the Apache Software Foundation (ASF) under one or more - contributor license agreements. See the NOTICE file distributed with - this work for additional information regarding copyright ownership. - The ASF licenses this file to You under the Apache License, Version 2.0 - (the "License"); you may not use this file except in compliance with - the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. ---- - -Fory Row Format 是一种缓存友好的二进制格式,可实现高效的随机访问和部分反序列化。 -与对象图序列化不同,它允许读取方无需重建完整对象即可访问单个字段。 - -## 功能特性 - -- **零拷贝随机访问**:直接从编码数据中读取选定字段。 -- **部分反序列化**:只重建应用所需的值。 -- **跨语言兼容**:在 Java、Python、C++ 和 Rust 之间共享 Standard Row 字节。 -- **Apache Arrow 集成**:在 Java 和 Python 中将 Row 转换为 Arrow 数据。 - -## 格式边界 - -Standard Row 内联存储固定宽度值,并通过偏移量和大小存储变长值。Row、数组和 Map 使用 -Schema 解析字段位置和元素类型。规范性的字节布局、对齐规则、类型表和字节序由 -[Row Format 规范](../specification/row_format_spec.md)中定义。 - -Row Format 仅用于可信的分析数据,适合分析、内存映射数据、选择性字段访问和数据管道。 -如果应用需要通用对象图、共享或循环引用,或者以完整对象重建为主要访问方式,请使用 -[对象序列化](../object-serialization/index.md)。 - -## 实现 - -| 运行时 | Standard Row 兼容性 | 运行时指南 | 其他集成 | -| ------ | ------------------- | ------------------- | ------------------------------ | -| Java | 兼容 | [Java](java.md) | Arrow 转换;接口和扩展类型映射 | -| Python | 兼容 | [Python](python.md) | PyArrow Schema 和 table 转换 | -| C++ | 兼容 | [C++](cpp.md) | 原生 Row 读取器和写入器 | -| Rust | 兼容 | [Rust](rust.md) | 借用式结构体、数组和 Map 视图 | - -安装、Schema 构建、编码、随机访问、部分读取和特定语言集成请参阅各运行时指南。 --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
