This is an automated email from the ASF dual-hosted git repository.
chaokunyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/fory.git
The following commit(s) were added to refs/heads/main by this push:
new 1171637d4 feat(rust): enhance rust row format (#3915)
1171637d4 is described below
commit 1171637d4e4f1ca569844afc8228b771709b9998
Author: Shawn Yang <[email protected]>
AuthorDate: Mon Aug 3 23:32:21 2026 +0800
feat(rust): enhance rust row format (#3915)
<!--
**Thanks for contributing to Apache Fory™.**
**If this is your first time opening a PR on fory, you can refer to
[CONTRIBUTING.md](https://github.com/apache/fory/blob/main/CONTRIBUTING.md).**
Contribution Checklist
- The **Apache Fory™** community has requirements on the naming of pr
titles. You can also find instructions in
[CONTRIBUTING.md](https://github.com/apache/fory/blob/main/CONTRIBUTING.md).
- Apache Fory™ has a strong focus on performance. If the PR you submit
will have an impact on performance, please benchmark it first and
provide the benchmark result here.
-->
## Why?
<!-- Describe the purpose of this PR. -->
## What does this PR do?
<!-- Describe the details of this PR. -->
## Related issues
<!--
Is there any related issue? If this PR closes them you say say
fix/closes:
- #xxxx0
- #xxxx1
- Fixes #xxxx2
-->
## AI Contribution Checklist
<!-- Full requirements and disclosure template:
https://github.com/apache/fory/blob/main/AI_POLICY.md#9-contributor-checklist-for-ai-assisted-prs
-->
- [ ] Substantial AI assistance was used in this PR: `yes` / `no`
- [ ] If `yes`, I included a completed [AI Contribution
Checklist](https://github.com/apache/fory/blob/main/AI_POLICY.md#9-contributor-checklist-for-ai-assisted-prs)
in this PR description and the required `AI Usage Disclosure`.
- [ ] If `yes`, my PR description includes the required `ai_review`
summary and screenshot evidence or equivalent persisted links of the
final clean AI review results from both fresh reviewers described in
`AI_POLICY.md`, the Fory-guided reviewer and the independent general
reviewer, on the current PR diff or current HEAD after the latest code
changes.
<!-- If substantial AI assistance = `yes`, paste the completed checklist
and disclosure block here, including the final ai_review summary and
screenshot evidence or equivalent persisted links from both fresh
reviewers described in AI_POLICY.md, the Fory-guided reviewer and the
independent general reviewer, on the current PR diff or current HEAD
after the latest code changes. -->
## Does this PR introduce any user-facing change?
<!--
If any user-facing interface changes, please [open an
issue](https://github.com/apache/fory/issues/new/choose) describing the
need to do so and update the document if necessary.
Delete section if not applicable.
-->
- [ ] Does this PR introduce any public API change?
- [ ] Does this PR introduce any binary protocol compatibility change?
## Benchmark
<!--
When the PR has an impact on performance (if you don't know whether the
PR will have an impact on performance, you can submit the PR first, and
if it will have impact on performance, the code reviewer will explain
it), be sure to attach a benchmark data here.
Delete section if not applicable.
-->
---
docs/guide/rust/row-format.md | 34 ++++++-
rust/README.md | 11 ++-
rust/api-tests/facade-only/src/lib.rs | 63 ++++++++++--
rust/api-tests/renamed-facade/src/lib.rs | 17 ++--
rust/fory-core/src/row/mod.rs | 4 +-
rust/fory-core/src/row/reader.rs | 120 +++++++++++++++++++++-
rust/fory-core/src/row/writer.rs | 22 ++++-
rust/fory-derive/src/fory_row.rs | 28 +++++-
rust/fory-derive/src/lib.rs | 1 +
rust/fory/src/lib.rs | 2 +-
rust/tests/tests/test_row.rs | 165 ++++++++++++++++++++++++++++++-
11 files changed, 434 insertions(+), 33 deletions(-)
diff --git a/docs/guide/rust/row-format.md b/docs/guide/rust/row-format.md
index 5ae2e3703..1479ae752 100644
--- a/docs/guide/rust/row-format.md
+++ b/docs/guide/rust/row-format.md
@@ -38,7 +38,7 @@ Row Format is schema-driven: the Rust type supplied to
`from_row` determines the
## Basic Usage
```rust
-use fory::{from_row, to_row, Error, ForyRow};
+use fory::{from_row, to_row, Error, ForyRow, RowView};
use std::collections::BTreeMap;
#[derive(ForyRow)]
@@ -77,17 +77,41 @@ fn main() -> Result<(), Error> {
assert_eq!(scores.len(), 4);
assert_eq!(scores.get(0)?, 95);
assert_eq!(scores.get(1)?, 87);
+ assert_eq!(
+ scores.iter().collect::<Result<Vec<_>, _>>()?,
+ [95, 87, 92, 88]
+ );
let preferences = row.preferences()?;
- assert_eq!(preferences.keys().len(), 2);
- assert_eq!(preferences.keys().get(0)?, "language");
- assert_eq!(preferences.values().get(0)?, "en");
+ assert_eq!(preferences.len(), 2);
+ assert_eq!(preferences.key(0)?, "language");
+ assert_eq!(preferences.value(0)?, "en");
+ assert_eq!(row.as_bytes(), row_data);
Ok(())
}
```
`to_row` accepts Row Format roots: derived structs, supported arrays, and
`BTreeMap` values. Scalar, string, binary, and `Option<T>` values are field or
element values rather than standalone roots.
+## View Traversal and Buffer Reuse
+
+`ArrayView::iter` and `IntoIterator for &ArrayView` read elements on demand
through the same checked path as `get`. Each item is a `Result`, so malformed
data is reported when that element is visited.
+
+`MapView` exposes `len`, `is_empty`, `key(index)`, and `value(index)`. Its
`keys()` and `values()` array views remain available for independent iteration.
+
+Struct, array, and map views are cheap `Copy` and `Clone` values. The
`RowView` trait provides `as_bytes()`, which returns the exact encoded slice
bound to the view, and `encoded_len()`, which returns its length. A nested view
returns only its size-delimited child bytes.
+
+Use `to_row_into` to replace a caller-owned buffer while retaining its
capacity:
+
+```rust
+use fory::to_row_into;
+
+let mut row_data = Vec::with_capacity(4096);
+to_row_into(&vec![1i32, 2, 3], &mut row_data).unwrap();
+```
+
+Repeated calls discard the previous logical contents. If encoding returns an
error, the buffer is left empty. Row framing remains the application's
responsibility.
+
## Nullability and Field Order
`Option<T>` declares a nullable field or array element. `None` sets the
corresponding null bit, and the field method returns `None` without reading a
value body. `Some(value)` uses the same fixed slot width as `T`.
@@ -131,7 +155,7 @@ For the normative layout and size formulas, see the [Row
Format Specification](h
## Validation and Errors
-`from_row`, generated field methods, and array `get` calls return `Result`.
They reject truncated fixed regions, invalid counts, out-of-range offsets and
sizes, invalid UTF-8, fixed-array length mismatches, and mismatched map
key/value counts.
+`from_row`, generated field methods, array `get`/iteration, and map indexed
access return `Result`. They reject truncated fixed regions, invalid counts,
out-of-range offsets and sizes, invalid UTF-8, fixed-array length mismatches,
and mismatched map key/value counts.
Array access is also bounds-checked:
diff --git a/rust/README.md b/rust/README.md
index f5936d385..8558c5569 100644
--- a/rust/README.md
+++ b/rust/README.md
@@ -543,7 +543,7 @@ Apache Fory™ Rust implements the Standard Row Format shared
with Java, C++, an
- Standard Row Format interchange with other Fory implementations
```rust
-use fory::{from_row, to_row, Error, ForyRow};
+use fory::{from_row, to_row, Error, ForyRow, RowView};
#[derive(ForyRow)]
struct UserProfile {
@@ -572,17 +572,22 @@ fn main() -> Result<(), Error> {
let scores = row.scores()?;
assert_eq!(scores.len(), 4);
assert_eq!(scores.get(1)?, 87);
+ assert_eq!(
+ scores.iter().collect::<Result<Vec<_>, _>>()?,
+ [95, 87, 92, 88]
+ );
+ assert_eq!(row.as_bytes(), bytes);
Ok(())
}
```
-`#[derive(ForyRow)]` supports named structs, including generic structs, and
encodes fields in source declaration order. `Option<T>` supplies field or
array-element nullability without changing `T`'s slot width. Generated field
methods and array `get` calls return `Result`, validating variable ranges and
UTF-8 when accessed.
+`#[derive(ForyRow)]` supports named structs, including generic structs, and
encodes fields in source declaration order. `Option<T>` supplies field or
array-element nullability without changing `T`'s slot width. Generated field
methods, array access and iteration, and map indexed access return `Result`,
validating variable ranges and UTF-8 when accessed. Immutable views are cheap
`Copy` values, and the `RowView` trait exposes their exact encoded slice
through `as_bytes`.
Supported fixed-width values are `bool`, `i8`, `i16`, `i32`, `i64`, `f32`,
`f64`, `Date`, `Timestamp`, and `Duration`. Supported variable-width values are
UTF-8 `String`/`&str`, binary `Vec<u8>`/`&[u8]`, fixed and dynamic arrays over
supported element types, `BTreeMap`, nested derived structs, and `Option<T>`.
`Vec<u8>` uses the binary encoding rather than the Standard Array encoding.
`Float16` and `Decimal` are not supported by Row Format because the standard
specification does not defi [...]
Standard rows use an 8-byte-aligned null bitmap and one 8-byte slot per struct
field. Fixed-width fields are stored little-endian in their slots.
Variable-width slots encode the little-endian `u64` value `(relative_offset <<
32) | size`; variable bodies and array slot regions have zero padding to 8-byte
alignment. Standard arrays use natural-width storage for fixed elements, and
maps contain complete key and value arrays.
-`to_row` accepts derived structs, supported arrays, and `BTreeMap` roots.
Scalar, string, binary, and `Option<T>` values are field or element values
rather than standalone roots. See the [Rust Row Format
guide](https://fory.apache.org/docs/guide/rust/row-format) and [Row Format
specification](https://fory.apache.org/docs/specification/row_format_spec) for
details.
+`to_row` accepts derived structs, supported arrays, and `BTreeMap` roots.
`to_row_into` writes the same bytes into a reusable caller-owned `Vec<u8>` and
clears partial output on error. Scalar, string, binary, and `Option<T>` values
are field or element values rather than standalone roots. See the [Rust Row
Format guide](https://fory.apache.org/docs/guide/rust/row-format) and [Row
Format
specification](https://fory.apache.org/docs/specification/row_format_spec) for
details.
## Cross-Language Serialization
diff --git a/rust/api-tests/facade-only/src/lib.rs
b/rust/api-tests/facade-only/src/lib.rs
index 726d5abce..53974b6b1 100644
--- a/rust/api-tests/facade-only/src/lib.rs
+++ b/rust/api-tests/facade-only/src/lib.rs
@@ -348,6 +348,43 @@
//! pub value: i64,
//! }
//! ```
+//!
+//! Public Row views do not expose private field types in declaration bounds:
+//!
+//! ```
+//! #![deny(private_bounds)]
+//! use fory::ForyRow;
+//!
+//! #[derive(ForyRow)]
+//! struct PrivateChild {
+//! value: i32,
+//! }
+//!
+//! #[derive(ForyRow)]
+//! pub struct PublicParent {
+//! child: PrivateChild,
+//! }
+//! ```
+//!
+//! Generated Row views preserve generic parameter defaults:
+//!
+//! ```
+//! use fory::{from_row, to_row, ForyRow};
+//!
+//! #[derive(ForyRow)]
+//! struct DefaultSchema<T = i32, const N: usize = 2> {
+//! value: T,
+//! values: [T; N],
+//! }
+//!
+//! let value: DefaultSchema = DefaultSchema {
+//! value: 7,
+//! values: [8, 9],
+//! };
+//! let bytes = to_row(&value).unwrap();
+//! let view: DefaultSchemaRowView<'_> =
from_row::<DefaultSchema>(&bytes).unwrap();
+//! assert_eq!(view.value().unwrap(), 7);
+//! ```
use fory::ForyRow;
@@ -369,12 +406,13 @@ impl VisibilitySchema {
#[cfg(test)]
mod tests {
use fory::{
- from_row, register_trait_type, to_row, ArcSerializer,
ArcWeakSerializer, ArraySerializer,
- ArrayView, BTreeMapSerializer, BTreeSetSerializer,
BinaryHeapSerializer, BoxSerializer,
- Error, Fory, ForyEnum, ForyObject, ForyRow, ForyStruct, ForyUnion,
HashMapSerializer,
- HashSetSerializer, LinkedListSerializer, MapView, MutexSerializer,
OptionSerializer,
- RcSerializer, RcWeakSerializer, ReadContext, Reader,
RefCellSerializer, Row, Serializer,
- VecDequeSerializer, VecSerializer, WriteContext,
+ from_row, register_trait_type, to_row, to_row_into, ArcSerializer,
ArcWeakSerializer,
+ ArrayIter, ArraySerializer, ArrayView, BTreeMapSerializer,
BTreeSetSerializer,
+ BinaryHeapSerializer, BoxSerializer, Error, Fory, ForyEnum,
ForyObject, ForyRow,
+ ForyStruct, ForyUnion, HashMapSerializer, HashSetSerializer,
LinkedListSerializer, MapView,
+ MutexSerializer, OptionSerializer, RcSerializer, RcWeakSerializer,
ReadContext, Reader,
+ RefCellSerializer, Row, RowView, Serializer, VecDequeSerializer,
VecSerializer,
+ WriteContext,
};
use fory_external_model::{Command, ExternalId, Key, Marker, Point, Status,
User, Value};
use std::collections::{BTreeMap, HashMap};
@@ -481,6 +519,8 @@ mod tests {
fn assert_array_view(_: &ArrayView<'_, i32>) {}
+ fn assert_array_iter(_: ArrayIter<'_, '_, i32>) {}
+
fn assert_map_view(_: &MapView<'_, String, i32>) {}
trait Animal: ForyObject {
@@ -679,14 +719,23 @@ mod tests {
let decoded = from_row::<RowUser>(&row).unwrap();
assert_eq!(decoded.id().unwrap(), 7);
assert_eq!(decoded.name().unwrap(), "Grace");
+ assert_eq!(decoded.as_bytes(), row);
+ let copied = decoded;
+ assert_eq!(copied.encoded_len(), row.len());
- let array_bytes = to_row(&vec![1i32]).unwrap();
+ let mut array_bytes = Vec::new();
+ to_row_into(&vec![1i32], &mut array_bytes).unwrap();
let array = from_row::<Vec<i32>>(&array_bytes).unwrap();
assert_array_view(&array);
+ assert_array_iter(array.iter());
+ assert_eq!(array.iter().next().unwrap().unwrap(), 1);
let map_bytes = to_row(&BTreeMap::from([("one".to_owned(),
1i32)])).unwrap();
let map = from_row::<BTreeMap<String, i32>>(&map_bytes).unwrap();
assert_map_view(&map);
+ assert_eq!(map.len(), 1);
+ assert_eq!(map.key(0).unwrap(), "one");
+ assert_eq!(map.value(0).unwrap(), 1);
}
#[test]
diff --git a/rust/api-tests/renamed-facade/src/lib.rs
b/rust/api-tests/renamed-facade/src/lib.rs
index 54a1dbaad..51ae598e5 100644
--- a/rust/api-tests/renamed-facade/src/lib.rs
+++ b/rust/api-tests/renamed-facade/src/lib.rs
@@ -19,12 +19,12 @@
mod tests {
use fory_external_model::{ExternalId, User};
use fory_facade::{
- from_row, register_trait_type, to_row, ArcSerializer,
ArcWeakSerializer, ArraySerializer,
- BTreeMapSerializer, BTreeSetSerializer, BinaryHeapSerializer,
BoxSerializer, Error, Fory,
- ForyEnum, ForyObject, ForyRow, ForyStruct, ForyUnion,
HashMapSerializer, HashSetSerializer,
- LinkedListSerializer, MutexSerializer, OptionSerializer, RcSerializer,
RcWeakSerializer,
- ReadContext, RefCellSerializer, Serializer, VecDequeSerializer,
VecSerializer,
- WriteContext,
+ from_row, register_trait_type, to_row, to_row_into, ArcSerializer,
ArcWeakSerializer,
+ ArraySerializer, BTreeMapSerializer, BTreeSetSerializer,
BinaryHeapSerializer,
+ BoxSerializer, Error, Fory, ForyEnum, ForyObject, ForyRow, ForyStruct,
ForyUnion,
+ HashMapSerializer, HashSetSerializer, LinkedListSerializer,
MutexSerializer,
+ OptionSerializer, RcSerializer, RcWeakSerializer, ReadContext,
RefCellSerializer, RowView,
+ Serializer, VecDequeSerializer, VecSerializer, WriteContext,
};
use std::collections::HashMap;
use std::rc::Rc;
@@ -234,9 +234,12 @@ mod tests {
#[test]
fn renamed_facade_row_derive() {
- let row = to_row(&RenamedRow { id: 9 }).unwrap();
+ let mut row = Vec::new();
+ to_row_into(&RenamedRow { id: 9 }, &mut row).unwrap();
let decoded = from_row::<RenamedRow>(&row).unwrap();
assert_eq!(decoded.id().unwrap(), 9);
+ assert_eq!(decoded.as_bytes(), row);
+ assert_eq!(row, to_row(&RenamedRow { id: 9 }).unwrap());
}
#[test]
diff --git a/rust/fory-core/src/row/mod.rs b/rust/fory-core/src/row/mod.rs
index 7cf81900e..f094f76c9 100644
--- a/rust/fory-core/src/row/mod.rs
+++ b/rust/fory-core/src/row/mod.rs
@@ -23,10 +23,10 @@ mod writer;
#[doc(hidden)]
pub use reader::StructView;
-pub use reader::{from_row, ArrayView, MapView};
+pub use reader::{from_row, ArrayIter, ArrayView, MapView, RowView};
pub use row::Row;
#[doc(hidden)]
pub use row::RowValue;
-pub use writer::to_row;
+pub use writer::{to_row, to_row_into};
#[doc(hidden)]
pub use writer::{StructWriter, ValueWriter};
diff --git a/rust/fory-core/src/row/reader.rs b/rust/fory-core/src/row/reader.rs
index 613fe4938..f3f853e8b 100644
--- a/rust/fory-core/src/row/reader.rs
+++ b/rust/fory-core/src/row/reader.rs
@@ -23,11 +23,24 @@ use crate::error::Error;
use super::bit_util::{bitmap_width, is_bit_set, round_up_to_word, slot_width};
use super::row::{Row, RowValue};
+/// Backing-byte access shared by immutable Standard Row Format views.
+pub trait RowView<'a> {
+ /// Returns the complete encoded bytes bound to this view.
+ fn as_bytes(&self) -> &'a [u8];
+
+ /// Returns the number of encoded bytes bound to this view.
+ #[inline]
+ fn encoded_len(&self) -> usize {
+ self.as_bytes().len()
+ }
+}
+
/// A zero-copy view over one Standard Row Format struct.
///
/// This type is public only because `ForyRow` views are generated in
/// downstream crates.
#[doc(hidden)]
+#[derive(Clone, Copy)]
pub struct StructView<'a> {
bytes: &'a [u8],
bitmap_width: usize,
@@ -91,6 +104,13 @@ impl<'a> StructView<'a> {
}
}
+impl<'a> RowView<'a> for StructView<'a> {
+ #[inline]
+ fn as_bytes(&self) -> &'a [u8] {
+ self.bytes
+ }
+}
+
/// A zero-copy view over one Standard Row Format array.
pub struct ArrayView<'a, T: RowValue> {
bytes: &'a [u8],
@@ -102,6 +122,14 @@ pub struct ArrayView<'a, T: RowValue> {
marker: PhantomData<T>,
}
+impl<T: RowValue> Copy for ArrayView<'_, T> {}
+
+impl<T: RowValue> Clone for ArrayView<'_, T> {
+ fn clone(&self) -> Self {
+ *self
+ }
+}
+
impl<'a, T: RowValue> ArrayView<'a, T> {
pub(crate) fn new(bytes: &'a [u8]) -> Result<Self, Error> {
let count = read_u64(bytes, 0)?;
@@ -141,6 +169,14 @@ impl<'a, T: RowValue> ArrayView<'a, T> {
self.num_elements == 0
}
+ /// Returns an iterator that reads elements on demand.
+ pub fn iter(&self) -> ArrayIter<'_, 'a, T> {
+ ArrayIter {
+ view: self,
+ index: 0,
+ }
+ }
+
/// Reads one array element without materializing the rest of the array.
pub fn get(&self, index: usize) -> Result<T::View<'a>, Error> {
self.check_index(index)?;
@@ -176,12 +212,63 @@ impl<'a, T: RowValue> ArrayView<'a, T> {
}
}
+impl<'a, T: RowValue> RowView<'a> for ArrayView<'a, T> {
+ #[inline]
+ fn as_bytes(&self) -> &'a [u8] {
+ self.bytes
+ }
+}
+
+/// An iterator over the elements of an [`ArrayView`].
+pub struct ArrayIter<'view, 'row, T: RowValue> {
+ view: &'view ArrayView<'row, T>,
+ index: usize,
+}
+
+impl<'row, T: RowValue> Iterator for ArrayIter<'_, 'row, T> {
+ type Item = Result<T::View<'row>, Error>;
+
+ fn next(&mut self) -> Option<Self::Item> {
+ if self.index == self.view.len() {
+ return None;
+ }
+ let index = self.index;
+ self.index += 1;
+ Some(self.view.get(index))
+ }
+
+ fn size_hint(&self) -> (usize, Option<usize>) {
+ let remaining = self.view.len() - self.index;
+ (remaining, Some(remaining))
+ }
+}
+
+impl<T: RowValue> ExactSizeIterator for ArrayIter<'_, '_, T> {}
+
+impl<'view, 'row, T: RowValue> IntoIterator for &'view ArrayView<'row, T> {
+ type Item = Result<T::View<'row>, Error>;
+ type IntoIter = ArrayIter<'view, 'row, T>;
+
+ fn into_iter(self) -> Self::IntoIter {
+ self.iter()
+ }
+}
+
/// A zero-copy view over one Standard Row Format map.
pub struct MapView<'a, K: RowValue, V: RowValue> {
+ bytes: &'a [u8],
keys: ArrayView<'a, K>,
values: ArrayView<'a, V>,
}
+impl<K: RowValue, V: RowValue> Copy for MapView<'_, K, V> {}
+
+impl<K: RowValue, V: RowValue> Clone for MapView<'_, K, V> {
+ fn clone(&self) -> Self {
+ *self
+ }
+}
+
impl<'a, K: RowValue, V: RowValue> MapView<'a, K, V> {
pub(crate) fn new(bytes: &'a [u8]) -> Result<Self, Error> {
let keys_size = read_u64(bytes, 0)?;
@@ -198,7 +285,31 @@ impl<'a, K: RowValue, V: RowValue> MapView<'a, K, V> {
"row map key and value arrays have different lengths",
));
}
- Ok(Self { keys, values })
+ Ok(Self {
+ bytes,
+ keys,
+ values,
+ })
+ }
+
+ /// Returns the number of key-value pairs encoded in this map.
+ pub fn len(&self) -> usize {
+ self.keys.len()
+ }
+
+ /// Returns true when this map contains no key-value pairs.
+ pub fn is_empty(&self) -> bool {
+ self.keys.is_empty()
+ }
+
+ /// Reads the key at `index` without materializing the map.
+ pub fn key(&self, index: usize) -> Result<K::View<'a>, Error> {
+ self.keys.get(index)
+ }
+
+ /// Reads the value at `index` without materializing the map.
+ pub fn value(&self, index: usize) -> Result<V::View<'a>, Error> {
+ self.values.get(index)
}
/// Returns the map's key array.
@@ -226,6 +337,13 @@ impl<'a, K: RowValue, V: RowValue> MapView<'a, K, V> {
}
}
+impl<'a, K: RowValue, V: RowValue> RowView<'a> for MapView<'a, K, V> {
+ #[inline]
+ fn as_bytes(&self) -> &'a [u8] {
+ self.bytes
+ }
+}
+
fn variable_slice(bytes: &[u8], slot_offset: usize, fixed_end: usize) ->
Result<&[u8], Error> {
let offset_and_size = read_u64(bytes, slot_offset)?;
let relative_offset = usize::try_from(offset_and_size >> 32)
diff --git a/rust/fory-core/src/row/writer.rs b/rust/fory-core/src/row/writer.rs
index 41f13f8aa..7548b16b4 100644
--- a/rust/fory-core/src/row/writer.rs
+++ b/rust/fory-core/src/row/writer.rs
@@ -32,7 +32,8 @@ enum WriteTarget<'a, 'b> {
/// The exact destination for one Row Format value.
///
/// This type is public only because `ForyRow` implementations are generated in
-/// downstream crates. Applications should use [`to_row`] instead.
+/// downstream crates. Applications should use [`to_row`] or [`to_row_into`]
+/// instead.
#[doc(hidden)]
pub struct ValueWriter<'a, 'b> {
target: WriteTarget<'a, 'b>,
@@ -321,7 +322,22 @@ fn write_variable<T: RowValue + ?Sized>(
/// Encodes a struct, array, or map as a Standard Row Format root.
pub fn to_row<T: Row + ?Sized>(value: &T) -> Result<Vec<u8>, Error> {
let mut buffer = Vec::new();
- let mut writer = Writer::from_buffer(&mut buffer);
- value.write(ValueWriter::variable(&mut writer))?;
+ to_row_into(value, &mut buffer)?;
Ok(buffer)
}
+
+/// Replaces `buffer` with one encoded Standard Row Format root.
+///
+/// Existing capacity is retained for reuse. If encoding fails, `buffer` is
+/// left empty.
+pub fn to_row_into<T: Row + ?Sized>(value: &T, buffer: &mut Vec<u8>) ->
Result<(), Error> {
+ buffer.clear();
+ let result = {
+ let mut writer = Writer::from_buffer(buffer);
+ value.write(ValueWriter::variable(&mut writer))
+ };
+ if result.is_err() {
+ buffer.clear();
+ }
+ result
+}
diff --git a/rust/fory-derive/src/fory_row.rs b/rust/fory-derive/src/fory_row.rs
index f7145d2fb..59fa41234 100644
--- a/rust/fory-derive/src/fory_row.rs
+++ b/rust/fory-derive/src/fory_row.rs
@@ -86,7 +86,7 @@ fn expand_row(
let mut view_generics = SelfTypeRewriter {
source_path: source_path.clone(),
}
- .fold_generics(row_generics.clone());
+ .fold_generics(ast.generics.clone());
view_generics.params.insert(
0,
GenericParam::Lifetime(LifetimeParam::new(row_lifetime.clone())),
@@ -118,7 +118,10 @@ fn expand_row(
) -> ::core::result::Result<
<#field_ty as
#runtime_root::row::RowValue>::View<#row_lifetime>,
#runtime_root::error::Error,
- > {
+ >
+ where
+ #field_ty: #runtime_root::row::RowValue,
+ {
self.struct_data.get::<#field_ty>(#index)
}
});
@@ -126,7 +129,7 @@ fn expand_row(
Ok(quote! {
#[doc = #view_doc]
- #visibility struct #view #view_impl_generics #view_where_clause {
+ #visibility struct #view #view_generics #view_where_clause {
struct_data: #runtime_root::row::StructView<#row_lifetime>,
_marker: ::core::marker::PhantomData<fn() -> *const #name
#ty_generics>,
}
@@ -135,6 +138,25 @@ fn expand_row(
#(#field_methods)*
}
+ impl #view_impl_generics ::core::marker::Copy for #view
#view_ty_generics #view_where_clause {}
+
+ impl #view_impl_generics ::core::clone::Clone for #view
#view_ty_generics #view_where_clause {
+ fn clone(&self) -> Self {
+ *self
+ }
+ }
+
+ // Backing bytes are a trait capability so schema fields named
+ // `as_bytes` or `encoded_len` keep owning their generated methods.
+ impl #view_impl_generics #runtime_root::row::RowView<#row_lifetime>
+ for #view #view_ty_generics #view_where_clause
+ {
+ #[inline]
+ fn as_bytes(&self) -> &#row_lifetime [u8] {
+ #runtime_root::row::RowView::as_bytes(&self.struct_data)
+ }
+ }
+
impl #impl_generics #runtime_root::row::RowValue for #name
#ty_generics #where_clause {
type View<#row_lifetime> = #view #view_ty_generics;
diff --git a/rust/fory-derive/src/lib.rs b/rust/fory-derive/src/lib.rs
index d30efe199..0a7b8e766 100644
--- a/rust/fory-derive/src/lib.rs
+++ b/rust/fory-derive/src/lib.rs
@@ -121,6 +121,7 @@
//! The macro generates:
//! - A `RowValue` implementation and a root `Row` marker implementation
//! - A borrowed view type whose visibility matches the source struct
+//! - `RowView` backing-byte access and cheap `Copy`/`Clone` views
//! - One declaration-order field method preserving each source field's
visibility
//! - Field methods returning `Result<<Field as RowValue>::View<'_>, Error>`
//!
diff --git a/rust/fory/src/lib.rs b/rust/fory/src/lib.rs
index aa7914870..3b73bb27e 100644
--- a/rust/fory/src/lib.rs
+++ b/rust/fory/src/lib.rs
@@ -92,7 +92,7 @@ pub use fory_core::{
fory::Fory,
fory::ForyBuilder,
register_trait_type,
- row::{from_row, to_row, ArrayView, MapView, Row},
+ row::{from_row, to_row, to_row_into, ArrayIter, ArrayView, MapView, Row,
RowView},
ArcSerializer, ArcWeak, ArcWeakSerializer, ArraySerializer, BFloat16,
BTreeMapSerializer,
BTreeSetSerializer, BinaryHeapSerializer, BoxSerializer, Date, Decimal,
Duration, Float16,
ForyObject, HashMapSerializer, HashSetSerializer, LinkedListSerializer,
MutexSerializer,
diff --git a/rust/tests/tests/test_row.rs b/rust/tests/tests/test_row.rs
index 31cec0e04..60158fa25 100644
--- a/rust/tests/tests/test_row.rs
+++ b/rust/tests/tests/test_row.rs
@@ -17,7 +17,7 @@
use std::collections::BTreeMap;
-use fory_core::row::{from_row, to_row};
+use fory_core::row::{from_row, to_row, to_row_into, RowView};
use fory_core::types::{Date, Duration, Timestamp};
use fory_derive::ForyRow;
@@ -97,6 +97,17 @@ where
value: T,
}
+#[derive(ForyRow)]
+struct ByteMethodFields {
+ as_bytes: i32,
+ encoded_len: i32,
+}
+
+#[derive(ForyRow)]
+struct GenericValue<T> {
+ value: T,
+}
+
#[test]
fn standard_row_bytes() {
let bytes = to_row(&MixedRow {
@@ -368,3 +379,155 @@ fn container_shape_is_validated() {
mismatched[40..48].copy_from_slice(&2u64.to_le_bytes());
assert!(from_row::<BTreeMap<String, i32>>(&mismatched).is_err());
}
+
+#[test]
+fn collection_view_navigation() {
+ let fixed_bytes = to_row(&vec![1i32, 2, 3]).unwrap();
+ let fixed = from_row::<Vec<i32>>(&fixed_bytes).unwrap();
+ let mut fixed_iter = fixed.iter();
+ assert_eq!(fixed_iter.len(), 3);
+ assert_eq!(fixed_iter.next().unwrap().unwrap(), 1);
+ assert_eq!(fixed_iter.len(), 2);
+ assert_eq!(fixed_iter.collect::<Result<Vec<_>, _>>().unwrap(), [2, 3]);
+
+ let optional_bytes = to_row(&vec![Some("a".to_owned()), None,
Some("b".to_owned())]).unwrap();
+ let optional = from_row::<Vec<Option<String>>>(&optional_bytes).unwrap();
+ assert_eq!(
+ (&optional)
+ .into_iter()
+ .collect::<Result<Vec<_>, _>>()
+ .unwrap(),
+ [Some("a"), None, Some("b")]
+ );
+
+ let nested_bytes = to_row(&vec![NestedChild { value: 4 }, NestedChild {
value: 5 }]).unwrap();
+ let nested = from_row::<Vec<NestedChild>>(&nested_bytes).unwrap();
+ let nested_values = nested
+ .iter()
+ .map(|value| value.and_then(|view| view.value()))
+ .collect::<Result<Vec<_>, _>>()
+ .unwrap();
+ assert_eq!(nested_values, [4, 5]);
+
+ let map_bytes = to_row(&BTreeMap::from([
+ ("a".to_owned(), 1i32),
+ ("b".to_owned(), 2i32),
+ ]))
+ .unwrap();
+ let map = from_row::<BTreeMap<String, i32>>(&map_bytes).unwrap();
+ assert_eq!(map.len(), 2);
+ assert!(!map.is_empty());
+ assert_eq!(map.key(0).unwrap(), "a");
+ assert_eq!(map.value(0).unwrap(), 1);
+ assert!(map.key(map.len()).is_err());
+ assert!(map.value(map.len()).is_err());
+
+ let empty_bytes = to_row(&BTreeMap::<String, i32>::new()).unwrap();
+ let empty = from_row::<BTreeMap<String, i32>>(&empty_bytes).unwrap();
+ assert!(empty.is_empty());
+ assert_eq!(empty.len(), 0);
+ assert_eq!(empty.keys().iter().len(), 0);
+}
+
+#[test]
+fn iteration_is_lazy() {
+ let mut bytes = to_row(&vec!["a".to_owned(), "b".to_owned()]).unwrap();
+ bytes[24..32].copy_from_slice(&(((40u64) << 32) | 100).to_le_bytes());
+
+ let view = from_row::<Vec<String>>(&bytes).unwrap();
+ let mut iter = view.iter();
+ assert_eq!(iter.next().unwrap().unwrap(), "a");
+ assert!(iter.next().unwrap().is_err());
+ assert!(iter.next().is_none());
+}
+
+#[test]
+fn view_bytes_and_copies() {
+ fn copy<T: Copy>(value: T) -> T {
+ value
+ }
+
+ fn clone<T: Clone>(value: &T) -> T {
+ value.clone()
+ }
+
+ let nested_bytes = to_row(&NestedParent {
+ child: NestedChild { value: 7 },
+ })
+ .unwrap();
+ let parent = from_row::<NestedParent>(&nested_bytes).unwrap();
+ assert_eq!(parent.as_bytes(), nested_bytes);
+ assert_eq!(parent.encoded_len(), nested_bytes.len());
+ assert_eq!(copy(parent).child().unwrap().value().unwrap(), 7);
+ assert_eq!(clone(&parent).child().unwrap().value().unwrap(), 7);
+
+ let generic_bytes = to_row(&GenericValue {
+ value: "owned".to_owned(),
+ })
+ .unwrap();
+ let generic = from_row::<GenericValue<String>>(&generic_bytes).unwrap();
+ assert_eq!(copy(generic).value().unwrap(), "owned");
+ assert_eq!(clone(&generic).value().unwrap(), "owned");
+
+ let child = parent.child().unwrap();
+ assert_eq!(child.as_bytes(), &nested_bytes[16..32]);
+ assert_eq!(child.encoded_len(), 16);
+
+ let array_bytes = to_row(&vec![NestedChild { value: 8 }]).unwrap();
+ let array = from_row::<Vec<NestedChild>>(&array_bytes).unwrap();
+ assert_eq!(copy(array).as_bytes(), array_bytes);
+ assert_eq!(array.encoded_len(), array_bytes.len());
+
+ let map_bytes = to_row(&BTreeMap::from([("k".to_owned(), 9i32)])).unwrap();
+ let map = from_row::<BTreeMap<String, i32>>(&map_bytes).unwrap();
+ assert_eq!(copy(map).as_bytes(), map_bytes);
+ assert_eq!(map.encoded_len(), map_bytes.len());
+
+ let named_bytes = to_row(&ByteMethodFields {
+ as_bytes: 10,
+ encoded_len: 20,
+ })
+ .unwrap();
+ let named = from_row::<ByteMethodFields>(&named_bytes).unwrap();
+ assert_eq!(named.as_bytes().unwrap(), 10);
+ assert_eq!(named.encoded_len().unwrap(), 20);
+ assert_eq!(RowView::as_bytes(&named), named_bytes);
+ assert_eq!(RowView::encoded_len(&named), named_bytes.len());
+}
+
+#[test]
+fn reusable_row_buffer() {
+ let mut buffer = Vec::with_capacity(256);
+ let original_pointer = buffer.as_ptr();
+ let original_capacity = buffer.capacity();
+
+ let value = MixedRow {
+ number: 7,
+ text: "reused".to_owned(),
+ short: 3,
+ };
+ let expected = to_row(&value).unwrap();
+ to_row_into(&value, &mut buffer).unwrap();
+ assert_eq!(buffer, expected);
+ assert_eq!(buffer.as_ptr(), original_pointer);
+ assert_eq!(buffer.capacity(), original_capacity);
+
+ let array = vec![1i16, 2];
+ let expected = to_row(&array).unwrap();
+ to_row_into(&array, &mut buffer).unwrap();
+ assert_eq!(buffer, expected);
+
+ let map = BTreeMap::from([("k".to_owned(), "v".to_owned())]);
+ let expected = to_row(&map).unwrap();
+ to_row_into(&map, &mut buffer).unwrap();
+ assert_eq!(buffer, expected);
+
+ let invalid = TemporalRow {
+ date: Date::from_epoch_days(i64::from(i32::MAX) + 1),
+ timestamp: Timestamp::from_epoch_micros(0),
+ duration: Duration::from_micros(0),
+ };
+ assert!(to_row_into(&invalid, &mut buffer).is_err());
+ assert!(buffer.is_empty());
+ assert_eq!(buffer.capacity(), original_capacity);
+}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]