This is an automated email from the ASF dual-hosted git repository.
chaokunyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/fory-site.git
The following commit(s) were added to refs/heads/main by this push:
new c2271e476 ๐ synced local 'docs/docs/guide/' with remote 'docs/guide/'
c2271e476 is described below
commit c2271e47692cfc39a8d324365d9fb773897fcec0
Author: chaokunyang <[email protected]>
AuthorDate: Thu Dec 11 03:28:00 2025 +0000
๐ synced local 'docs/docs/guide/' with remote 'docs/guide/'
---
docs/docs/guide/DEVELOPMENT.md | 2 +-
docs/docs/guide/graalvm_guide.md | 2 +-
docs/docs/guide/python_guide.md | 1394 ++++++++++++++++++++++++++++++++++++++
docs/docs/guide/rust_guide.md | 729 ++++++++++++++++++++
4 files changed, 2125 insertions(+), 2 deletions(-)
diff --git a/docs/docs/guide/DEVELOPMENT.md b/docs/docs/guide/DEVELOPMENT.md
index ce7b4f193..ed8633f43 100644
--- a/docs/docs/guide/DEVELOPMENT.md
+++ b/docs/docs/guide/DEVELOPMENT.md
@@ -1,6 +1,6 @@
---
title: Development
-sidebar_position: 8
+sidebar_position: 20
id: development
license: |
Licensed to the Apache Software Foundation (ASF) under one or more
diff --git a/docs/docs/guide/graalvm_guide.md b/docs/docs/guide/graalvm_guide.md
index 2a1f6468f..e74820a22 100644
--- a/docs/docs/guide/graalvm_guide.md
+++ b/docs/docs/guide/graalvm_guide.md
@@ -1,6 +1,6 @@
---
title: GraalVM Guide
-sidebar_position: 6
+sidebar_position: 19
id: graalvm_serialization
license: |
Licensed to the Apache Software Foundation (ASF) under one or more
diff --git a/docs/docs/guide/python_guide.md b/docs/docs/guide/python_guide.md
new file mode 100644
index 000000000..801ee70ff
--- /dev/null
+++ b/docs/docs/guide/python_guide.md
@@ -0,0 +1,1394 @@
+---
+title: Python Serialization
+sidebar_position: 1
+id: python_serialization
+license: |
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+---
+# Apache Foryโข Python
+
+[](https://github.com/apache/fory/actions/workflows/ci.yml)
+[](https://pypi.org/project/pyfory/)
+[](https://pypi.org/project/pyfory/)
+[](https://opensource.org/licenses/Apache-2.0)
+[](https://join.slack.com/t/fory-project/shared_invite/zt-36g0qouzm-kcQSvV_dtfbtBKHRwT5gsw)
+[](https://x.com/ApacheFory)
+
+**Apache Foryโข** is a blazing fast multi-language serialization framework
powered by **JIT compilation** and **zero-copy** techniques, providing up to
**ultra-fast performance** while maintaining ease of use and safety.
+
+`pyfory` provides the Python implementation of Apache Foryโข, offering both
high-performance object serialization and advanced row-format capabilities for
data processing tasks.
+
+## ๐ Key Features
+
+### ๐ง **Flexible Serialization Modes**
+
+- **Python native Mode**: Full Python compatibility, drop-in replacement for
pickle/cloudpickle
+- **Cross-Language Mode**: Optimized for multi-language data exchange
+- **Row Format**: Zero-copy row format for analytics workloads
+
+### ๐ฏ Versatile Serialization Features
+
+- **Shared/circular reference support** for complex object graphs in both
Python-native and cross-language modes
+- **Polymorphism support** for customized types with automatic type dispatching
+- **Schema evolution** support for backward/forward compatibility when using
dataclasses in cross-language mode
+- **Out-of-band buffer support** for zero-copy serialization of large data
structures like NumPy arrays and Pandas DataFrames, compatible with pickle
protocol 5
+
+### โก **Blazing Fast Performance**
+
+- **Extremely fast performance** compared to other serialization frameworks
+- **Runtime code generation** and **Cython-accelerated** core implementation
for optimal performance
+
+### ๐ฆ Compact Data Size
+
+- **Compact object graph protocol** with minimal space overheadโup to 3ร size
reduction compared to pickle/cloudpickle
+- **Meta packing and sharing** to minimize type forward/backward compatibility
space overhead
+
+### ๐ก๏ธ **Security & Safety**
+
+- **Strict mode** prevents deserialization of untrusted types by type
registration and checks.
+- **Reference tracking** for handling circular references safely
+
+## ๐ฆ Installation
+
+### Basic Installation
+
+Install pyfory using pip:
+
+```bash
+pip install pyfory
+```
+
+### Optional Dependencies
+
+```bash
+# Install with row format support (requires Apache Arrow)
+pip install pyfory[format]
+
+# Install from source for development
+git clone https://github.com/apache/fory.git
+cd fory/python
+pip install -e ".[dev,format]"
+```
+
+### Requirements
+
+- **Python**: 3.8 or higher
+- **OS**: Linux, macOS, Windows
+
+## ๐ Python Native Serialization
+
+`pyfory` provides a Python-native serialization mode that offers the same
functionality as pickle/cloudpickle, but with **significantly better
performance, smaller data size, and enhanced security features**.
+
+The binary protocol and API are similar to Fory's xlang mode, but
Python-native mode can serialize any Python objectโincluding global functions,
local functions, lambdas, local classes and types with customized serialization
using `__getstate__/__reduce__/__reduce_ex__`, which are not allowed in xlang
mode.
+
+To use Python-native mode, create `Fory` with `xlang=False`. This mode is
optimized for pure Python applications:
+
+```python
+import pyfory
+fory = pyfory.Fory(xlang=False, ref=False, strict=True)
+```
+
+### Basic Object Serialization
+
+Serialize and deserialize Python objects with a simple API. This example shows
serializing a dictionary with mixed types:
+
+```python
+import pyfory
+
+# Create Fory instance
+fory = pyfory.Fory(xlang=True)
+
+# Serialize any Python object
+data = fory.dumps({"name": "Alice", "age": 30, "scores": [95, 87, 92]})
+
+# Deserialize back to Python object
+obj = fory.loads(data)
+print(obj) # {'name': 'Alice', 'age': 30, 'scores': [95, 87, 92]}
+```
+
+**Note**: `dumps()`/`loads()` are aliases for `serialize()`/`deserialize()`.
Both APIs are identical, use whichever feels more intuitive.
+
+### Custom Class Serialization
+
+Fory automatically handles dataclasses and custom types. Register your class
once, then serialize instances seamlessly:
+
+```python
+import pyfory
+from dataclasses import dataclass
+from typing import List, Dict
+
+@dataclass
+class Person:
+ name: str
+ age: int
+ scores: List[int]
+ metadata: Dict[str, str]
+
+# Python mode - supports all Python types including dataclasses
+fory = pyfory.Fory(xlang=False, ref=True)
+fory.register(Person)
+person = Person("Bob", 25, [88, 92, 85], {"team": "engineering"})
+data = fory.serialize(person)
+result = fory.deserialize(data)
+print(result) # Person(name='Bob', age=25, ...)
+```
+
+### Drop-in Replacement for Pickle/Cloudpickle
+
+`pyfory` can serialize any Python object with the following configuration:
+
+- **For circular references**: Set `ref=True` to enable reference tracking
+- **For functions/classes**: Set `strict=False` to allow deserialization of
dynamic types
+
+**โ ๏ธ Security Warning**: When `strict=False`, Fory will deserialize arbitrary
types, which can pose security risks if data comes from untrusted sources. Only
use `strict=False` in controlled environments where you trust the data source
completely. If you do need to use `strict=False`, please configure a
`DeserializationPolicy` when creating fory using `policy=your_policy` to
controlling deserialization behavior.
+
+#### Common Usage
+
+Serialize common Python objects including dicts, lists, and custom classes
without any registration:
+
+```python
+import pyfory
+
+# Create Fory instance
+fory = pyfory.Fory(xlang=False, ref=True, strict=False)
+
+# serialize common Python objects
+data = fory.dumps({"name": "Alice", "age": 30, "scores": [95, 87, 92]})
+print(fory.loads(data))
+
+# serialize custom objects
+from dataclasses import dataclass
+
+@dataclass
+class Person:
+ name: str
+ age: int
+
+person = Person("Bob", 25)
+data = fory.dumps(person)
+print(fory.loads(data)) # Person(name='Bob', age=25)
+```
+
+#### Serialize Global Functions
+
+Capture and get functions defined at module level. Fory deserialize and return
same function object:
+
+```python
+import pyfory
+
+# Create Fory instance
+fory = pyfory.Fory(xlang=False, ref=True, strict=False)
+
+# serialize global functions
+def my_global_function(x):
+ return 10 * x
+
+data = fory.dumps(my_global_function)
+print(fory.loads(data)(10)) # 100
+```
+
+#### Serialize Local Functions/Lambdas
+
+Serialize functions with closures and lambda expressions. Fory captures the
closure variables automatically:
+
+```python
+import pyfory
+
+# Create Fory instance
+fory = pyfory.Fory(xlang=False, ref=True, strict=False)
+
+# serialize local functions with closures
+def my_function():
+ local_var = 10
+ def local_func(x):
+ return x * local_var
+ return local_func
+
+data = fory.dumps(my_function())
+print(fory.loads(data)(10)) # 100
+
+# serialize lambdas
+data = fory.dumps(lambda x: 10 * x)
+print(fory.loads(data)(10)) # 100
+```
+
+#### Serialize Global Classes/Methods
+
+Serialize class objects, instance methods, class methods, and static methods.
All method types are supported:
+
+```python
+from dataclasses import dataclass
+import pyfory
+fory = pyfory.Fory(xlang=False, ref=True, strict=False)
+
+# serialize global class
+@dataclass
+class Person:
+ name: str
+ age: int
+
+ def f(self, x):
+ return self.age * x
+
+ @classmethod
+ def g(cls, x):
+ return 10 * x
+
+ @staticmethod
+ def h(x):
+ return 10 * x
+
+print(fory.loads(fory.dumps(Person))("Bob", 25)) # Person(name='Bob', age=25)
+# serialize global class instance method
+print(fory.loads(fory.dumps(Person("Bob", 20).f))(10)) # 200
+# serialize global class class method
+print(fory.loads(fory.dumps(Person.g))(10)) # 100
+# serialize global class static method
+print(fory.loads(fory.dumps(Person.h))(10)) # 100
+```
+
+#### Serialize Local Classes/Methods
+
+Serialize classes defined inside functions along with their methods. Useful
for dynamic class creation:
+
+```python
+from dataclasses import dataclass
+import pyfory
+fory = pyfory.Fory(xlang=False, ref=True, strict=False)
+
+def create_local_class():
+ class LocalClass:
+ def f(self, x):
+ return 10 * x
+
+ @classmethod
+ def g(cls, x):
+ return 10 * x
+
+ @staticmethod
+ def h(x):
+ return 10 * x
+ return LocalClass
+
+# serialize local class
+data = fory.dumps(create_local_class())
+print(fory.loads(data)().f(10)) # 100
+
+# serialize local class instance method
+data = fory.dumps(create_local_class()().f)
+print(fory.loads(data)(10)) # 100
+
+# serialize local class method
+data = fory.dumps(create_local_class().g)
+print(fory.loads(data)(10)) # 100
+
+# serialize local class static method
+data = fory.dumps(create_local_class().h)
+print(fory.loads(data)(10)) # 100
+```
+
+### Out-of-Band Buffer Serialization
+
+Fory supports pickle5-compatible out-of-band buffer serialization for
efficient zero-copy handling of large data structures. This is particularly
useful for NumPy arrays, Pandas DataFrames, and other objects with large memory
footprints.
+
+Out-of-band serialization separates metadata from the actual data buffers,
allowing for:
+
+- **Zero-copy transfers** when sending data over networks or IPC using
`memoryview`
+- **Improved performance** for large datasets
+- **Pickle5 compatibility** using `pickle.PickleBuffer`
+- **Flexible stream support** - write to any writable object (files, BytesIO,
sockets, etc.)
+
+#### Basic Out-of-Band Serialization
+
+```python
+import pyfory
+import numpy as np
+
+fory = pyfory.Fory(xlang=False, ref=False, strict=False)
+
+# Large numpy array
+array = np.arange(10000, dtype=np.float64)
+
+# Serialize with out-of-band buffers
+buffer_objects = []
+serialized_data = fory.serialize(array, buffer_callback=buffer_objects.append)
+
+# Convert buffer objects to memoryview for zero-copy transmission
+# For contiguous buffers (bytes, numpy arrays), this is zero-copy
+# For non-contiguous data, a copy may be created to ensure contiguity
+buffers = [obj.getbuffer() for obj in buffer_objects]
+
+# Deserialize with out-of-band buffers (accepts memoryview, bytes, or Buffer)
+deserialized_array = fory.deserialize(serialized_data, buffers=buffers)
+
+assert np.array_equal(array, deserialized_array)
+```
+
+#### Out-of-Band with Pandas DataFrames
+
+```python
+import pyfory
+import pandas as pd
+import numpy as np
+
+fory = pyfory.Fory(xlang=False, ref=False, strict=False)
+
+# Create a DataFrame with numeric columns
+df = pd.DataFrame({
+ 'a': np.arange(1000, dtype=np.float64),
+ 'b': np.arange(1000, dtype=np.int64),
+ 'c': ['text'] * 1000
+})
+
+# Serialize with out-of-band buffers
+buffer_objects = []
+serialized_data = fory.serialize(df, buffer_callback=buffer_objects.append)
+buffers = [obj.getbuffer() for obj in buffer_objects]
+
+# Deserialize
+deserialized_df = fory.deserialize(serialized_data, buffers=buffers)
+
+assert df.equals(deserialized_df)
+```
+
+#### Selective Out-of-Band Serialization
+
+You can control which buffers go out-of-band by providing a callback that
returns `True` to keep data in-band or `False` (and appending to a list) to
send it out-of-band:
+
+```python
+import pyfory
+import numpy as np
+
+fory = pyfory.Fory(xlang=False, ref=True, strict=False)
+
+arr1 = np.arange(1000, dtype=np.float64)
+arr2 = np.arange(2000, dtype=np.float64)
+data = [arr1, arr2]
+
+buffer_objects = []
+counter = 0
+
+def selective_callback(buffer_object):
+ global counter
+ counter += 1
+ # Only send even-numbered buffers out-of-band
+ if counter % 2 == 0:
+ buffer_objects.append(buffer_object)
+ return False # Out-of-band
+ return True # In-band
+
+serialized = fory.serialize(data, buffer_callback=selective_callback)
+buffers = [obj.getbuffer() for obj in buffer_objects]
+deserialized = fory.deserialize(serialized, buffers=buffers)
+```
+
+#### Pickle5 Compatibility
+
+Fory's out-of-band serialization is fully compatible with pickle protocol 5.
When objects implement `__reduce_ex__(protocol)`, Fory automatically uses
protocol 5 to enable `pickle.PickleBuffer` support:
+
+```python
+import pyfory
+import pickle
+
+fory = pyfory.Fory(xlang=False, ref=False, strict=False)
+
+# PickleBuffer objects are automatically supported
+data = b"Large binary data"
+pickle_buffer = pickle.PickleBuffer(data)
+
+# Serialize with buffer callback for out-of-band handling
+buffer_objects = []
+serialized = fory.serialize(pickle_buffer,
buffer_callback=buffer_objects.append)
+buffers = [obj.getbuffer() for obj in buffer_objects]
+
+# Deserialize with buffers
+deserialized = fory.deserialize(serialized, buffers=buffers)
+assert bytes(deserialized.raw()) == data
+```
+
+#### Writing Buffers to Different Streams
+
+The `BufferObject.write_to()` method accepts any writable stream object,
making it flexible for various use cases:
+
+```python
+import pyfory
+import numpy as np
+import io
+
+fory = pyfory.Fory(xlang=False, ref=False, strict=False)
+
+array = np.arange(1000, dtype=np.float64)
+
+# Collect out-of-band buffers
+buffer_objects = []
+serialized = fory.serialize(array, buffer_callback=buffer_objects.append)
+
+# Write to different stream types
+for buffer_obj in buffer_objects:
+ # Write to BytesIO (in-memory stream)
+ bytes_stream = io.BytesIO()
+ buffer_obj.write_to(bytes_stream)
+
+ # Write to file
+ with open('/tmp/buffer_data.bin', 'wb') as f:
+ buffer_obj.write_to(f)
+
+ # Get zero-copy memoryview (for contiguous buffers)
+ mv = buffer_obj.getbuffer()
+ assert isinstance(mv, memoryview)
+```
+
+**Note**: For contiguous memory buffers (like bytes, numpy arrays),
`getbuffer()` returns a zero-copy `memoryview`. For non-contiguous data, a copy
may be created to ensure contiguity.
+
+## ๐โโ๏ธ Cross-Language Object Graph Serialization
+
+`pyfory` supports cross-language object graph serialization, allowing you to
serialize data in Python and deserialize it in Java, Go, Rust, or other
supported languages.
+
+The binary protocol and API are similar to `pyfory`'s python-native mode, but
Python-native mode can serialize any Python objectโincluding global functions,
local functions, lambdas, local classes, and types with customized
serialization using `__getstate__/__reduce__/__reduce_ex__`, which are not
allowed in xlang mode.
+
+To use xlang mode, create `Fory` with `xlang=True`. This mode is for xlang
serialization applications:
+
+```python
+import pyfory
+fory = pyfory.Fory(xlang=True, ref=False, strict=True)
+```
+
+### Cross-Language Sserialization
+
+Serialize data in Python and deserialize it in Java, Go, Rust, or other
supported languages. Both sides must register the same type with matching names:
+
+**Python (Serializer)**
+
+```python
+import pyfory
+
+# Cross-language mode for interoperability
+f = pyfory.Fory(xlang=True, ref=True)
+
+# Register type for cross-language compatibility
+@dataclass
+class Person:
+ name: str
+ age: pyfory.int32
+
+f.register(Person, typename="example.Person")
+
+person = Person("Charlie", 35)
+binary_data = f.serialize(person)
+# binary_data can now be sent to Java, Go, etc.
+```
+
+**Java (Deserializer)**
+
+```java
+import org.apache.fory.*;
+
+public class Person {
+ public String name;
+ public int age;
+}
+
+Fory fory = Fory.builder()
+ .withLanguage(Language.XLANG)
+ .withRefTracking(true)
+ .build();
+
+fory.register(Person.class, "example.Person");
+Person person = (Person) fory.deserialize(binaryData);
+```
+
+## ๐ Row Format - Zero-Copy Processing
+
+Apache Furyโข provides a random-access row format that enables reading nested
fields from binary data without full deserialization. This drastically reduces
overhead when working with large objects where only partial data access is
needed. The format also supports memory-mapped files for ultra-low memory
footprint.
+
+### Basic Row Format Usage
+
+Encode objects to row format for random access without full deserialization.
Ideal for large datasets:
+
+**Python**
+
+```python
+import pyfory
+import pyarrow as pa
+from dataclasses import dataclass
+from typing import List, Dict
+
+@dataclass
+class Bar:
+ f1: str
+ f2: List[pa.int64]
+
+@dataclass
+class Foo:
+ f1: pa.int32
+ f2: List[pa.int32]
+ f3: Dict[str, pa.int32]
+ f4: List[Bar]
+
+# Create encoder for row format
+encoder = pyfory.encoder(Foo)
+
+# Create large dataset
+foo = Foo(
+ f1=10,
+ f2=list(range(1_000_000)),
+ f3={f"k{i}": i for i in range(1_000_000)},
+ f4=[Bar(f1=f"s{i}", f2=list(range(10))) for i in range(1_000_000)]
+)
+
+# Encode to row format
+binary: bytes = encoder.to_row(foo).to_bytes()
+
+# Zero-copy access - no full deserialization needed!
+foo_row = pyfory.RowData(encoder.schema, binary)
+print(foo_row.f2[100000]) # Access 100,000th element directly
+print(foo_row.f4[100000].f1) # Access nested field directly
+print(foo_row.f4[200000].f2[5]) # Access deeply nested field directly
+```
+
+### Cross-Language Compatibility
+
+Row format works across languages. Here's the same data structure accessed in
Java:
+
+**Java**
+
+```java
+public class Bar {
+ String f1;
+ List<Long> f2;
+}
+
+public class Foo {
+ int f1;
+ List<Integer> f2;
+ Map<String, Integer> f3;
+ List<Bar> f4;
+}
+
+RowEncoder<Foo> encoder = Encoders.bean(Foo.class);
+
+// Create large dataset
+Foo foo = new Foo();
+foo.f1 = 10;
+foo.f2 = IntStream.range(0, 1_000_000).boxed().collect(Collectors.toList());
+foo.f3 = IntStream.range(0, 1_000_000).boxed().collect(Collectors.toMap(i ->
"k" + i, i -> i));
+List<Bar> bars = new ArrayList<>(1_000_000);
+for (int i = 0; i < 1_000_000; i++) {
+ Bar bar = new Bar();
+ bar.f1 = "s" + i;
+ bar.f2 = LongStream.range(0, 10).boxed().collect(Collectors.toList());
+ bars.add(bar);
+}
+foo.f4 = bars;
+
+// Encode to row format (cross-language compatible with Python)
+BinaryRow binaryRow = encoder.toRow(foo);
+
+// Zero-copy random access without full deserialization
+BinaryArray f2Array = binaryRow.getArray(1); // Access f2 list
+BinaryArray f4Array = binaryRow.getArray(3); // Access f4 list
+BinaryRow bar10 = f4Array.getStruct(10); // Access 11th Bar
+long value = bar10.getArray(1).getInt64(5); // Access 6th
element of bar.f2
+
+// Partial deserialization - only deserialize what you need
+RowEncoder<Bar> barEncoder = Encoders.bean(Bar.class);
+Bar bar1 = barEncoder.fromRow(f4Array.getStruct(10)); // Deserialize 11th
Bar only
+Bar bar2 = barEncoder.fromRow(f4Array.getStruct(20)); // Deserialize 21st
Bar only
+
+// Full deserialization when needed
+Foo newFoo = encoder.fromRow(binaryRow);
+```
+
+**C++**
+
+And in C++ with compile-time type information:
+
+```cpp
+#include "fory/encoder/row_encoder.h"
+#include "fory/row/writer.h"
+
+struct Bar {
+ std::string f1;
+ std::vector<int64_t> f2;
+};
+
+FORY_FIELD_INFO(Bar, f1, f2);
+
+struct Foo {
+ int32_t f1;
+ std::vector<int32_t> f2;
+ std::map<std::string, int32_t> f3;
+ std::vector<Bar> f4;
+};
+
+FORY_FIELD_INFO(Foo, f1, f2, f3, f4);
+
+// Create large dataset
+Foo foo;
+foo.f1 = 10;
+for (int i = 0; i < 1000000; i++) {
+ foo.f2.push_back(i);
+ foo.f3["k" + std::to_string(i)] = i;
+}
+for (int i = 0; i < 1000000; i++) {
+ Bar bar;
+ bar.f1 = "s" + std::to_string(i);
+ for (int j = 0; j < 10; j++) {
+ bar.f2.push_back(j);
+ }
+ foo.f4.push_back(bar);
+}
+
+// Encode to row format (cross-language compatible with Python/Java)
+fory::encoder::RowEncoder<Foo> encoder;
+encoder.Encode(foo);
+auto row = encoder.GetWriter().ToRow();
+
+// Zero-copy random access without full deserialization
+auto f2_array = row->GetArray(1); // Access f2 list
+auto f4_array = row->GetArray(3); // Access f4 list
+auto bar10 = f4_array->GetStruct(10); // Access 11th Bar
+int64_t value = bar10->GetArray(1)->GetInt64(5); // Access 6th element of
bar.f2
+std::string str = bar10->GetString(0); // Access bar.f1
+```
+
+### Key Benefits
+
+- **Zero-Copy Access**: Read nested fields without deserializing the entire
object
+- **Memory Efficiency**: Memory-map large datasets directly from disk
+- **Cross-Language**: Binary format is compatible between Python, Java, and
other Fury implementations
+- **Partial Deserialization**: Deserialize only the specific elements you need
+- **High Performance**: Skip unnecessary data parsing for analytics and big
data workloads
+
+## ๐๏ธ Core API Reference
+
+### Fory Class
+
+The main serialization interface:
+
+```python
+class Fory:
+ def __init__(
+ self,
+ xlang: bool = False,
+ ref: bool = False,
+ strict: bool = True,
+ compatible: bool = False,
+ max_depth: int = 50
+ )
+```
+
+### ThreadSafeFory Class
+
+Thread-safe serialization interface using thread-local storage:
+
+```python
+class ThreadSafeFory:
+ def __init__(
+ self,
+ xlang: bool = False,
+ ref: bool = False,
+ strict: bool = True,
+ compatible: bool = False,
+ max_depth: int = 50
+ )
+```
+
+`ThreadSafeFory` provides thread-safe serialization by maintaining a pool of
`Fory` instances protected by a lock. When a thread needs to
serialize/deserialize, it gets an instance from the pool, uses it, and returns
it. All type registrations must be done before any serialization to ensure
consistency across all instances.
+
+**Thread Safety Example:**
+
+```python
+import pyfory
+import threading
+from dataclasses import dataclass
+
+@dataclass
+class Person:
+ name: str
+ age: int
+
+# Create thread-safe Fory instance
+fory = pyfory.ThreadSafeFory(xlang=False, ref=True)
+fory.register(Person)
+
+# Use in multiple threads safely
+def serialize_in_thread(thread_id):
+ person = Person(name=f"User{thread_id}", age=25 + thread_id)
+ data = fory.serialize(person)
+ result = fory.deserialize(data)
+ print(f"Thread {thread_id}: {result}")
+
+threads = [threading.Thread(target=serialize_in_thread, args=(i,)) for i in
range(10)]
+for t in threads: t.start()
+for t in threads: t.join()
+```
+
+**Key Features:**
+
+- **Instance Pool**: Maintains a pool of `Fory` instances protected by a lock
for thread safety
+- **Shared Configuration**: All registrations must be done upfront and are
applied to all instances
+- **Same API**: Drop-in replacement for `Fory` class with identical methods
+- **Registration Safety**: Prevents registration after first use to ensure
consistency
+
+**When to Use:**
+
+- **Multi-threaded Applications**: Web servers, concurrent workers, parallel
processing
+- **Shared Fory Instances**: When multiple threads need to
serialize/deserialize data
+- **Thread Pools**: Applications using thread pools or concurrent.futures
+
+**Parameters:**
+
+- **`xlang`** (`bool`, default=`False`): Enable cross-language serialization.
When `False`, enables Python-native mode supporting all Python objects. When
`True`, enables cross-language mode compatible with Java, Go, Rust, etc.
+- **`ref`** (`bool`, default=`False`): Enable reference tracking for
shared/circular references. Disable for better performance if your data has no
shared references.
+- **`strict`** (`bool`, default=`True`): Require type registration for
security. **Highly recommended** for production. Only disable in trusted
environments.
+- **`compatible`** (`bool`, default=`False`): Enable schema evolution in
cross-language mode, allowing fields to be added/removed while maintaining
compatibility.
+- **`max_depth`** (`int`, default=`50`): Maximum deserialization depth for
security, preventing stack overflow attacks.
+
+**Key Methods:**
+
+```python
+# Serialization (serialize/deserialize are identical to dumps/loads)
+data: bytes = fory.serialize(obj)
+obj = fory.deserialize(data)
+
+# Alternative API (aliases)
+data: bytes = fory.dumps(obj)
+obj = fory.loads(data)
+
+# Type registration by id (for Python mode)
+fory.register(MyClass, type_id=123)
+fory.register(MyClass, type_id=123, serializer=custom_serializer)
+
+# Type registration by name (for cross-language mode)
+fory.register(MyClass, typename="my.package.MyClass")
+fory.register(MyClass, typename="my.package.MyClass",
serializer=custom_serializer)
+```
+
+### Language Modes Comparison
+
+| Feature | Python Mode (`xlang=False`) |
Cross-Language Mode (`xlang=True`) |
+| --------------------- | ------------------------------------ |
------------------------------------- |
+| **Use Case** | Pure Python applications |
Multi-language systems |
+| **Compatibility** | Python only | Java, Go,
Rust, C++, JavaScript, etc. |
+| **Supported Types** | All Python types |
Cross-language compatible types only |
+| **Functions/Lambdas** | โ Supported | โ Not allowed
|
+| **Local Classes** | โ Supported | โ Not allowed
|
+| **Dynamic Classes** | โ Supported | โ Not allowed
|
+| **Schema Evolution** | โ Supported (with `compatible=True`) | โ Supported
(with `compatible=True`) |
+| **Performance** | Extremely fast | Very fast
|
+| **Data Size** | Compact | Compact with
type metadata |
+
+#### Python Mode (`xlang=False`)
+
+Python mode supports all Python types including functions, classes, and
closures. Perfect for pure Python applications:
+
+```python
+import pyfory
+
+# Full Python compatibility mode
+fory = pyfory.Fory(xlang=False, ref=True, strict=False)
+
+# Supports ALL Python objects:
+data = fory.dumps({
+ 'function': lambda x: x * 2, # Functions and lambdas
+ 'class': type('Dynamic', (), {}), # Dynamic classes
+ 'method': str.upper, # Methods
+ 'nested': {'circular_ref': None} # Circular references (when ref=True)
+})
+
+# Drop-in replacement for pickle/cloudpickle
+import pickle
+obj = [1, 2, {"nested": [3, 4]}]
+assert fory.loads(fory.dumps(obj)) == pickle.loads(pickle.dumps(obj))
+
+# Significantly faster and more compact than pickle
+import timeit
+obj = {f"key{i}": f"value{i}" for i in range(10000)}
+print(f"Fory: {timeit.timeit(lambda: fory.dumps(obj), number=1000):.3f}s")
+print(f"Pickle: {timeit.timeit(lambda: pickle.dumps(obj), number=1000):.3f}s")
+```
+
+#### Cross-Language Mode (`xlang=True`)
+
+Cross-language mode restricts types to those compatible across all Fory
implementations. Use for multi-language systems:
+
+```python
+import pyfory
+
+# Cross-language compatibility mode
+f = pyfory.Fory(xlang=True, ref=True)
+
+# Only supports cross-language compatible types
+f.register(MyDataClass, typename="com.example.MyDataClass")
+
+# Data can be read by Java, Go, Rust, etc.
+data = f.serialize(MyDataClass(field1="value", field2=42))
+```
+
+## ๐ง Advanced Features
+
+### Reference Tracking & Circular References
+
+Handle shared references and circular dependencies safely. Set `ref=True` to
deduplicate objects:
+
+```python
+import pyfory
+
+f = pyfory.Fory(ref=True) # Enable reference tracking
+
+# Handle circular references safely
+class Node:
+ def __init__(self, value):
+ self.value = value
+ self.children = []
+ self.parent = None
+
+root = Node("root")
+child = Node("child")
+child.parent = root # Circular reference
+root.children.append(child)
+
+# Serializes without infinite recursion
+data = f.serialize(root)
+result = f.deserialize(data)
+assert result.children[0].parent is result # Reference preserved
+```
+
+### Type Registration & Security
+
+In strict mode, only registered types can be deserialized. This prevents
arbitrary code execution:
+
+```python
+import pyfory
+
+# Strict mode (recommended for production)
+f = pyfory.Fory(strict=True)
+
+class SafeClass:
+ def __init__(self, data):
+ self.data = data
+
+# Must register types in strict mode
+f.register(SafeClass, typename="com.example.SafeClass")
+
+# Now serialization works
+obj = SafeClass("safe data")
+data = f.serialize(obj)
+result = f.deserialize(data)
+
+# Unregistered types will raise an exception
+class UnsafeClass:
+ pass
+
+# This will fail in strict mode
+try:
+ f.serialize(UnsafeClass())
+except Exception as e:
+ print("Security protection activated!")
+```
+
+### Custom Serializers
+
+Implement custom serialization logic for specialized types. Override
`write/read` for Python mode, `xwrite/xread` for cross-language:
+
+```python
+import pyfory
+from pyfory.serializer import Serializer
+from dataclasses import dataclass
+
+@dataclass
+class Foo:
+ f1: int
+ f2: str
+
+class FooSerializer(Serializer):
+ def __init__(self, fory, cls):
+ super().__init__(fory, cls)
+
+ def write(self, buffer, obj: Foo):
+ # Custom serialization logic
+ buffer.write_varint32(obj.f1)
+ buffer.write_string(obj.f2)
+
+ def read(self, buffer):
+ # Custom deserialization logic
+ f1 = buffer.read_varint32()
+ f2 = buffer.read_string()
+ return Foo(f1, f2)
+
+ # For cross-language mode
+ def xwrite(self, buffer, obj: Foo):
+ buffer.write_int32(obj.f1)
+ buffer.write_string(obj.f2)
+
+ def xread(self, buffer):
+ return Foo(buffer.read_int32(), buffer.read_string())
+
+f = pyfory.Fory()
+f.register(Foo, type_id=100, serializer=FooSerializer(f, Foo))
+
+# Now Foo uses your custom serializer
+data = f.dumps(Foo(42, "hello"))
+result = f.loads(data)
+print(result) # Foo(f1=42, f2='hello')
+```
+
+### Numpy & Scientific Computing
+
+Fory natively supports numpy arrays with optimized serialization. Large arrays
use zero-copy when possible:
+
+```python
+import pyfory
+import numpy as np
+
+f = pyfory.Fory()
+
+# Numpy arrays are supported natively
+arrays = {
+ 'matrix': np.random.rand(1000, 1000),
+ 'vector': np.arange(10000),
+ 'bool_mask': np.random.choice([True, False], size=5000)
+}
+
+data = f.serialize(arrays)
+result = f.deserialize(data)
+
+# Zero-copy for compatible array types
+assert np.array_equal(arrays['matrix'], result['matrix'])
+```
+
+## ๐ก Best Practices
+
+### Production Configuration
+
+Use these recommended settings to balance security, performance, and
functionality in production:
+
+```python
+import pyfory
+
+# Recommended settings for production
+fory = pyfory.Fory(
+ xlang=False, # Use True if you need cross-language support
+ ref=False, # Enable if you have shared/circular references
+ strict=True, # CRITICAL: Always True in production
+ compatible=False, # Enable only if you need schema evolution
+ max_depth=20 # Adjust based on your data structure depth
+)
+
+# Register all types upfront
+fory.register(UserModel, type_id=100)
+fory.register(OrderModel, type_id=101)
+fory.register(ProductModel, type_id=102)
+```
+
+### Performance Tips
+
+Optimize serialization speed and memory usage with these guidelines:
+
+1. **Disable `ref=True` if not needed**: Reference tracking has overhead
+2. **Use type_id instead of typename**: Integer IDs are faster than string
names
+3. **Reuse Fory instances**: Create once, use many times
+4. **Enable Cython**: Make sure `ENABLE_FORY_CYTHON_SERIALIZATION=1`, should
be enabled by default
+5. **Use row format for large arrays**: Zero-copy access for analytics
+
+```python
+# Good: Reuse instance
+fory = pyfory.Fory()
+for obj in objects:
+ data = fory.dumps(obj)
+
+# Bad: Create new instance each time
+for obj in objects:
+ fory = pyfory.Fory() # Wasteful!
+ data = fory.dumps(obj)
+```
+
+### Type Registration Patterns
+
+Choose the right registration approach for your use case:
+
+```python
+# Pattern 1: Simple registration
+fory.register(MyClass, type_id=100)
+
+# Pattern 2: Cross-language with typename
+fory.register(MyClass, typename="com.example.MyClass")
+
+# Pattern 3: With custom serializer
+fory.register(MyClass, type_id=100, serializer=MySerializer(fory, MyClass))
+
+# Pattern 4: Batch registration
+type_id = 100
+for model_class in [User, Order, Product, Invoice]:
+ fory.register(model_class, type_id=type_id)
+ type_id += 1
+```
+
+### Error Handling
+
+Handle common serialization errors gracefully. Catch specific exceptions for
better error recovery:
+
+```python
+import pyfory
+from pyfory.error import TypeUnregisteredError, TypeNotCompatibleError
+
+fory = pyfory.Fory(strict=True)
+
+try:
+ data = fory.dumps(my_object)
+except TypeUnregisteredError as e:
+ print(f"Type not registered: {e}")
+ # Register the type and retry
+ fory.register(type(my_object), type_id=100)
+ data = fory.dumps(my_object)
+except Exception as e:
+ print(f"Serialization failed: {e}")
+
+try:
+ obj = fory.loads(data)
+except TypeNotCompatibleError as e:
+ print(f"Schema mismatch: {e}")
+ # Handle version mismatch
+except Exception as e:
+ print(f"Deserialization failed: {e}")
+```
+
+## ๐ ๏ธ Migration Guide
+
+### From Pickle
+
+Replace pickle with Fory for better performance while keeping the same API:
+
+```python
+# Before (pickle)
+import pickle
+data = pickle.dumps(obj)
+result = pickle.loads(data)
+
+# After (Fory - drop-in replacement with better performance)
+import pyfory
+f = pyfory.Fory(xlang=False, ref=True, strict=False)
+data = f.dumps(obj) # Faster and more compact
+result = f.loads(data) # Faster deserialization
+
+# Benefits:
+# - 2-10x faster serialization
+# - 2-5x faster deserialization
+# - Up to 3x smaller data size
+# - Same API, better performance
+```
+
+### From JSON
+
+Unlike JSON, Fory supports arbitrary Python types including functions:
+
+```python
+# Before (JSON - limited types)
+import json
+data = json.dumps({"name": "Alice", "age": 30})
+result = json.loads(data)
+
+# After (Fory - all Python types)
+import pyfory
+f = pyfory.Fory()
+data = f.dumps({"name": "Alice", "age": 30, "func": lambda x: x})
+result = f.loads(data)
+```
+
+## ๐จ Security Best Practices
+
+### Production Configuration
+
+Never disable `strict=True` in production unless your environment is
completely trusted:
+
+```python
+import pyfory
+
+# Recommended production settings
+f = pyfory.Fory(
+ xlang=False, # or True for cross-language
+ ref=True, # Handle circular references
+ strict=True, # IMPORTANT: Prevent malicious data
+ max_depth=100 # Prevent deep recursion attacks
+)
+
+# Explicitly register allowed types
+f.register(UserModel, type_id=100)
+f.register(OrderModel, type_id=101)
+# Never set strict=False in production with untrusted data!
+```
+
+### Development vs Production
+
+Use environment variables to switch between development and production
configurations:
+
+```python
+import pyfory
+import os
+
+# Development configuration
+if os.getenv('ENV') == 'development':
+ fory = pyfory.Fory(
+ xlang=False,
+ ref=True,
+ strict=False, # Allow any type for development
+ max_depth=1000 # Higher limit for development
+ )
+else:
+ # Production configuration (security hardened)
+ fory = pyfory.Fory(
+ xlang=False,
+ ref=True,
+ strict=True, # CRITICAL: Require registration
+ max_depth=100 # Reasonable limit
+ )
+ # Register only known safe types
+ for idx, model_class in enumerate([UserModel, ProductModel, OrderModel]):
+ fory.register(model_class, type_id=100 + idx)
+```
+
+### DeserializationPolicy
+
+When `strict=False` is necessary (e.g., deserializing functions/lambdas), use
`DeserializationPolicy` to implement fine-grained security controls during
deserialization. This provides protection similar to
`pickle.Unpickler.find_class()` but with more comprehensive hooks.
+
+**Why use DeserializationPolicy?**
+
+- Block dangerous classes/modules (e.g., `subprocess.Popen`)
+- Intercept and validate `__reduce__` callables before invocation
+- Sanitize sensitive data during `__setstate__`
+- Replace or reject deserialized objects based on custom rules
+
+**Example: Blocking Dangerous Classes**
+
+```python
+import pyfory
+from pyfory import DeserializationPolicy
+
+dangerous_modules = {'subprocess', 'os', '__builtin__'}
+
+class SafeDeserializationPolicy(DeserializationPolicy):
+ """Block potentially dangerous classes during deserialization."""
+
+ def validate_class(self, cls, is_local, **kwargs):
+ # Block dangerous modules
+ if cls.__module__ in dangerous_modules:
+ raise ValueError(f"Blocked dangerous class:
{cls.__module__}.{cls.__name__}")
+ return None
+
+ def intercept_reduce_call(self, callable_obj, args, **kwargs):
+ # Block specific callable invocations during __reduce__
+ if getattr(callable_obj, '__name__', "") == 'Popen':
+ raise ValueError("Blocked attempt to invoke subprocess.Popen")
+ return None
+
+ def intercept_setstate(self, obj, state, **kwargs):
+ # Sanitize sensitive data
+ if isinstance(state, dict) and 'password' in state:
+ state['password'] = '***REDACTED***'
+ return None
+
+# Create Fory with custom security policy
+policy = SafeDeserializationPolicy()
+fory = pyfory.Fory(xlang=False, ref=True, strict=False, policy=policy)
+
+# Now deserialization is protected by your custom policy
+data = fory.serialize(my_object)
+result = fory.deserialize(data) # Policy hooks will be invoked
+```
+
+**Available Policy Hooks:**
+
+- `validate_class(cls, is_local)` - Validate/block class types during
deserialization
+- `validate_module(module, is_local)` - Validate/block module imports
+- `validate_function(func, is_local)` - Validate/block function references
+- `intercept_reduce_call(callable_obj, args)` - Intercept `__reduce__`
invocations
+- `inspect_reduced_object(obj)` - Inspect/replace objects created via
`__reduce__`
+- `intercept_setstate(obj, state)` - Sanitize state before `__setstate__`
+- `authorize_instantiation(cls, args, kwargs)` - Control class instantiation
+
+**See also:** `pyfory/policy.py` contains detailed documentation and examples
for each hook.
+
+## ๐ Troubleshooting
+
+### Common Issues
+
+**Q: ImportError with format features**
+
+```python
+# A: Install Row format support
+pip install pyfory[format]
+
+# Or install from source with format support
+pip install -e ".[format]"
+```
+
+**Q: Slow serialization performance**
+
+```python
+# A: Check if Cython acceleration is enabled
+import pyfory
+print(pyfory.ENABLE_FORY_CYTHON_SERIALIZATION) # Should be True
+
+# If False, Cython extension may not be compiled correctly
+# Reinstall with: pip install --force-reinstall --no-cache-dir pyfory
+
+# For debugging, you can disable Cython mode before importing
+import os
+os.environ['ENABLE_FORY_CYTHON_SERIALIZATION'] = '0'
+import pyfory # Now uses pure Python mode
+```
+
+**Q: Cross-language compatibility issues**
+
+```python
+# A: Use explicit type registration with consistent naming
+f = pyfory.Fory(xlang=True)
+f.register(MyClass, typename="com.package.MyClass") # Use same name in all
languages
+```
+
+**Q: Circular reference errors or duplicate data**
+
+```python
+# A: Enable reference tracking
+f = pyfory.Fory(ref=True) # Required for circular references
+
+# Example with circular reference
+class Node:
+ def __init__(self, value):
+ self.value = value
+ self.next = None
+
+node1 = Node(1)
+node2 = Node(2)
+node1.next = node2
+node2.next = node1 # Circular reference
+
+data = f.dumps(node1)
+result = f.loads(data)
+assert result.next.next is result # Circular reference preserved
+```
+
+### Debug Mode
+
+```python
+# Set environment variable BEFORE importing pyfory to disable Cython for
debugging
+import os
+os.environ['ENABLE_FORY_CYTHON_SERIALIZATION'] = '0'
+import pyfory # Now uses pure Python implementation
+
+# This is useful for:
+# 1. Debugging protocol issues
+# 2. Understanding serialization behavior
+# 3. Development without recompiling Cython
+```
+
+**Q: Schema evolution not working**
+
+```python
+# A: Enable compatible mode for schema evolution
+f = pyfory.Fory(xlang=True, compatible=True)
+
+# Version 1: Original class
+@dataclass
+class User:
+ name: str
+ age: int
+
+f.register(User, typename="User")
+data = f.dumps(User("Alice", 30))
+
+# Version 2: Add new field (backward compatible)
+@dataclass
+class User:
+ name: str
+ age: int
+ email: str = "[email protected]" # New field with default
+
+# Can still deserialize old data
+user = f.loads(data)
+print(user.email) # "[email protected]"
+```
+
+**Q: Type registration errors in strict mode**
+
+```python
+# A: Register all custom types before serialization
+f = pyfory.Fory(strict=True)
+
+# Must register before use
+f.register(MyClass, type_id=100)
+f.register(AnotherClass, type_id=101)
+
+# Or disable strict mode (NOT recommended for production)
+f = pyfory.Fory(strict=False) # Use only in trusted environments
+```
+
+## ๐ค Contributing
+
+Apache Foryโข is an open-source project under the Apache Software Foundation.
We welcome all forms of contributions:
+
+### How to Contribute
+
+1. **Report Issues**: Found a bug? [Open an
issue](https://github.com/apache/fory/issues)
+2. **Suggest Features**: Have an idea? Start a discussion
+3. **Improve Docs**: Documentation improvements are always welcome
+4. **Submit Code**: See our [Contributing
Guide](https://github.com/apache/fory/blob/main/CONTRIBUTING.md)
+
+### Development Setup
+
+```bash
+git clone https://github.com/apache/fory.git
+cd fory/python
+
+# Install dependencies
+pip install -e ".[dev,format]"
+
+# Run tests
+pytest -v -s .
+
+# Run specific test
+pytest -v -s pyfory/tests/test_serializer.py
+
+# Format code
+ruff format .
+ruff check --fix .
+```
+
+## ๐ License
+
+Apache License 2.0. See
[LICENSE](https://github.com/apache/fory/blob/main/LICENSE) for details.
+
+---
+
+**Apache Foryโข** - Blazing fast, secure, and versatile serialization for
modern applications.
+
+## ๐ Links
+
+- **Documentation**: https://fory.apache.org/docs/latest/python_guide/
+- **GitHub**: https://github.com/apache/fory
+- **PyPI**: https://pypi.org/project/pyfory/
+- **Issues**: https://github.com/apache/fory/issues
+- **Slack**:
https://join.slack.com/t/fory-project/shared_invite/zt-36g0qouzm-kcQSvV_dtfbtBKHRwT5gsw
+- **Benchmarks**: https://fory.apache.org/docs/latest/benchmarks/
+
+## ๐ Community
+
+We welcome contributions! Whether it's bug reports, feature requests,
documentation improvements, or code contributions, we appreciate your help.
+
+- Star the project on [GitHub](https://github.com/apache/fory) โญ
+- Join our [Slack
community](https://join.slack.com/t/fory-project/shared_invite/zt-36g0qouzm-kcQSvV_dtfbtBKHRwT5gsw)
๐ฌ
+- Follow us on [X/Twitter](https://x.com/ApacheFory) ๐ฆ
diff --git a/docs/docs/guide/rust_guide.md b/docs/docs/guide/rust_guide.md
new file mode 100644
index 000000000..2b7f97dd8
--- /dev/null
+++ b/docs/docs/guide/rust_guide.md
@@ -0,0 +1,729 @@
+---
+title: Rust Serialization
+sidebar_position: 2
+id: rust_serialization
+license: |
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+---
+# Apache Foryโข Rust
+
+[](https://crates.io/crates/fory)
+[](https://docs.rs/fory)
+[](https://github.com/apache/fory/blob/main/LICENSE)
+
+**Apache Foryโข** is a blazing fast multi-language serialization framework
powered by **JIT compilation** and **zero-copy** techniques, providing up to
**ultra-fast performance** while maintaining ease of use and safety.
+
+The Rust implementation provides versatile and high-performance serialization
with automatic memory management and compile-time type safety.
+
+## ๐ Why Apache Foryโข Rust?
+
+- **๐ฅ Blazingly Fast**: Zero-copy deserialization and optimized binary
protocols
+- **๐ Cross-Language**: Seamlessly serialize/deserialize data across Java,
Python, C++, Go, JavaScript, and Rust
+- **๐ฏ Type-Safe**: Compile-time type checking with derive macros
+- **๐ Circular References**: Automatic tracking of shared and circular
references with `Rc`/`Arc` and weak pointers
+- **๐งฌ Polymorphic**: Serialize trait objects with `Box<dyn Trait>`, `Rc<dyn
Trait>`, and `Arc<dyn Trait>`
+- **๐ฆ Schema Evolution**: Compatible mode for independent schema changes
+- **โก Two Formats**: Object graph serialization and zero-copy row-based format
+
+## ๐ฆ Crates
+
+| Crate
| Description | Version
|
+| ---------------------------------------------------------------------------
| --------------------------------- |
-----------------------------------------------------------------------------------------------------
|
+| [`fory`](https://github.com/apache/fory/blob/main/rust/fory)
| High-level API with derive macros |
[](https://crates.io/crates/fory)
|
+| [`fory-core`](https://github.com/apache/fory/blob/main/rust/fory-core/)
| Core serialization engine |
[](https://crates.io/crates/fory-core)
|
+| [`fory-derive`](https://github.com/apache/fory/blob/main/rust/fory-derive/)
| Procedural macros |
[](https://crates.io/crates/fory-derive)
|
+
+## ๐ Quick Start
+
+Add Apache Foryโข to your `Cargo.toml`:
+
+```toml
+[dependencies]
+fory = "0.13"
+```
+
+### Basic Example
+
+```rust
+use fory::{Fory, Error, Reader};
+use fory::ForyObject;
+
+#[derive(ForyObject, Debug, PartialEq)]
+struct User {
+ name: String,
+ age: i32,
+ email: String,
+}
+
+fn main() -> Result<(), Error> {
+ let mut fory = Fory::default();
+ fory.register::<User>(1)?;
+
+ let user = User {
+ name: "Alice".to_string(),
+ age: 30,
+ email: "[email protected]".to_string(),
+ };
+
+ // Serialize
+ let bytes = fory.serialize(&user)?;
+ // Deserialize
+ let decoded: User = fory.deserialize(&bytes)?;
+ assert_eq!(user, decoded);
+
+ // Serialize to specified buffer
+ let mut buf: Vec<u8> = vec![];
+ fory.serialize_to(&user, &mut buf)?;
+ // Deserialize from specified buffer
+ let mut reader = Reader::new(&buf);
+ let decoded: User = fory.deserialize_from(&mut reader)?;
+ assert_eq!(user, decoded);
+ Ok(())
+}
+```
+
+## ๐ Core Features
+
+### 1. Object Graph Serialization
+
+Apache Foryโข provides automatic serialization of complex object graphs,
preserving the structure and relationships between objects. The
`#[derive(ForyObject)]` macro generates efficient serialization code at compile
time, eliminating runtime overhead.
+
+**Key capabilities:**
+
+- Nested struct serialization with arbitrary depth
+- Collection types (Vec, HashMap, HashSet, BTreeMap)
+- Optional fields with `Option<T>`
+- Automatic handling of primitive types and strings
+- Efficient binary encoding with variable-length integers
+
+```rust
+use fory::{Fory, Error};
+use fory::ForyObject;
+use std::collections::HashMap;
+
+#[derive(ForyObject, Debug, PartialEq)]
+struct Person {
+ name: String,
+ age: i32,
+ address: Address,
+ hobbies: Vec<String>,
+ metadata: HashMap<String, String>,
+}
+
+#[derive(ForyObject, Debug, PartialEq)]
+struct Address {
+ street: String,
+ city: String,
+ country: String,
+}
+
+let mut fory = Fory::default();
+fory.register::<Address>(100);
+fory.register::<Person>(200);
+
+let person = Person {
+ name: "John Doe".to_string(),
+ age: 30,
+ address: Address {
+ street: "123 Main St".to_string(),
+ city: "New York".to_string(),
+ country: "USA".to_string(),
+ },
+ hobbies: vec!["reading".to_string(), "coding".to_string()],
+ metadata: HashMap::from([
+ ("role".to_string(), "developer".to_string()),
+ ]),
+};
+
+let bytes = fory.serialize(&person);
+let decoded: Person = fory.deserialize(&bytes)?;
+assert_eq!(person, decoded);
+```
+
+### 2. Shared and Circular References
+
+Apache Foryโข automatically tracks and preserves reference identity for shared
objects using `Rc<T>` and `Arc<T>`. When the same object is referenced multiple
times, Fory serializes it only once and uses reference IDs for subsequent
occurrences. This ensures:
+
+- **Space efficiency**: No data duplication in serialized output
+- **Reference identity preservation**: Deserialized objects maintain the same
sharing relationships
+- **Circular reference support**: Use `RcWeak<T>` and `ArcWeak<T>` to break
cycles
+
+#### Shared References with Rc/Arc
+
+```rust
+use fory::Fory;
+use std::rc::Rc;
+
+let fory = Fory::default();
+
+// Create a shared value
+let shared = Rc::new(String::from("shared_value"));
+
+// Reference it multiple times
+let data = vec![shared.clone(), shared.clone(), shared.clone()];
+
+// The shared value is serialized only once
+let bytes = fory.serialize(&data);
+let decoded: Vec<Rc<String>> = fory.deserialize(&bytes)?;
+
+// Verify reference identity is preserved
+assert_eq!(decoded.len(), 3);
+assert_eq!(*decoded[0], "shared_value");
+
+// All three Rc pointers point to the same object
+assert!(Rc::ptr_eq(&decoded[0], &decoded[1]));
+assert!(Rc::ptr_eq(&decoded[1], &decoded[2]));
+```
+
+For thread-safe shared references, use `Arc<T>`.
+
+#### Circular References with Weak Pointers
+
+To serialize circular references like parent-child relationships or
doubly-linked structures, use `RcWeak<T>` or `ArcWeak<T>` to break the cycle.
These weak pointers are serialized as references to their strong counterparts,
preserving the graph structure without causing memory leaks or infinite
recursion.
+
+**How it works:**
+
+- Weak pointers serialize as references to their target objects
+- If the strong pointer has been dropped, weak serializes as `Null`
+- Forward references (weak appearing before target) are resolved via callbacks
+- All clones of a weak pointer share the same internal cell for automatic
updates
+
+```rust
+use fory::{Fory, Error};
+use fory::ForyObject;
+use fory::RcWeak;
+use std::rc::Rc;
+use std::cell::RefCell;
+
+#[derive(ForyObject, Debug)]
+struct Node {
+ value: i32,
+ parent: RcWeak<RefCell<Node>>,
+ children: Vec<Rc<RefCell<Node>>>,
+}
+
+let mut fory = Fory::default();
+fory.register::<Node>(2000);
+
+// Build a parent-child tree
+let parent = Rc::new(RefCell::new(Node {
+ value: 1,
+ parent: RcWeak::new(),
+ children: vec![],
+}));
+
+let child1 = Rc::new(RefCell::new(Node {
+ value: 2,
+ parent: RcWeak::from(&parent),
+ children: vec![],
+}));
+
+let child2 = Rc::new(RefCell::new(Node {
+ value: 3,
+ parent: RcWeak::from(&parent),
+ children: vec![],
+}));
+
+parent.borrow_mut().children.push(child1.clone());
+parent.borrow_mut().children.push(child2.clone());
+
+// Serialize and deserialize the circular structure
+let bytes = fory.serialize(&parent);
+let decoded: Rc<RefCell<Node>> = fory.deserialize(&bytes)?;
+
+// Verify the circular relationship
+assert_eq!(decoded.borrow().children.len(), 2);
+for child in &decoded.borrow().children {
+ let upgraded_parent = child.borrow().parent.upgrade().unwrap();
+ assert!(Rc::ptr_eq(&decoded, &upgraded_parent));
+}
+```
+
+### 3. Trait Object Serialization
+
+Apache Foryโข supports polymorphic serialization through trait objects,
enabling dynamic dispatch and type flexibility. This is essential for plugin
systems, heterogeneous collections, and extensible architectures.
+
+**Supported trait object types:**
+
+- `Box<dyn Trait>` - Owned trait objects
+- `Rc<dyn Trait>` - Reference-counted trait objects
+- `Arc<dyn Trait>` - Thread-safe reference-counted trait objects
+- `Box<dyn Any>`/`Rc<dyn Any>`/`Arc<dyn Any>` - Any trait type objects
+- `Vec<Box<dyn Trait>>`, `HashMap<K, Box<dyn Trait>>` - Collections of trait
objects
+
+**Basic Trait Object Serialization Example:**
+
+```rust
+use fory::{Fory, register_trait_type};
+use fory::Serializer;
+use fory::ForyObject;
+
+trait Animal: Serializer {
+ fn speak(&self) -> String;
+ fn name(&self) -> &str;
+}
+
+#[derive(ForyObject)]
+struct Dog { name: String, breed: String }
+
+impl Animal for Dog {
+ fn speak(&self) -> String { "Woof!".to_string() }
+ fn name(&self) -> &str { &self.name }
+}
+
+#[derive(ForyObject)]
+struct Cat { name: String, color: String }
+
+impl Animal for Cat {
+ fn speak(&self) -> String { "Meow!".to_string() }
+ fn name(&self) -> &str { &self.name }
+}
+
+// Register trait implementations
+register_trait_type!(Animal, Dog, Cat);
+
+#[derive(ForyObject)]
+struct Zoo {
+ star_animal: Box<dyn Animal>,
+}
+
+let mut fory = Fory::default().compatible(true);
+fory.register::<Dog>(100);
+fory.register::<Cat>(101);
+fory.register::<Zoo>(102);
+
+let zoo = Zoo {
+ star_animal: Box::new(Dog {
+ name: "Buddy".to_string(),
+ breed: "Labrador".to_string(),
+ }),
+};
+
+let bytes = fory.serialize(&zoo);
+let decoded: Zoo = fory.deserialize(&bytes)?;
+
+assert_eq!(decoded.star_animal.name(), "Buddy");
+assert_eq!(decoded.star_animal.speak(), "Woof!");
+```
+
+### 4. Schema Evolution
+
+Apache Foryโข supports schema evolution in **Compatible mode**, allowing
serialization and deserialization peers to have different type definitions.
This enables independent evolution of services in distributed systems without
breaking compatibility.
+
+**Features:**
+
+- Add new fields with default values
+- Remove obsolete fields (skipped during deserialization)
+- Change field nullability (`T` โ `Option<T>`)
+- Reorder fields (matched by name, not position)
+- Type-safe fallback to default values for missing fields
+
+**Compatibility rules:**
+
+- Field names must match (case-sensitive)
+- Type changes are not supported (except nullable/non-nullable)
+- Nested struct types must be registered on both sides
+
+```rust
+use fory::Fory;
+use fory::ForyObject;
+use std::collections::HashMap;
+
+#[derive(ForyObject, Debug)]
+struct PersonV1 {
+ name: String,
+ age: i32,
+ address: String,
+}
+
+#[derive(ForyObject, Debug)]
+struct PersonV2 {
+ name: String,
+ age: i32,
+ // address removed
+ // phone added
+ phone: Option<String>,
+ metadata: HashMap<String, String>,
+}
+
+let mut fory1 = Fory::default().compatible(true);
+fory1.register::<PersonV1>(1);
+
+let mut fory2 = Fory::default().compatible(true);
+fory2.register::<PersonV2>(1);
+
+let person_v1 = PersonV1 {
+ name: "Alice".to_string(),
+ age: 30,
+ address: "123 Main St".to_string(),
+};
+
+// Serialize with V1
+let bytes = fory1.serialize(&person_v1);
+
+// Deserialize with V2 - missing fields get default values
+let person_v2: PersonV2 = fory2.deserialize(&bytes)?;
+assert_eq!(person_v2.name, "Alice");
+assert_eq!(person_v2.age, 30);
+assert_eq!(person_v2.phone, None);
+```
+
+### 5. Enum Support
+
+Apache Foryโข supports three types of enum variants with full schema evolution
in Compatible mode:
+
+**Variant Types:**
+
+- **Unit**: C-style enums (`Status::Active`)
+- **Unnamed**: Tuple-like variants (`Message::Pair(String, i32)`)
+- **Named**: Struct-like variants (`Event::Click { x: i32, y: i32 }`)
+
+**Features:**
+
+- Efficient varint encoding for variant ordinals
+- Schema evolution support (add/remove variants, add/remove fields)
+- Default variant support with `#[default]`
+- Automatic type mismatch handling
+
+```rust
+use fory::{Fory, ForyObject};
+
+#[derive(Default, ForyObject, Debug, PartialEq)]
+enum Value {
+ #[default]
+ Null,
+ Bool(bool),
+ Number(f64),
+ Text(String),
+ Object { name: String, value: i32 },
+}
+
+let mut fory = Fory::default();
+fory.register::<Value>(1)?;
+
+let value = Value::Object { name: "score".to_string(), value: 100 };
+let bytes = fory.serialize(&value)?;
+let decoded: Value = fory.deserialize(&bytes)?;
+assert_eq!(value, decoded);
+```
+
+**Evolution capabilities:**
+
+- **Unknown variants** โ Falls back to default variant
+- **Named variant fields** โ Add/remove fields (missing fields use defaults)
+- **Unnamed variant elements** โ Add/remove elements (extras skipped, missing
use defaults)
+- **Variant type mismatches** โ Automatically uses default value for current
variant
+
+**Best practices:**
+
+- Always mark a default variant with `#[default]`
+- Named variants provide better evolution than unnamed
+- Use compatible mode for cross-version communication
+
+### 6. Tuple Support
+
+Apache Foryโข supports tuples up to 22 elements out of the box with efficient
serialization in both compatible and non-compatible modes.
+
+**Features:**
+
+- Automatic serialization for tuples from 1 to 22 elements
+- Heterogeneous type support (each element can be a different type)
+- Schema evolution in Compatible mode (handles missing/extra elements)
+
+**Serialization modes:**
+
+1. **Non-compatible mode**: Serializes elements sequentially without
collection headers for minimal overhead
+2. **Compatible mode**: Uses collection protocol with type metadata for schema
evolution
+
+```rust
+use fory::{Fory, Error};
+
+let mut fory = Fory::default();
+
+// Tuple with heterogeneous types
+let data: (i32, String, bool, Vec<i32>) = (
+ 42,
+ "hello".to_string(),
+ true,
+ vec![1, 2, 3],
+);
+
+let bytes = fory.serialize(&data)?;
+let decoded: (i32, String, bool, Vec<i32>) = fory.deserialize(&bytes)?;
+assert_eq!(data, decoded);
+```
+
+### 7. Custom Serializers
+
+For types that don't support `#[derive(ForyObject)]`, implement the
`Serializer` trait manually. This is useful for:
+
+- External types from other crates
+- Types with special serialization requirements
+- Legacy data format compatibility
+- Performance-critical custom encoding
+
+```rust
+use fory::{Fory, ReadContext, WriteContext, Serializer, ForyDefault, Error};
+use std::any::Any;
+
+#[derive(Debug, PartialEq)]
+struct CustomType {
+ value: i32,
+ name: String,
+}
+
+impl Serializer for CustomType {
+ fn fory_write_data(&self, context: &mut WriteContext, is_field: bool) {
+ context.writer.write_i32(self.value);
+ context.writer.write_varuint32(self.name.len() as u32);
+ context.writer.write_utf8_string(&self.name);
+ }
+
+ fn fory_read_data(context: &mut ReadContext, is_field: bool) ->
Result<Self, Error> {
+ let value = context.reader.read_i32();
+ let len = context.reader.read_varuint32() as usize;
+ let name = context.reader.read_utf8_string(len);
+ Ok(Self { value, name })
+ }
+
+ fn fory_type_id_dyn(&self, type_resolver: &TypeResolver) -> u32 {
+ Self::fory_get_type_id(type_resolver)
+ }
+
+ fn as_any(&self) -> &dyn Any {
+ self
+ }
+}
+
+impl ForyDefault for CustomType {
+ fn fory_default() -> Self {
+ Self::default()
+ }
+}
+
+let mut fory = Fory::default();
+fory.register_serializer::<CustomType>(100);
+
+let custom = CustomType {
+ value: 42,
+ name: "test".to_string(),
+};
+let bytes = fory.serialize(&custom);
+let decoded: CustomType = fory.deserialize(&bytes)?;
+assert_eq!(custom, decoded);
+```
+
+### 7. Row-Based Serialization
+
+Apache Foryโข provides a high-performance **row format** for zero-copy
deserialization. Unlike traditional object serialization that reconstructs
entire objects in memory, row format enables **random access** to fields
directly from binary data without full deserialization.
+
+**Key benefits:**
+
+- **Zero-copy access**: Read fields without allocating or copying data
+- **Partial deserialization**: Access only the fields you need
+- **Memory-mapped files**: Work with data larger than RAM
+- **Cache-friendly**: Sequential memory layout for better CPU cache utilization
+- **Lazy evaluation**: Defer expensive operations until field access
+
+**When to use row format:**
+
+- Analytics workloads with selective field access
+- Large datasets where only a subset of fields is needed
+- Memory-constrained environments
+- High-throughput data pipelines
+- Reading from memory-mapped files or shared memory
+
+**How it works:**
+
+- Fields are encoded in a binary row with fixed offsets for primitives
+- Variable-length data (strings, collections) stored with offset pointers
+- Null bitmap tracks which fields are present
+- Nested structures supported through recursive row encoding
+
+```rust
+use fory::{to_row, from_row};
+use fory::ForyRow;
+use std::collections::BTreeMap;
+
+#[derive(ForyRow)]
+struct UserProfile {
+ id: i64,
+ username: String,
+ email: String,
+ scores: Vec<i32>,
+ preferences: BTreeMap<String, String>,
+ is_active: bool,
+}
+
+let profile = UserProfile {
+ id: 12345,
+ username: "alice".to_string(),
+ email: "[email protected]".to_string(),
+ scores: vec![95, 87, 92, 88],
+ preferences: BTreeMap::from([
+ ("theme".to_string(), "dark".to_string()),
+ ("language".to_string(), "en".to_string()),
+ ]),
+ is_active: true,
+};
+
+// Serialize to row format
+let row_data = to_row(&profile);
+
+// Zero-copy deserialization - no object allocation!
+let row = from_row::<UserProfile>(&row_data);
+
+// Access fields directly from binary data
+assert_eq!(row.id(), 12345);
+assert_eq!(row.username(), "alice");
+assert_eq!(row.email(), "[email protected]");
+assert_eq!(row.is_active(), true);
+
+// Access collections efficiently
+let scores = row.scores();
+assert_eq!(scores.size(), 4);
+assert_eq!(scores.get(0), 95);
+assert_eq!(scores.get(1), 87);
+
+let prefs = row.preferences();
+assert_eq!(prefs.keys().size(), 2);
+assert_eq!(prefs.keys().get(0), "language");
+assert_eq!(prefs.values().get(0), "en");
+```
+
+**Performance comparison:**
+
+| Operation | Object Format | Row Format
|
+| -------------------- | ----------------------------- |
------------------------------- |
+| Full deserialization | Allocates all objects | Zero allocation
|
+| Single field access | Full deserialization required | Direct offset read
|
+| Memory usage | Full object graph in memory | Only accessed fields
in memory |
+| Suitable for | Small objects, full access | Large objects,
selective access |
+
+## ๐ Cross-Language Serialization
+
+Apache Foryโข supports seamless data exchange across multiple languages:
+
+```rust
+use fory::Fory;
+
+// Enable cross-language mode
+let mut fory = Fory::default()
+ .compatible(true)
+ .xlang(true);
+
+// Register types with consistent IDs across languages
+fory.register::<MyStruct>(100);
+
+// Or use namespace-based registration
+fory.register_by_namespace::<MyStruct>("com.example", "MyStruct");
+```
+
+See
[xlang_type_mapping.md](https://fory.apache.org/docs/specification/xlang_type_mapping)
for type mapping across languages.
+
+## โก Performance
+
+Apache Foryโข Rust is designed for maximum performance:
+
+- **Zero-Copy Deserialization**: Row format enables direct memory access
without copying
+- **Buffer Pre-allocation**: Minimizes memory allocations during serialization
+- **Compact Encoding**: Variable-length encoding for space efficiency
+- **Little-Endian**: Optimized for modern CPU architectures
+- **Reference Deduplication**: Shared objects serialized only once
+
+Run benchmarks:
+
+```bash
+cd benchmarks/rust_benchmark
+cargo bench
+```
+
+## ๐ Documentation
+
+- **[User Guide](https://fory.apache.org/docs/next/docs/guide/rust)** -
Comprehensive User documents
+- **[API Documentation](https://docs.rs/fory)** - Complete API reference
+- **[Protocol
Specification](https://fory.apache.org/docs/specification/fory_xlang_serialization_spec)**
- Serialization protocol details
+- **[Type
Mapping](https://fory.apache.org/docs/specification/xlang_type_mapping)** -
Cross-language type mappings
+- **[Source](https://github.com/apache/fory/tree/main/docs/guide/rust)** -
Source code for doc
+
+## ๐ฏ Use Cases
+
+### Object Serialization
+
+- Complex data structures with nested objects and references
+- Cross-language communication in microservices
+- General-purpose serialization with full type safety
+- Schema evolution with compatible mode
+- Graph-like data structures with circular references
+
+### Row-Based Serialization
+
+- High-throughput data processing
+- Analytics workloads requiring fast field access
+- Memory-constrained environments
+- Real-time data streaming applications
+- Zero-copy scenarios
+
+## ๐ ๏ธ Development
+
+### Building
+
+```bash
+cd rust
+cargo build
+```
+
+### Testing
+
+```bash
+# Run all tests
+cargo test --features tests
+
+# Run specific test
+cargo test -p tests --test test_complex_struct
+```
+
+### Code Quality
+
+```bash
+# Format code
+cargo fmt
+
+# Check formatting
+cargo fmt --check
+
+# Run linter
+cargo clippy --all-targets --all-features -- -D warnings
+```
+
+## ๐ License
+
+Licensed under the Apache License, Version 2.0. See
[LICENSE](https://github.com/apache/fory/blob/main/LICENSE) for details.
+
+## ๐ค Contributing
+
+We welcome contributions! Please see our [Contributing
Guide](https://github.com/apache/fory/blob/main/CONTRIBUTING.md) for details.
+
+## ๐ Support
+
+- **Documentation**: [docs.rs/fory](https://docs.rs/fory)
+- **Issues**: [GitHub Issues](https://github.com/apache/fory/issues)
+- **Discussions**: [GitHub
Discussions](https://github.com/apache/fory/discussions)
+- **Slack**: [Apache Fory
Slack](https://join.slack.com/t/fory-project/shared_invite/zt-1u8soj4qc-ieYEu7ciHOqA2mo47llS8A)
+
+---
+
+**Apache Foryโข** - Blazingly fast multi-language serialization framework.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
