This is an automated email from the ASF dual-hosted git repository. chaokunyang pushed a commit to branch chaokunyang-patch-2 in repository https://gitbox.apache.org/repos/asf/fory-site.git
commit 535902467ec8f92017864410a24a7123fbb9e67c Author: Shawn Yang <[email protected]> AuthorDate: Tue Nov 11 22:49:58 2025 +0800 fix markdown table error in java serialization guide doc --- .../docs/guide/java_serialization_guide.md | 319 +++++++++++++++++++-- 1 file changed, 291 insertions(+), 28 deletions(-) diff --git a/versioned_docs/version-0.13/docs/guide/java_serialization_guide.md b/versioned_docs/version-0.13/docs/guide/java_serialization_guide.md index e00eb4d7a..8b3af5580 100644 --- a/versioned_docs/version-0.13/docs/guide/java_serialization_guide.md +++ b/versioned_docs/version-0.13/docs/guide/java_serialization_guide.md @@ -1,7 +1,7 @@ --- title: Java Serialization Guide sidebar_position: 0 -id: java_object_graph_guide +id: java_serialization license: | Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with @@ -47,6 +47,7 @@ public class Example { .build(); // Registering types can reduce class name serialization overhead, but not mandatory. // If class registration enabled, all custom types must be registered. + // Registration order must be consistent if id is not specified fory.register(SomeClass.class); byte[] bytes = fory.serialize(object); System.out.println(fory.deserialize(bytes)); @@ -108,32 +109,31 @@ public class Example { ## ForyBuilder options -| Option Name | Description [...] -| --------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- [...] -| `timeRefIgnored` | Whether to ignore reference tracking of all time types registered in `TimeSerializers` and subclasses of those types when ref tracking is enabled. If ignored, ref tracking of every time type can be enabled by invoking `Fory#registerSerializer(Class, Serializer)`. For example, `fory.registerSerializer(Date.class, new DateSerializer(fory, true))`. Note that enabling ref tracking should happen before serializer codegen of any types which contain time fields. [...] -| `compressInt` | Enables or disables int compression for smaller size. [...] -| `compressLong` | Enables or disables long compression for smaller size. [...] -| `compressIntArray` | Enables or disables SIMD-accelerated compression for int arrays when values can fit in smaller data types. Requires Java 16+. [...] -| `compressLongArray` | Enables or disables SIMD-accelerated compression for long arrays when values can fit in smaller data types. Requires Java 16+. [...] -| `compressString` | Enables or disables string compression for smaller size. [...] -| `classLoader` | The classloader should not be updated; Fory caches class metadata. Use `LoaderBinding` or `ThreadSafeFory` for classloader updates. [...] -| `compatibleMode` | Type forward/backward compatibility config. Also Related to `checkClassVersion` config. `SCHEMA_CONSISTENT`: Class schema must be consistent between serialization peer and deserialization peer. `COMPATIBLE`: Class schema can be different between serialization peer and deserialization peer. They can add/delete fields independently. [See more](#class-inconsistency-and-class-version-check). [...] -| `checkClassVersion` | Determines whether to check the consistency of the class schema. If enabled, Fory checks, writes, and checks consistency using the `classVersionHash`. It will be automatically disabled when `CompatibleMode#COMPATIBLE` is enabled. Disabling is not recommended unless you can ensure the class won't evolve. [...] -| `checkJdkClassSerializable` | Enables or disables checking of `Serializable` interface for classes under `java.*`. If a class under `java.*` is not `Serializable`, Fory will throw an `UnsupportedOperationException`. [...] -| `registerGuavaTypes` | Whether to pre-register Guava types such as `RegularImmutableMap`/`RegularImmutableList`. These types are not public API, but seem pretty stable. [...] -| `requireClassRegistration` | Disabling may allow unknown classes to be deserialized, potentially causing security risks. [...] -| `requireClassRegistration` | Set max depth for deserialization, when depth exceeds, an exception will be thrown. This can be used to refuse deserialization DDOS attack. [...] - -| `suppressClassRegistrationWarnings` | Whether to suppress class registration warnings. The warnings can be used for security audit, but may be annoying, this suppression will be enabled by default. | `true` | -| `metaShareEnabled` | Enables or disables meta share mode. | `true` if `CompatibleMode.Compatible` is set, otherwise false. | -| `scopedMetaShareEnabled` | Scoped meta share focuses on a single serialization process. Metadata created or identified during this process is exclusive to it and is not shared with by other serializations. | `true` if `CompatibleMode.Compatible` is set, otherwise false. | -| `metaCompressor` | Set a compressor for meta compression. Note that the passed MetaCompressor should be thread-safe. By default, a `Deflater` based compressor `DeflaterMetaCompressor` will be used. Users can pass other compressor such as `zstd` for better compression rate. | `DeflaterMetaCompressor` | -| `deserializeNonexistentClass` | Enables or disables deserialization/skipping of data for non-existent classes. | `true` if `CompatibleMode.Compatible` is set, otherwise false. | -| `codeGenEnabled` | Disabling may result in faster initial serialization but slower subsequent serializations. | `true` | -| `asyncCompilationEnabled` | If enabled, serialization uses interpreter mode first and switches to JIT serialization after async serializer JIT for a class is finished. | `false` | -| `scalaOptimizationEnabled` | Enables or disables Scala-specific serialization optimization. | `false` | -| `copyRef` | When disabled, the copy performance will be better. But fory deep copy will ignore circular and shared reference. Same reference of an object graph will be copied into different objects in one `Fory#copy`. | `true` | -| `serializeEnumByName` | When Enabled, fory serialize enum by name instead of ordinal. | `false` | +| Option Name | Description [...] +| ----------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- [...] +| `timeRefIgnored` | Whether to ignore reference tracking of all time types registered in `TimeSerializers` and subclasses of those types when ref tracking is enabled. If ignored, ref tracking of every time type can be enabled by invoking `Fory#registerSerializer(Class, Serializer)`. For example, `fory.registerSerializer(Date.class, new DateSerializer(fory, true))`. Note that enabling ref tracking should happen before serializer codegen of any types which contain time [...] +| `compressInt` | Enables or disables int compression for smaller size. [...] +| `compressLong` | Enables or disables long compression for smaller size. [...] +| `compressIntArray` | Enables or disables SIMD-accelerated compression for int arrays when values can fit in smaller data types. Requires Java 16+. [...] +| `compressLongArray` | Enables or disables SIMD-accelerated compression for long arrays when values can fit in smaller data types. Requires Java 16+. [...] +| `compressString` | Enables or disables string compression for smaller size. [...] +| `classLoader` | The classloader should not be updated; Fory caches class metadata. Use `LoaderBinding` or `ThreadSafeFory` for classloader updates. [...] +| `compatibleMode` | Type forward/backward compatibility config. Also Related to `checkClassVersion` config. `SCHEMA_CONSISTENT`: Class schema must be consistent between serialization peer and deserialization peer. `COMPATIBLE`: Class schema can be different between serialization peer and deserialization peer. They can add/delete fields independently. [See more](#class-inconsistency-and-class-version-check). [...] +| `checkClassVersion` | Determines whether to check the consistency of the class schema. If enabled, Fory checks, writes, and checks consistency using the `classVersionHash`. It will be automatically disabled when `CompatibleMode#COMPATIBLE` is enabled. Disabling is not recommended unless you can ensure the class won't evolve. [...] +| `checkJdkClassSerializable` | Enables or disables checking of `Serializable` interface for classes under `java.*`. If a class under `java.*` is not `Serializable`, Fory will throw an `UnsupportedOperationException`. [...] +| `registerGuavaTypes` | Whether to pre-register Guava types such as `RegularImmutableMap`/`RegularImmutableList`. These types are not public API, but seem pretty stable. [...] +| `requireClassRegistration` | Disabling may allow unknown classes to be deserialized, potentially causing security risks. [...] +| `requireClassRegistration` | Set max depth for deserialization, when depth exceeds, an exception will be thrown. This can be used to refuse deserialization DDOS attack. [...] +| `suppressClassRegistrationWarnings` | Whether to suppress class registration warnings. The warnings can be used for security audit, but may be annoying, this suppression will be enabled by default. [...] +| `metaShareEnabled` | Enables or disables meta share mode. [...] +| `scopedMetaShareEnabled` | Scoped meta share focuses on a single serialization process. Metadata created or identified during this process is exclusive to it and is not shared with by other serializations. [...] +| `metaCompressor` | Set a compressor for meta compression. Note that the passed MetaCompressor should be thread-safe. By default, a `Deflater` based compressor `DeflaterMetaCompressor` will be used. Users can pass other compressor such as `zstd` for better compression rate. [...] +| `deserializeNonexistentClass` | Enables or disables deserialization/skipping of data for non-existent classes. [...] +| `codeGenEnabled` | Disabling may result in faster initial serialization but slower subsequent serializations. [...] +| `asyncCompilationEnabled` | If enabled, serialization uses interpreter mode first and switches to JIT serialization after async serializer JIT for a class is finished. [...] +| `scalaOptimizationEnabled` | Enables or disables Scala-specific serialization optimization. [...] +| `copyRef` | When disabled, the copy performance will be better. But fory deep copy will ignore circular and shared reference. Same reference of an object graph will be copied into different objects in one `Fory#copy`. [...] +| `serializeEnumByName` | When Enabled, fory serialize enum by name instead of ordinal. [...] ## Advanced Usage @@ -286,7 +286,7 @@ For Maven: <dependency> <groupId>org.apache.fory</groupId> <artifactId>fory-simd</artifactId> - <version>0.13.0-SNAPSHOT</version> + <version>0.13.1</version> </dependency> ``` @@ -309,6 +309,213 @@ SomeClass a = xxx; SomeClass copied = fory.copy(a); ``` +### Cross-Language Serialization + +Apache Fory™ supports seamless data exchange between Java and other languages (Python, Rust, Go, JavaScript, etc.) through the xlang serialization format. This enables multi-language microservices, polyglot data pipelines, and cross-platform data sharing. + +#### Enable Cross-Language Mode + +To serialize data for consumption by other languages, use `Language.XLANG` mode: + +```java +import org.apache.fory.*; +import org.apache.fory.config.*; + +// Create Fory instance with XLANG mode +Fory fory = Fory.builder() + .withLanguage(Language.XLANG) + .withRefTracking(true) // Enable reference tracking for complex graphs + .build(); +``` + +#### Register Types for Cross-Language Compatibility + +Types must be registered with **consistent IDs or names** across all languages. Fory supports two registration methods: + +**1. Register by ID (Recommended for Performance)** + +```java +public record Person(String name, int age) {} + +// Register with numeric ID - faster and more compact +fory.register(Person.class, 1); + +Person person = new Person("Alice", 30); +byte[] bytes = fory.serialize(person); +// bytes can be deserialized by Python, Rust, Go, etc. +``` + +**Benefits**: Faster serialization, smaller binary size +**Trade-offs**: Requires coordination to avoid ID conflicts across teams/services + +**2. Register by Name (Recommended for Flexibility)** + +```java +public record Person(String name, int age) {} + +// Register with string name - more flexible +fory.register(Person.class, "example.Person"); + +Person person = new Person("Alice", 30); +byte[] bytes = fory.serialize(person); +// bytes can be deserialized by Python, Rust, Go, etc. +``` + +**Benefits**: Less prone to conflicts, easier management across teams, no coordination needed +**Trade-offs**: Slightly larger binary size due to string encoding + +#### Cross-Language Example: Java ↔ Python + +**Java (Serializer)** + +```java +import org.apache.fory.*; +import org.apache.fory.config.*; + +public record Person(String name, int age) {} + +public class Example { + public static void main(String[] args) { + Fory fory = Fory.builder() + .withLanguage(Language.XLANG) + .withRefTracking(true) + .build(); + + // Register with consistent name + fory.register(Person.class, "example.Person"); + + Person person = new Person("Bob", 25); + byte[] bytes = fory.serialize(person); + + // Send bytes to Python service via network/file/queue + } +} +``` + +**Python (Deserializer)** + +```python +import pyfory +from dataclasses import dataclass + +@dataclass +class Person: + name: str + age: pyfory.int32 + +# Create Fory in xlang mode +fory = pyfory.Fory(ref_tracking=True) + +# Register with the SAME name as Java +fory.register_type(Person, typename="example.Person") + +# Deserialize bytes from Java +person = fory.deserialize(bytes_from_java) +print(f"{person.name}, {person.age}") # Output: Bob, 25 +``` + +#### Handling Circular and Shared References + +Cross-language mode supports circular and shared references when reference tracking is enabled: + +```java +public class Node { + public String value; + public Node next; + public Node parent; +} + +Fory fory = Fory.builder() + .withLanguage(Language.XLANG) + .withRefTracking(true) // Required for circular references + .build(); + +fory.register(Node.class, "example.Node"); + +// Create circular reference +Node node1 = new Node(); +node1.value = "A"; +Node node2 = new Node(); +node2.value = "B"; +node1.next = node2; +node2.parent = node1; // Circular reference + +byte[] bytes = fory.serialize(node1); +// Python/Rust/Go can correctly deserialize this with circular references preserved +``` + +#### Type Mapping Considerations + +Not all Java types have equivalents in other languages. When using xlang mode: + +- Use **primitive types** (`int`, `long`, `double`, `String`) for maximum compatibility +- Use **standard collections** (`List`, `Map`, `Set`) instead of language-specific ones +- Avoid **Java-specific types** like `Optional`, `BigDecimal` (unless the target language supports them) +- See [Type Mapping Guide](https://fory.apache.org/docs/specification/xlang_type_mapping) for complete compatibility matrix + +**Compatible Types**: + +```java +public record UserData( + String name, // ✅ Compatible + int age, // ✅ Compatible + List<String> tags, // ✅ Compatible + Map<String, Integer> scores // ✅ Compatible +) {} +``` + +**Problematic Types**: + +```java +public record UserData( + Optional<String> name, // ❌ Not cross-language compatible + BigDecimal balance, // ❌ Limited support + EnumSet<Status> statuses // ❌ Java-specific collection +) {} +``` + +#### Performance Considerations + +Cross-language mode has additional overhead compared to Java-only mode: + +- **Type metadata encoding**: Adds extra bytes per type +- **Type resolution**: Requires name/ID lookup during deserialization + +**For best performance**: + +- Use **ID-based registration** when possible (smaller encoding) +- **Disable reference tracking** if you don't need circular references (`withRefTracking(false)`) +- **Use Java mode** (`Language.JAVA`) when only Java serialization is needed + +#### Cross-Language Best Practices + +1. **Consistent Registration**: Ensure all services register types with identical IDs/names +2. **Version Compatibility**: Use compatible mode for schema evolution across services + +#### Troubleshooting Cross-Language Serialization + +**"Type not registered" errors**: + +- Verify type is registered with same ID/name on both sides +- Check if type name has typos or case differences + +**"Type mismatch" errors**: + +- Ensure field types are compatible across languages +- Review [Type Mapping Guide](https://fory.apache.org/docs/next/specification/xlang_type_mapping) + +**Data corruption or unexpected values**: + +- Verify both sides use `Language.XLANG` mode +- Ensure both sides have compatible Fory versions + +**See Also**: + +- [Cross-Language Serialization Specification](https://fory.apache.org/docs/next/specification/fory_xlang_serialization_spec) +- [Type Mapping Reference](https://fory.apache.org/docs/next/specification/xlang_type_mapping) +- [Python Cross-Language Guide](python_guide.md#cross-language-serialization) +- [Rust Cross-Language Guide](rust_guide.md#-cross-language-serialization) + ### Implement a customized serializer In some cases, you may want to implement a serializer for your type, especially some class customize serialization by @@ -1422,6 +1629,62 @@ public class StructMappingExample { } ``` +### Logging + +#### ForyLogger + +By default Fory uses a custom logger `ForyLogger` for internal needs. +It builds resulting logged data into single string and sends it directly to `System.out`. +The result line layout is similar to (in Log4j notation): + +``` +%d{yyyy-MM-dd hh:mm:ss} %p %C:%L [%t] - %m%n +``` + +The layout can't be changed. + +Example output: + +``` +2025-11-07 08:49:59 INFO CompileUnit:55 [main] - Generate code for org.apache.fory.builder.SerializedLambdaForyCodec_0 took 35 ms. +2025-11-07 08:50:00 INFO JaninoUtils:121 [main] - Compile [SerializedLambdaForyCodec_0] take 144 ms +``` + +#### Slf4jLogger + +If more sophisticated logger is required, configure Fory to use Slf4j via `LoggerFactory.useSlf4jLogging()`. +For example enabling Slf4j before creating Fory: + +```java +public static final ThreadSafeFory FORY; + +static { + LoggerFactory.useSlf4jLogging(true); + FORY = Fory.builder() + .buildThreadSafeFory(); +} +``` + +**Note:** Enabling Slf4j via `useSlf4jLogging` will be ignored when the application runs in a GraalVM native image. + +#### Suppress Fory logs + +Both `ForyLogger` and `Slf4jLogger` allow controling log outpul level or suppressing logs entirely. +Configure logger level via `LoggerFactory.setLogLevel()`, for example: + +```java +static { + // to log only WARN and high + LoggerFactory.setLogLevel(LogLevel.WARN_LEVEL); + + // to disable at all + LoggerFactory.disableLogging(); +} +``` + +**Note:** Selected logging level is applied before Slfj4 implementation's logger level. So if you set `WARN_LEVEL` (as in +the example above) then you will not see INFO messages from Fory even if INFO is enabled in Logback. + ## Migration ### JDK migration --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
