This is an automated email from the ASF dual-hosted git repository.
chaokunyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/fory.git
The following commit(s) were added to refs/heads/main by this push:
new 1e4339461 fix(xlang): close field tag validation gaps (#3984)
1e4339461 is described below
commit 1e433946138c32c9203b87232c0f3409e10815a6
Author: Shawn Yang <[email protected]>
AuthorDate: Wed Aug 26 12:32:55 2026 +0800
fix(xlang): close field tag validation gaps (#3984)
## Why?
PR #3982 established the signed-int32 field-tag protocol range.
Follow-up integration checks found a
small set of validation ownership, exact-schema coverage, and
documentation gaps that were not part
of the squash-merged boundary.
This PR completes those paths without changing the field-tag wire format
introduced by #3982.
## What does this PR do?
- Rejects duplicate Java native TypeDef tags across the complete
inheritance hierarchy.
- Validates Go field tags before public registration mutates resolver
state while continuing to
ignore unexported fields.
- Keeps Rust protocol metadata equality independent from local
compatible-reader dispatch state.
- Adds causal C++ remote-compatible and oversized-tag decoder coverage.
- Adds an exact-schema cross-language fixture for tags `15`, `65551`,
and `536870911` across all
supported peers, including normal concrete Java test discovery.
- Runs Kotlin's maximum tag through a real KSP-generated serializer,
descriptor, and round trip.
- Documents the exact field-tag range, uniqueness, stability, and
extended encoding rules in the
protocol, compiler, and runtime schema guides.
## Related issues
Follow-up to #3982.
## Does this PR introduce any user-facing change?
- [ ] Does this PR introduce any public API change?
- [ ] Does this PR introduce any binary protocol compatibility change?
Invalid duplicate or out-of-range field-tag schemas are now rejected
consistently at their owning
registration or metadata boundary. Existing valid schemas and wire bytes
are unchanged.
## Test
- Java native TypeDef, Go registration, C++ metadata, Rust metadata
equality, and Kotlin KSP/static
serializer tests passed.
- Exact-schema and compatible Java-driven field-tag cases passed against
C++, C#, Dart, Go,
JavaScript, Python, Rust, Scala, and Swift in both Java codegen modes.
- Dart IDL regeneration, compilation, and Java/Dart semantic round trips
passed.
- Java Spotless, Rust fmt/clippy, Go formatting, C++ formatting,
Markdown Prettier, and full diff
checks passed.
## Benchmark
The production changes in this follow-up are Java remote native-TypeDef
miss validation, Go
registration-time validation, and Rust `FieldInfo` equality. None is
reached by the steady-state
serialize/deserialize benchmark loops from #3982, so those paired
results remain the causal hot-path
evidence. Every measured runtime/path stayed below the `+1.0%`
regression ceiling; the largest
positive paired medians were C++ deserialize at `+0.858%` and Java
static serialize at `+0.779%`.
---
.github/workflows/ci.yml | 2 +-
cpp/fory/serialization/struct_test.cc | 35 ++++++++++++
cpp/fory/serialization/xlang_test_main.cc | 29 ++++++++++
csharp/tests/Fory.XlangPeer/Program.cs | 19 +++++++
.../fory-test/lib/entity/xlang_test_models.dart | 14 +++++
.../test/cross_lang_test/xlang_test_main.dart | 5 ++
docs/compiler/schema-idl.md | 17 +++---
docs/object-serialization/cpp/schema-metadata.md | 5 +-
.../object-serialization/csharp/schema-metadata.md | 3 +-
docs/object-serialization/dart/schema-metadata.md | 1 +
docs/object-serialization/go/schema-metadata.md | 4 ++
docs/object-serialization/go/troubleshooting.md | 10 +++-
docs/object-serialization/java/schema-metadata.md | 15 +++---
.../javascript/schema-metadata.md | 18 +++++++
.../object-serialization/kotlin/schema-metadata.md | 4 ++
.../object-serialization/python/schema-metadata.md | 7 +--
docs/object-serialization/rust/schema-metadata.md | 3 +-
docs/object-serialization/scala/schema-metadata.md | 4 ++
docs/object-serialization/swift/schema-metadata.md | 18 +++++++
docs/specification/java_serialization_spec.md | 12 ++---
docs/specification/xlang_serialization_spec.md | 6 ++-
go/fory/doc.go | 2 +-
go/fory/tag.go | 3 ++
go/fory/tag_test.go | 27 ++++++++--
go/fory/tests/xlang/xlang_test_main.go | 28 ++++++++++
go/fory/type_resolver.go | 6 +++
.../org/apache/fory/meta/NativeTypeDefDecoder.java | 23 +++++---
.../apache/fory/meta/NativeTypeDefEncoderTest.java | 62 ++++++++++++++++++++++
.../java/org/apache/fory/xlang/CPPXlangTest.java | 6 +++
.../org/apache/fory/xlang/CSharpXlangTest.java | 6 +++
.../java/org/apache/fory/xlang/DartXlangTest.java | 6 +++
.../java/org/apache/fory/xlang/GoXlangTest.java | 6 +++
.../org/apache/fory/xlang/JavaScriptXlangTest.java | 6 +++
.../org/apache/fory/xlang/PythonXlangTest.java | 6 +++
.../java/org/apache/fory/xlang/RustXlangTest.java | 6 +++
.../java/org/apache/fory/xlang/ScalaXlangTest.java | 6 +++
.../java/org/apache/fory/xlang/SwiftXlangTest.java | 6 +++
.../java/org/apache/fory/xlang/XlangTestBase.java | 34 ++++++++++++
javascript/test/crossLanguage.test.ts | 21 ++++++++
.../fory/kotlin/ksp/ProcessorValidationTest.kt | 4 +-
.../apache/fory/kotlin/xlang/KotlinXlangPeer.kt | 12 +++++
python/pyfory/field.py | 7 +--
python/pyfory/tests/xlang_test_main.py | 21 ++++++++
rust/fory-core/src/meta/type_meta.rs | 15 +++++-
rust/tests/tests/test_cross_language.rs | 28 ++++++++++
.../fory/serializer/scala/ScalaXlangPeer.scala | 14 +++++
swift/Tests/ForyXlangTests/main.swift | 18 +++++++
47 files changed, 560 insertions(+), 50 deletions(-)
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 7b7e070d7..16108ec7d 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -559,7 +559,7 @@ jobs:
~/.swiftpm
~/Library/Caches/org.swift.swiftpm
swift/.build
- key: ${{ runner.os }}-${{ runner.arch }}-swiftpm-xlang-release-${{
steps.swift-package-cache.outputs.toolchain }}-${{
hashFiles('swift/Package.resolved', 'swift/Package.swift', 'swift/Sources/**')
}}
+ key: ${{ runner.os }}-${{ runner.arch }}-swiftpm-xlang-release-${{
steps.swift-package-cache.outputs.toolchain }}-${{
hashFiles('swift/Package.resolved', 'swift/Package.swift', 'swift/Sources/**',
'swift/Tests/ForyXlangTests/**') }}
restore-keys: |
${{ runner.os }}-${{ runner.arch }}-swiftpm-xlang-release-${{
steps.swift-package-cache.outputs.toolchain }}-
- name: Prebuild Swift xlang peer for cache reuse
diff --git a/cpp/fory/serialization/struct_test.cc
b/cpp/fory/serialization/struct_test.cc
index 67d856185..30bd8c735 100644
--- a/cpp/fory/serialization/struct_test.cc
+++ b/cpp/fory/serialization/struct_test.cc
@@ -146,6 +146,20 @@ struct MaximumFieldTagStruct {
FORY_STRUCT(MaximumFieldTagStruct, (value, fory::F(536870911)));
};
+struct HighTagCompatibleWriter {
+ std::string removed;
+ int32_t shared;
+
+ FORY_STRUCT(HighTagCompatibleWriter, (removed, fory::F(65551)),
+ (shared, fory::F(536870911)));
+};
+
+struct HighTagCompatibleReader {
+ int32_t shared = 0;
+
+ FORY_STRUCT(HighTagCompatibleReader, (shared, fory::F(536870911)));
+};
+
struct SignedToUnsignedWriter {
int32_t value;
@@ -1403,6 +1417,12 @@ TEST(StructComprehensiveTest, FieldTagRange) {
invalid.field_id = 536870912;
EXPECT_FALSE(invalid.to_bytes().ok());
+ Buffer invalid_wire;
+ invalid_wire.write_uint8(static_cast<uint8_t>((3u << 6) | (15u << 2)));
+ invalid_wire.write_var_uint32(536870912u - 15u);
+ invalid_wire.write_uint8(static_cast<uint8_t>(TypeId::INT32));
+ EXPECT_FALSE(FieldInfo::from_bytes(invalid_wire).ok());
+
TypeMeta duplicate;
duplicate.type_id = static_cast<uint32_t>(TypeId::COMPATIBLE_STRUCT);
duplicate.user_type_id = 632;
@@ -1412,6 +1432,21 @@ TEST(StructComprehensiveTest, FieldTagRange) {
EXPECT_FALSE(duplicate.to_bytes().ok());
}
+TEST(StructComprehensiveTest, CompatibleHighTagsRead) {
+ auto writer =
+ Fory::builder().xlang(true).compatible(true).track_ref(false).build();
+ auto reader =
+ Fory::builder().xlang(true).compatible(true).track_ref(false).build();
+ ASSERT_TRUE(writer.register_struct<HighTagCompatibleWriter>(633).ok());
+ ASSERT_TRUE(reader.register_struct<HighTagCompatibleReader>(633).ok());
+
+ auto encoded = writer.serialize(HighTagCompatibleWriter{"skip", 42});
+ ASSERT_TRUE(encoded.ok()) << encoded.error().to_string();
+ auto decoded = reader.deserialize<HighTagCompatibleReader>(*encoded);
+ ASSERT_TRUE(decoded.ok()) << decoded.error().to_string();
+ EXPECT_EQ(decoded->shared, 42);
+}
+
TEST(StructComprehensiveTest, NonPrimitiveFieldsSortByFieldIdentifier) {
auto fields = TypeMeta::sort_field_infos({
make_test_field_info("string_value", 20,
diff --git a/cpp/fory/serialization/xlang_test_main.cc
b/cpp/fory/serialization/xlang_test_main.cc
index 0009504e7..7ccc5049c 100644
--- a/cpp/fory/serialization/xlang_test_main.cc
+++ b/cpp/fory/serialization/xlang_test_main.cc
@@ -137,6 +137,17 @@ struct SimpleStruct {
(f7, fory::F(65551)), f8, (last, fory::F(536870911)));
};
+struct ExactFieldTags {
+ int32_t first;
+ int32_t second;
+ int32_t last;
+ bool operator==(const ExactFieldTags &other) const {
+ return first == other.first && second == other.second && last ==
other.last;
+ }
+ FORY_STRUCT(ExactFieldTags, (first, fory::F(15)), (second, fory::F(65551)),
+ (last, fory::F(536870911)));
+};
+
struct EvolvingOverrideStruct {
std::string f1;
bool operator==(const EvolvingOverrideStruct &other) const {
@@ -938,6 +949,7 @@ namespace {
void run_test_buffer(const std::string &data_file);
void run_test_buffer_var(const std::string &data_file);
void run_test_murmur_hash3(const std::string &data_file);
+void run_test_exact_field_tags(const std::string &data_file);
void run_test_string_serializer(const std::string &data_file);
void run_test_cross_language_serializer(const std::string &data_file);
void run_test_simple_struct(const std::string &data_file);
@@ -1030,6 +1042,8 @@ int main(int argc, char **argv) {
run_test_cross_language_serializer(data_file);
} else if (case_name == "test_simple_struct") {
run_test_simple_struct(data_file);
+ } else if (case_name == "test_exact_field_tags") {
+ run_test_exact_field_tags(data_file);
} else if (case_name == "test_named_simple_struct") {
run_test_simple_named_struct(data_file);
} else if (case_name == "test_struct_evolving_override") {
@@ -1570,6 +1584,21 @@ void run_test_simple_struct(const std::string
&data_file) {
write_file(data_file, out);
}
+void run_test_exact_field_tags(const std::string &data_file) {
+ auto fory = build_fory(false, true);
+ ensure_ok(fory.register_struct<ExactFieldTags>(104),
+ "register ExactFieldTags");
+ auto bytes = read_file(data_file);
+ Buffer buffer = make_buffer(bytes);
+ auto value = read_next<ExactFieldTags>(fory, buffer);
+ if (!(value == ExactFieldTags{39, 40, 41})) {
+ fail("ExactFieldTags mismatch");
+ }
+ std::vector<uint8_t> out;
+ append_serialized(fory, value, out);
+ write_file(data_file, out);
+}
+
void run_test_simple_named_struct(const std::string &data_file) {
auto bytes = read_file(data_file);
auto fory = build_fory(true, true);
diff --git a/csharp/tests/Fory.XlangPeer/Program.cs
b/csharp/tests/Fory.XlangPeer/Program.cs
index a3abca8df..c78a114d6 100644
--- a/csharp/tests/Fory.XlangPeer/Program.cs
+++ b/csharp/tests/Fory.XlangPeer/Program.cs
@@ -194,6 +194,7 @@ internal static class Program
"test_string_serializer" => CaseStringSerializer(input),
"test_cross_language_serializer" =>
CaseCrossLanguageSerializer(input),
"test_simple_struct" => CaseSimpleStruct(input),
+ "test_exact_field_tags" => CaseExactFieldTags(input),
"test_named_simple_struct" => CaseNamedSimpleStruct(input),
"test_struct_evolving_override" =>
CaseStructEvolvingOverride(input),
"test_list" => CaseList(input),
@@ -461,6 +462,13 @@ internal static class Program
return RoundTripSingle<SimpleStruct>(input, fory);
}
+ private static byte[] CaseExactFieldTags(byte[] input)
+ {
+ ForyRuntime fory = BuildFory(compatible: false);
+ fory.Register<ExactFieldTags>(104);
+ return RoundTripSingle<ExactFieldTags>(input, fory);
+ }
+
private static byte[] CaseNamedSimpleStruct(byte[] input)
{
ForyRuntime fory = BuildFory(compatible: true);
@@ -1282,6 +1290,17 @@ public sealed class SimpleStruct
public int Last { get; set; }
}
+[ForyStruct]
+public sealed class ExactFieldTags
+{
+ [ForyField(15)]
+ public int First { get; set; }
+ [ForyField(65551)]
+ public int Second { get; set; }
+ [ForyField(536870911)]
+ public int Last { get; set; }
+}
+
[ForyStruct]
public sealed class EvolvingOverrideStruct
{
diff --git a/dart/packages/fory-test/lib/entity/xlang_test_models.dart
b/dart/packages/fory-test/lib/entity/xlang_test_models.dart
index 5c7ac5229..467b73b49 100644
--- a/dart/packages/fory-test/lib/entity/xlang_test_models.dart
+++ b/dart/packages/fory-test/lib/entity/xlang_test_models.dart
@@ -79,6 +79,20 @@ class SimpleStruct {
int last = 0;
}
+@ForyStruct()
+class ExactFieldTags {
+ ExactFieldTags();
+
+ @ForyField(id: 15, type: Int32Type())
+ int first = 0;
+
+ @ForyField(id: 65551, type: Int32Type())
+ int second = 0;
+
+ @ForyField(id: 536870911, type: Int32Type())
+ int last = 0;
+}
+
@ForyStruct()
class EvolvingOverrideStruct {
EvolvingOverrideStruct();
diff --git a/dart/packages/fory-test/test/cross_lang_test/xlang_test_main.dart
b/dart/packages/fory-test/test/cross_lang_test/xlang_test_main.dart
index 2f6d4956e..033fb400a 100644
--- a/dart/packages/fory-test/test/cross_lang_test/xlang_test_main.dart
+++ b/dart/packages/fory-test/test/cross_lang_test/xlang_test_main.dart
@@ -491,6 +491,11 @@ void _runCase(String caseName) {
_registerSimpleById(fory);
_roundTripFory(fory);
return;
+ case 'test_exact_field_tags':
+ final fory = _newFory();
+ registerXlangType(fory, ExactFieldTags, id: 104);
+ _roundTripFory(fory);
+ return;
case 'test_named_simple_struct':
final fory = _newFory(compatible: true);
_registerSimpleByName(fory);
diff --git a/docs/compiler/schema-idl.md b/docs/compiler/schema-idl.md
index 4291e9b64..9c1244fdd 100644
--- a/docs/compiler/schema-idl.md
+++ b/docs/compiler/schema-idl.md
@@ -976,6 +976,11 @@ Fields define the properties of a message.
field_type field_name = field_number;
```
+`field_number` is the field tag ID. It must be unique within the message and
+satisfy `0 <= field_number < 2^29` (`0` through `536870911`). Keep
+assigned field numbers stable and reserve removed numbers instead of reusing
+them for different fields.
+
### With Modifiers
```protobuf
@@ -1149,22 +1154,22 @@ Use `ref(thread_safe=false)` in Fory IDL (or
## Field Numbers
-Each field must have a unique positive integer identifier:
+Each field must have a unique tag ID in the protocol range:
```protobuf
message Example {
- string first = 1;
- string second = 2;
- string third = 3;
+ string first = 0;
+ string second = 1;
+ string third = 2;
}
```
**Rules and best practices:**
- Numbers must be unique within a message.
-- Numbers must be positive integers.
+- Numbers must satisfy `0 <= field_number < 2^29` (`0` through `536870911`).
- Gaps are allowed and are useful when fields are removed.
-- Prefer sequential numbering from `1`.
+- Prefer sequential numbering.
- Never reuse a removed field number for a different field.
## Type System
diff --git a/docs/object-serialization/cpp/schema-metadata.md
b/docs/object-serialization/cpp/schema-metadata.md
index c60f3866e..5c05de1c0 100644
--- a/docs/object-serialization/cpp/schema-metadata.md
+++ b/docs/object-serialization/cpp/schema-metadata.md
@@ -59,8 +59,9 @@ FORY_STRUCT(DataV2, id, (timestamp, fory::F().tagged()),
version);
FORY_STRUCT(Counter, FORY_PROPERTY(value, fory::F().varint()));
```
-`fory::F(id)` uses explicit id-based field identity. IDs must be
-non-negative:
+`fory::F(id)` uses explicit id-based field identity. Configured IDs must be
+unique within the complete struct schema and satisfy `0 <= id < 2^29` (`0`
+through `536870911`):
```cpp
FORY_STRUCT(DataV2, (id, fory::F(0)), (timestamp, fory::F(1).tagged()),
diff --git a/docs/object-serialization/csharp/schema-metadata.md
b/docs/object-serialization/csharp/schema-metadata.md
index d9858982e..c55567144 100644
--- a/docs/object-serialization/csharp/schema-metadata.md
+++ b/docs/object-serialization/csharp/schema-metadata.md
@@ -23,7 +23,7 @@ This page covers schema metadata for C# generated serializers.
## `[ForyStruct]` and `[ForyField]`
-Use `[ForyStruct]` to enable source-generated serializers. Use `[ForyField]`
to assign an optional stable non-negative field id or to override the Fory
schema type used for a field.
+Use `[ForyStruct]` to enable source-generated serializers. Use `[ForyField]`
to assign an optional stable field ID or to override the Fory schema type used
for a field. Configured IDs must be unique within the complete struct schema
and satisfy `0 <= id < 2^29` (`0` through `536870911`).
External-type serialization puts `Target` on a local abstract serializer
declaration. Its properties own the field names, IDs, schema descriptors,
@@ -77,6 +77,7 @@ public sealed class Metrics
```
`Id` is optional. When it is omitted, compatible mode still matches the field
by name.
+Once assigned, keep an ID stable and do not reuse it for a different field.
```csharp
using Apache.Fory;
diff --git a/docs/object-serialization/dart/schema-metadata.md
b/docs/object-serialization/dart/schema-metadata.md
index 3073cf6bb..ed4a957cc 100644
--- a/docs/object-serialization/dart/schema-metadata.md
+++ b/docs/object-serialization/dart/schema-metadata.md
@@ -69,6 +69,7 @@ String name = '';
```
Once a payload is shared across services, never reuse an `id` for a different
field.
+Configured IDs must satisfy `0 <= id < 2^29` (`0` through `536870911`).
An ordinary child has one flattened field namespace. IDs must therefore be
unique across all fields included from its child, superclass, and
diff --git a/docs/object-serialization/go/schema-metadata.md
b/docs/object-serialization/go/schema-metadata.md
index 878661dab..7073e1e24 100644
--- a/docs/object-serialization/go/schema-metadata.md
+++ b/docs/object-serialization/go/schema-metadata.md
@@ -47,6 +47,10 @@ type User struct {
}
```
+Configured IDs must be unique within the struct schema and satisfy
+`0 <= id < 2^29` (`0` through `536870911`). Keep assigned IDs stable and do
+not reuse them for different fields.
+
**Benefits**:
- Smaller serialized size (numeric IDs vs field names)
diff --git a/docs/object-serialization/go/troubleshooting.md
b/docs/object-serialization/go/troubleshooting.md
index 9c51ba6be..be7fa1e9b 100644
--- a/docs/object-serialization/go/troubleshooting.md
+++ b/docs/object-serialization/go/troubleshooting.md
@@ -206,7 +206,8 @@ f2 := fory.New(fory.WithTrackRef(true)) // Must match!
**Common causes**:
-1. **Invalid tag ID**: ID must be non-negative
+1. **Invalid tag ID**: ID must satisfy `0 <= id < 2^29` (`0` through
+ `536870911`)
```go
// Wrong: negative ID
@@ -218,6 +219,11 @@ type Bad struct {
type Good struct {
Field int `fory:"id=0"`
}
+
+// Wrong: ID reaches the exclusive upper bound
+type TooLarge struct {
+ Field int `fory:"id=536870912"`
+}
```
2. **Duplicate tag IDs**: Each field must have a unique ID within the struct
@@ -256,7 +262,7 @@ type User struct {
}
```
-2. **Use field IDs for consistent ordering**: Field IDs (non-negative
integers) act as aliases for field names, used for both sorting and field
matching during deserialization:
+2. **Use field IDs for consistent ordering**: Field IDs in the protocol range
act as aliases for field names, used for both sorting and field matching during
deserialization:
```go
type User struct {
diff --git a/docs/object-serialization/java/schema-metadata.md
b/docs/object-serialization/java/schema-metadata.md
index 981e47184..8a1265d84 100644
--- a/docs/object-serialization/java/schema-metadata.md
+++ b/docs/object-serialization/java/schema-metadata.md
@@ -87,10 +87,10 @@ public class User {
### Parameters
-| Parameter | Type | Default | Description |
-| --------- | --------- | ------- | -------------------------------------- |
-| `id` | `int` | `-1` | Non-negative field tag ID, or no ID |
-| `dynamic` | `Dynamic` | `AUTO` | Control polymorphism for struct fields |
+| Parameter | Type | Default | Description
|
+| --------- | --------- | ------- |
-------------------------------------------- |
+| `id` | `int` | `-1` | Field tag ID, or the internal no-ID
sentinel |
+| `dynamic` | `Dynamic` | `AUTO` | Control polymorphism for struct fields
|
Use `@Nullable` on the field type or nested type position for nullable schema
metadata and `@Ref` for reference tracking. `@ForyField` does not carry either
@@ -123,10 +123,11 @@ public class User {
**Notes**:
-- IDs must be unique within a class
-- IDs must be >= 0 when configured
-- If not specified, the annotation default `-1` is ignored and field name is
used in metadata
+- Configured IDs must satisfy `0 <= id < 2^29` (`0` through `536870911`)
+- IDs must be unique within the complete struct schema, including inherited
fields
+- If not specified, the annotation default `-1` is the internal no-ID sentinel
and the field name is used in metadata
(larger overhead)
+- Once assigned, keep an ID stable and do not reuse it for a different field
**Without field IDs** (field names used in metadata):
diff --git a/docs/object-serialization/javascript/schema-metadata.md
b/docs/object-serialization/javascript/schema-metadata.md
index c891701b6..688479597 100644
--- a/docs/object-serialization/javascript/schema-metadata.md
+++ b/docs/object-serialization/javascript/schema-metadata.md
@@ -50,6 +50,24 @@ const byName = Type.struct(
Use `.` inside `typeName` to add a namespace prefix.
+## Field IDs
+
+Call `setId(id)` on a field type to assign a stable numeric field identity:
+
+```ts
+const userType = Type.struct(
+ { typeId: 1001 },
+ {
+ id: Type.int64().setId(0),
+ name: Type.string().setId(1),
+ },
+);
+```
+
+Configured IDs must be unique within the struct schema and satisfy
+`0 <= id < 2^29` (`0` through `536870911`). Keep assigned IDs stable and do
+not reuse them for different fields. Fields without an ID use their names.
+
## Decorator Metadata
Decorators keep the schema next to a TypeScript class declaration:
diff --git a/docs/object-serialization/kotlin/schema-metadata.md
b/docs/object-serialization/kotlin/schema-metadata.md
index b46bd14f9..a7fb57abd 100644
--- a/docs/object-serialization/kotlin/schema-metadata.md
+++ b/docs/object-serialization/kotlin/schema-metadata.md
@@ -51,6 +51,10 @@ Use `@ForyField(id = 1)` on constructor properties.
`@field:ForyField(id = 1)` i
for field-backed properties. Do not use `@get:ForyField` or `@set:ForyField`;
accessors are not
schema fields and the processor rejects them.
+Configured field IDs must be unique within the struct schema and satisfy
+`0 <= id < 2^29` (`0` through `536870911`). Keep assigned IDs stable and do
+not reuse them for different fields. If `id` is omitted, the field name is
used.
+
## Nullability
Use Kotlin `?` to describe nullable schema positions. Nullability is preserved
inside collections
diff --git a/docs/object-serialization/python/schema-metadata.md
b/docs/object-serialization/python/schema-metadata.md
index e7359c96a..cd8517dba 100644
--- a/docs/object-serialization/python/schema-metadata.md
+++ b/docs/object-serialization/python/schema-metadata.md
@@ -72,7 +72,7 @@ class User:
| Parameter | Type | Default | Description
|
| ----------------- | -------- | --------- |
------------------------------------ |
-| `id` | `int` | omitted | Non-negative field tag ID
|
+| `id` | `int` | omitted | Field tag ID in `0 <= id < 2^29`
|
| `nullable` | `bool` | `False` | Whether the field can be null
|
| `ref` | `bool` | `False` | Enable reference tracking
|
| `ignore` | `bool` | `False` | Exclude field from serialization
|
@@ -102,9 +102,10 @@ class User:
**Notes**:
-- IDs must be unique within a class
-- IDs must be >= 0
+- IDs must be unique within the struct schema
+- IDs must satisfy `0 <= id < 2^29` (`0` through `536870911`)
- If not specified, field name is used in metadata (larger overhead)
+- Once assigned, keep an ID stable and do not reuse it for a different field
**Without field IDs** (field names used in metadata):
diff --git a/docs/object-serialization/rust/schema-metadata.md
b/docs/object-serialization/rust/schema-metadata.md
index 249f9a906..4b18c04d7 100644
--- a/docs/object-serialization/rust/schema-metadata.md
+++ b/docs/object-serialization/rust/schema-metadata.md
@@ -83,8 +83,9 @@ struct User {
**Notes**:
- IDs must be unique within a struct
-- IDs must be non-negative
+- IDs must satisfy `0 <= id < 2^29` (`0` through `536870911`)
- If not specified, field name is used in metadata (larger overhead)
+- Once assigned, keep an ID stable and do not reuse it for a different field
### Skipping Fields (`skip`)
diff --git a/docs/object-serialization/scala/schema-metadata.md
b/docs/object-serialization/scala/schema-metadata.md
index d63069082..151c8cdb6 100644
--- a/docs/object-serialization/scala/schema-metadata.md
+++ b/docs/object-serialization/scala/schema-metadata.md
@@ -38,6 +38,10 @@ final case class Person(
) derives ForySerializer
```
+Configured field IDs must be unique within the struct schema and satisfy
+`0 <= id < 2^29` (`0` through `536870911`). Keep assigned IDs stable and do
+not reuse them for different fields. If `id` is omitted, the field name is
used.
+
Schema `optional T` fields are represented as `Option[T]`.
## Reference Tracking
diff --git a/docs/object-serialization/swift/schema-metadata.md
b/docs/object-serialization/swift/schema-metadata.md
index e80521000..2a216e4f7 100644
--- a/docs/object-serialization/swift/schema-metadata.md
+++ b/docs/object-serialization/swift/schema-metadata.md
@@ -26,6 +26,7 @@ This page covers macro-level schema metadata in Swift.
- `@ForyStruct` on struct/class models and external structural serializers
- `@ForyEnum` on C-style enum models and external enum serializers
- `@ForyUnion` and `@ForyCase` on associated-value enum models and external
union serializers
+- `@ForyField(id: ...)` for stable numeric field identity
- `@ForyField(encoding: ...)` on numeric fields
- `@ForyField(with: ...)` for exact serializer selection
- `@ListField`, `@ArrayField`, `@SetField`, and `@MapField` for collection
field metadata
@@ -47,6 +48,23 @@ Equivalent target arguments are available on `@ForyEnum` and
`@ForyUnion`.
See [External-Type Serialization](external-types.md) for target access and
construction requirements.
+## `@ForyField(id:)`
+
+Assign an ID when field identity must remain stable across renames or schema
+evolution:
+
+```swift
+@ForyStruct
+struct User {
+ @ForyField(id: 1)
+ var name: String
+}
+```
+
+Configured IDs must be unique within the struct schema and satisfy
+`0 <= id < 2^29` (`0` through `536870911`). Keep assigned IDs stable and do
+not reuse them for different fields. Fields without an ID use their names.
+
## `@ForyField(with:)`
An unannotated field implicitly selects its declared type when that type
diff --git a/docs/specification/java_serialization_spec.md
b/docs/specification/java_serialization_spec.md
index b2802f2a7..012d27f06 100644
--- a/docs/specification/java_serialization_spec.md
+++ b/docs/specification/java_serialization_spec.md
@@ -317,12 +317,12 @@ participate in field order.
Field identifiers are selected as follows:
-- If a field has an explicit non-negative `@ForyField(id = ...)`, that numeric
- ID is the field identifier.
-- Otherwise, the Java field name converted to snake_case is the field
- identifier.
-- Negative annotation values are not valid field IDs. The annotation default
- value `-1` means no explicit ID and is ignored for identifier selection.
+- If a field has an explicit `@ForyField(id = ...)` in the range
+ `0 <= id < 2^29`, that numeric ID is the field identifier.
+- The annotation default `id = -1`, including a field without an explicit
+ `@ForyField` ID, uses the Java field name converted to snake_case.
+- Every other annotation ID is invalid; values below `-1` and values greater
+ than or equal to `2^29` do not select name-based identity.
Identifier comparison is:
diff --git a/docs/specification/xlang_serialization_spec.md
b/docs/specification/xlang_serialization_spec.md
index fa77b9e1e..83f579a65 100644
--- a/docs/specification/xlang_serialization_spec.md
+++ b/docs/specification/xlang_serialization_spec.md
@@ -840,9 +840,13 @@ Field info list:
Each field is encoded as:
```
-| field header (1 byte) | field type info | [field name bytes] |
+| field header (1 byte) | [extended name or tag value] | field type info |
[field name bytes] |
```
+The optional extended value is a `varuint32`. It is present when the four-bit
+size field is saturated for a long encoded name or an extended tag ID, as
+defined below.
+
Field header layout:
- Bits 6-7: field name encoding (`UTF8`, `ALL_TO_LOWER_SPECIAL`,
diff --git a/go/fory/doc.go b/go/fory/doc.go
index af53747af..04e3646ad 100644
--- a/go/fory/doc.go
+++ b/go/fory/doc.go
@@ -152,7 +152,7 @@ Fory supports struct tags for field-level configuration:
}
Available tags:
- - id=N: Assign numeric field ID for compact encoding
+ - id=N: Assign a field ID in 0 <= N < 2^29 for compact encoding
- "-": Exclude field from serialization
- ref: Enable reference tracking for pointer/slice/map fields
- nullable: Write null flag for pointer fields
diff --git a/go/fory/tag.go b/go/fory/tag.go
index f5d2dc600..97d862331 100644
--- a/go/fory/tag.go
+++ b/go/fory/tag.go
@@ -53,6 +53,9 @@ func validateForyTags(t reflect.Type) error {
tagIDs := make(map[int32]string)
for i := 0; i < t.NumField(); i++ {
field := t.Field(i)
+ if field.PkgPath != "" {
+ continue
+ }
parsed, err := parseFieldTag(field)
if err != nil {
return err
diff --git a/go/fory/tag_test.go b/go/fory/tag_test.go
index 906eebd36..ece5ecbfa 100644
--- a/go/fory/tag_test.go
+++ b/go/fory/tag_test.go
@@ -289,7 +289,7 @@ func TestParseFieldSpecRejectsInvalidTags(t *testing.T) {
func TestValidateForyTagsStrict(t *testing.T) {
type Valid struct {
Field1 string `fory:"id=0"`
- Field2 []int32
`fory:"id=1,type=list(element=int32(encoding=fixed))"`
+ Field2 []int32
`fory:"id=536870911,type=list(element=int32(encoding=fixed))"`
}
type DuplicateIDs struct {
Field1 string `fory:"id=0"`
@@ -302,13 +302,30 @@ func TestValidateForyTagsStrict(t *testing.T) {
type InvalidID struct {
Field1 string `fory:"id=-1"`
}
+ type OversizedID struct {
+ Field1 string `fory:"id=536870912"`
+ }
+ type UnexportedInvalid struct {
+ Field1 string
+ hidden string `fory:"id=invalid"`
+ }
+
+ require.NoError(t, New().RegisterStruct(Valid{}, 100))
+ require.NoError(t, New().RegisterStructByName(IgnoredDuplicate{},
"test.IgnoredDuplicate"))
+ require.NoError(t, New().RegisterStructByName(UnexportedInvalid{},
"test.UnexportedInvalid"))
+
+ byID := New()
+ require.Error(t, byID.RegisterStruct(DuplicateIDs{}, 101))
+ require.NoError(t, byID.RegisterStruct(Valid{}, 101))
+
+ byName := New()
+ require.Error(t, byName.RegisterStructByName(DuplicateIDs{},
"test.Shared"))
+ require.NoError(t, byName.RegisterStructByName(Valid{}, "test.Shared"))
- require.NoError(t, validateForyTags(reflect.TypeOf(Valid{})))
- require.NoError(t, validateForyTags(reflect.TypeOf(IgnoredDuplicate{})))
- require.Error(t, validateForyTags(reflect.TypeOf(DuplicateIDs{})))
- err := validateForyTags(reflect.TypeOf(InvalidID{}))
+ err := New().RegisterStructByName(InvalidID{}, "test.InvalidID")
require.Error(t, err)
require.Contains(t, err.Error(), "id must be non-negative")
+ require.Error(t, New().RegisterStruct(OversizedID{}, 102))
}
func TestShouldIncludeField(t *testing.T) {
diff --git a/go/fory/tests/xlang/xlang_test_main.go
b/go/fory/tests/xlang/xlang_test_main.go
index 1a57fac0b..a21ea3dbd 100644
--- a/go/fory/tests/xlang/xlang_test_main.go
+++ b/go/fory/tests/xlang/xlang_test_main.go
@@ -300,6 +300,12 @@ type SimpleStruct struct {
Last int32 `fory:"id=536870911"`
}
+type ExactFieldTags struct {
+ First int32 `fory:"id=15"`
+ Second int32 `fory:"id=65551"`
+ Last int32 `fory:"id=536870911"`
+}
+
type EvolvingOverrideStruct struct {
F1 string
}
@@ -764,6 +770,26 @@ func testSimpleStruct() {
writeFile(dataFile, serialized)
}
+func testExactFieldTags() {
+ dataFile := getDataFile()
+ f := fory.New(fory.WithXlang(true), fory.WithCompatible(false))
+ if err := f.RegisterStruct(ExactFieldTags{}, 104); err != nil {
+ panic(fmt.Sprintf("Failed to register ExactFieldTags: %v", err))
+ }
+ var value ExactFieldTags
+ if err := f.Deserialize(readFile(dataFile), &value); err != nil {
+ panic(fmt.Sprintf("Failed to deserialize ExactFieldTags: %v",
err))
+ }
+ if value != (ExactFieldTags{First: 39, Second: 40, Last: 41}) {
+ panic(fmt.Sprintf("Unexpected ExactFieldTags: %+v", value))
+ }
+ serialized, err := f.Serialize(&value)
+ if err != nil {
+ panic(fmt.Sprintf("Failed to serialize ExactFieldTags: %v",
err))
+ }
+ writeFile(dataFile, serialized)
+}
+
func testNamedSimpleStruct() {
dataFile := getDataFile()
data := readFile(dataFile)
@@ -2902,6 +2928,8 @@ func main() {
testCrossLanguageSerializer()
case "test_simple_struct":
testSimpleStruct()
+ case "test_exact_field_tags":
+ testExactFieldTags()
case "test_named_simple_struct":
testNamedSimpleStruct()
case "test_struct_evolving_override":
diff --git a/go/fory/type_resolver.go b/go/fory/type_resolver.go
index c1a7c202e..ed82c94d3 100644
--- a/go/fory/type_resolver.go
+++ b/go/fory/type_resolver.go
@@ -506,6 +506,9 @@ func (r *TypeResolver) RegisterStruct(type_ reflect.Type,
typeID TypeId, userTyp
switch type_.Kind() {
case reflect.Struct:
+ if err := validateForyTags(type_); err != nil {
+ return err
+ }
if err := validateOptionalFields(type_); err != nil {
return err
}
@@ -671,6 +674,9 @@ func (r *TypeResolver) registerStructByName(type_
reflect.Type, namespace, typeN
if typeName == "" {
return fmt.Errorf("typeName must be non-empty")
}
+ if err := validateForyTags(type_); err != nil {
+ return err
+ }
tag := joinRegisteredName(namespace, typeName)
serializer := newStructSerializer(type_, tag)
r.typeToSerializers[type_] = serializer
diff --git
a/java/fory-core/src/main/java/org/apache/fory/meta/NativeTypeDefDecoder.java
b/java/fory-core/src/main/java/org/apache/fory/meta/NativeTypeDefDecoder.java
index b7e053fa6..b69c42af2 100644
---
a/java/fory-core/src/main/java/org/apache/fory/meta/NativeTypeDefDecoder.java
+++
b/java/fory-core/src/main/java/org/apache/fory/meta/NativeTypeDefDecoder.java
@@ -224,6 +224,7 @@ class NativeTypeDefDecoder {
List<FieldInfo> fieldInfos = readFieldsInfo(typeDefBuf, resolver,
className, numFields);
classFields.addAll(fieldInfos);
}
+ validateTagIds(classFields);
Preconditions.checkNotNull(classSpec);
boolean hasFieldMetadata = !classFields.isEmpty();
// Native TypeDef can carry class-layer fields even when the root wire
type is an enum,
@@ -327,7 +328,6 @@ class NativeTypeDefDecoder {
private static List<FieldInfo> readFieldsInfo(
MemoryBuffer buffer, ClassResolver resolver, String className, int
numFields) {
List<FieldInfo> fieldInfos = new ArrayList<>();
- Set<Integer> tagIds = null;
for (int i = 0; i < numFields; i++) {
int header = buffer.readByte() & 0xff;
// `3 bits size + 2 bits field name encoding + nullability flag + ref
tracking flag`
@@ -357,12 +357,6 @@ class NativeTypeDefDecoder {
// Read field name or tag ID
String fieldName;
if (useTagID) {
- if (tagIds == null) {
- tagIds = new HashSet<>();
- }
- if (!tagIds.add(tagId)) {
- throw new DeserializationException("Duplicate TypeDef field tag ID "
+ tagId);
- }
// Use placeholder field name since tag ID is used for identification
fieldName = "$tag" + tagId;
} else {
@@ -386,6 +380,21 @@ class NativeTypeDefDecoder {
return fieldInfos;
}
+ private static void validateTagIds(List<FieldInfo> fields) {
+ Set<Integer> tagIds = null;
+ for (FieldInfo field : fields) {
+ if (field.hasFieldId()) {
+ if (tagIds == null) {
+ tagIds = new HashSet<>();
+ }
+ if (!tagIds.add(field.getFieldId())) {
+ throw new DeserializationException(
+ "Duplicate TypeDef field tag ID " + field.getFieldId());
+ }
+ }
+ }
+ }
+
static String readPkgName(MemoryBuffer buffer) {
// - Package name encoding(omitted when class is registered):
// - encoding algorithm:
`UTF_8/ALL_TO_LOWER_SPECIAL/LOWER_UPPER_DIGIT_SPECIAL`
diff --git
a/java/fory-core/src/test/java/org/apache/fory/meta/NativeTypeDefEncoderTest.java
b/java/fory-core/src/test/java/org/apache/fory/meta/NativeTypeDefEncoderTest.java
index 8da611b6c..b772cf0b4 100644
---
a/java/fory-core/src/test/java/org/apache/fory/meta/NativeTypeDefEncoderTest.java
+++
b/java/fory-core/src/test/java/org/apache/fory/meta/NativeTypeDefEncoderTest.java
@@ -518,6 +518,52 @@ public class NativeTypeDefEncoderTest {
RuntimeException.class, () ->
TypeDef.readTypeDef(fory.getTypeResolver(), encoded));
}
+ @Test
+ public void testDecodeRejectsInheritedDuplicateTag() {
+ Fory fory =
+ Fory.builder()
+ .withXlang(false)
+ .withMetaShare(true)
+ .withCompatible(false)
+ .requireClassRegistration(false)
+ .build();
+ TypeDef typeDef = TypeDef.buildTypeDef(fory.getTypeResolver(),
TaggedChild.class);
+ byte[] encoded = typeDef.getEncoded();
+ MemoryBuffer encodedBuffer = MemoryBuffer.fromByteArray(encoded);
+ long header = encodedBuffer.readInt64();
+ byte[] body =
+ NativeTypeDefDecoder.decodeTypeDefBuf(
+ encodedBuffer, (ClassResolver) fory.getTypeResolver(), header)
+ .f0;
+
+ List<FieldInfo> fields =
+ buildFieldsInfo((ClassResolver) fory.getTypeResolver(),
TaggedChild.class);
+ FieldInfo parentField =
+ fields.stream()
+ .filter(field ->
field.getDefinedClass().equals(TaggedParent.class.getName()))
+ .findFirst()
+ .orElseThrow(AssertionError::new);
+ FieldInfo childField =
+ fields.stream()
+ .filter(field ->
field.getDefinedClass().equals(TaggedChild.class.getName()))
+ .findFirst()
+ .orElseThrow(AssertionError::new);
+ byte[] parentFieldBytes = encodedFieldInfo(parentField);
+ byte[] childFieldBytes = encodedFieldInfo(childField);
+ Assert.assertEquals(childFieldBytes.length, parentFieldBytes.length);
+
+ int childFieldOffset = indexOf(body, childFieldBytes, 0);
+ Assert.assertTrue(childFieldOffset >= 0);
+ System.arraycopy(parentFieldBytes, 0, body, childFieldOffset,
parentFieldBytes.length);
+ MemoryBuffer malformedBody = MemoryBuffer.newHeapBuffer(body.length);
+ malformedBody.writeBytes(body);
+ MemoryBuffer malformed = NativeTypeDefEncoder.prependHeader(malformedBody,
false);
+
+ Assert.assertThrows(
+ DeserializationException.class,
+ () -> TypeDef.readTypeDef(fory.getTypeResolver(), malformed));
+ }
+
@Test
public void testRejectsNamespaceEncoding() {
Fory fory =
@@ -601,6 +647,12 @@ public class NativeTypeDefEncoderTest {
return malformed;
}
+ private static byte[] encodedFieldInfo(FieldInfo fieldInfo) {
+ MemoryBuffer buffer = MemoryBuffer.newHeapBuffer(16);
+ NativeTypeDefEncoder.writeFieldsInfo(buffer,
Collections.singletonList(fieldInfo));
+ return buffer.getBytes(0, buffer.writerIndex());
+ }
+
private static byte[] rewriteHeaderWithBodyOnlyHash(TypeDef typeDef) {
byte[] malformed = typeDef.getEncoded().clone();
MemoryBuffer buffer = MemoryBuffer.fromByteArray(malformed);
@@ -782,6 +834,16 @@ public class NativeTypeDefEncoderTest {
private String fieldMax;
}
+ public static class TaggedParent {
+ @ForyField(id = 10)
+ private int parentValue;
+ }
+
+ public static class TaggedChild extends TaggedParent {
+ @ForyField(id = 20)
+ private int childValue;
+ }
+
@Data
public static class ClassWithMixedFields {
@ForyField(id = 15)
diff --git
a/java/fory-core/src/test/java/org/apache/fory/xlang/CPPXlangTest.java
b/java/fory-core/src/test/java/org/apache/fory/xlang/CPPXlangTest.java
index 050fd5d4a..721bda8cb 100644
--- a/java/fory-core/src/test/java/org/apache/fory/xlang/CPPXlangTest.java
+++ b/java/fory-core/src/test/java/org/apache/fory/xlang/CPPXlangTest.java
@@ -160,6 +160,12 @@ public class CPPXlangTest extends XlangTestBase {
super.testCrossLanguageSerializer();
}
+ @Override
+ @Test(groups = "xlang", dataProvider = "enableCodegenParallel")
+ public void testExactFieldTags(boolean enableCodegen) throws
java.io.IOException {
+ super.testExactFieldTags(enableCodegen);
+ }
+
@Test(groups = "xlang", dataProvider = "enableCodegenParallel")
public void testSimpleStruct(boolean enableCodegen) throws
java.io.IOException {
super.testSimpleStruct(enableCodegen);
diff --git
a/java/fory-core/src/test/java/org/apache/fory/xlang/CSharpXlangTest.java
b/java/fory-core/src/test/java/org/apache/fory/xlang/CSharpXlangTest.java
index a79167c77..2f971d7b0 100644
--- a/java/fory-core/src/test/java/org/apache/fory/xlang/CSharpXlangTest.java
+++ b/java/fory-core/src/test/java/org/apache/fory/xlang/CSharpXlangTest.java
@@ -155,6 +155,12 @@ public class CSharpXlangTest extends XlangTestBase {
super.testCrossLanguageSerializer();
}
+ @Override
+ @Test(groups = "xlang", dataProvider = "enableCodegenParallel")
+ public void testExactFieldTags(boolean enableCodegen) throws
java.io.IOException {
+ super.testExactFieldTags(enableCodegen);
+ }
+
@Test(groups = "xlang", dataProvider = "enableCodegenParallel")
public void testSimpleStruct(boolean enableCodegen) throws
java.io.IOException {
super.testSimpleStruct(enableCodegen);
diff --git
a/java/fory-core/src/test/java/org/apache/fory/xlang/DartXlangTest.java
b/java/fory-core/src/test/java/org/apache/fory/xlang/DartXlangTest.java
index f484a46da..959884e61 100644
--- a/java/fory-core/src/test/java/org/apache/fory/xlang/DartXlangTest.java
+++ b/java/fory-core/src/test/java/org/apache/fory/xlang/DartXlangTest.java
@@ -245,6 +245,12 @@ public class DartXlangTest extends XlangTestBase {
super.testCrossLanguageSerializer();
}
+ @Override
+ @Test(groups = "xlang", dataProvider = "enableCodegenParallel")
+ public void testExactFieldTags(boolean enableCodegen) throws
java.io.IOException {
+ super.testExactFieldTags(enableCodegen);
+ }
+
@Test(groups = "xlang", dataProvider = "enableCodegenParallel")
public void testSimpleStruct(boolean enableCodegen) throws
java.io.IOException {
super.testSimpleStruct(enableCodegen);
diff --git
a/java/fory-core/src/test/java/org/apache/fory/xlang/GoXlangTest.java
b/java/fory-core/src/test/java/org/apache/fory/xlang/GoXlangTest.java
index 98f96d926..f40f6c3d4 100644
--- a/java/fory-core/src/test/java/org/apache/fory/xlang/GoXlangTest.java
+++ b/java/fory-core/src/test/java/org/apache/fory/xlang/GoXlangTest.java
@@ -124,6 +124,12 @@ public class GoXlangTest extends XlangTestBase {
super.testCrossLanguageSerializer();
}
+ @Override
+ @Test(groups = "xlang", dataProvider = "enableCodegenParallel")
+ public void testExactFieldTags(boolean enableCodegen) throws
java.io.IOException {
+ super.testExactFieldTags(enableCodegen);
+ }
+
@Test(groups = "xlang", dataProvider = "enableCodegenParallel")
public void testSimpleStruct(boolean enableCodegen) throws
java.io.IOException {
super.testSimpleStruct(enableCodegen);
diff --git
a/java/fory-core/src/test/java/org/apache/fory/xlang/JavaScriptXlangTest.java
b/java/fory-core/src/test/java/org/apache/fory/xlang/JavaScriptXlangTest.java
index 576bb8da0..4dd06588d 100644
---
a/java/fory-core/src/test/java/org/apache/fory/xlang/JavaScriptXlangTest.java
+++
b/java/fory-core/src/test/java/org/apache/fory/xlang/JavaScriptXlangTest.java
@@ -282,6 +282,12 @@ public class JavaScriptXlangTest extends XlangTestBase {
super.testCrossLanguageSerializer();
}
+ @Override
+ @Test(groups = "xlang", dataProvider = "enableCodegenParallel")
+ public void testExactFieldTags(boolean enableCodegen) throws
java.io.IOException {
+ super.testExactFieldTags(enableCodegen);
+ }
+
@Test(groups = "xlang", dataProvider = "enableCodegenParallel")
public void testSimpleStruct(boolean enableCodegen) throws
java.io.IOException {
super.testSimpleStruct(enableCodegen);
diff --git
a/java/fory-core/src/test/java/org/apache/fory/xlang/PythonXlangTest.java
b/java/fory-core/src/test/java/org/apache/fory/xlang/PythonXlangTest.java
index b1bca830b..4d2ba3935 100644
--- a/java/fory-core/src/test/java/org/apache/fory/xlang/PythonXlangTest.java
+++ b/java/fory-core/src/test/java/org/apache/fory/xlang/PythonXlangTest.java
@@ -154,6 +154,12 @@ public class PythonXlangTest extends XlangTestBase {
super.testStringSerializer();
}
+ @Override
+ @Test(groups = "xlang", dataProvider = "enableCodegenParallel")
+ public void testExactFieldTags(boolean enableCodegen) throws IOException {
+ super.testExactFieldTags(enableCodegen);
+ }
+
@Override
@Test(groups = "xlang", dataProvider = "enableCodegenParallel")
public void testSimpleStruct(boolean enableCodegen) throws IOException {
diff --git
a/java/fory-core/src/test/java/org/apache/fory/xlang/RustXlangTest.java
b/java/fory-core/src/test/java/org/apache/fory/xlang/RustXlangTest.java
index bad2cd90e..43be8356f 100644
--- a/java/fory-core/src/test/java/org/apache/fory/xlang/RustXlangTest.java
+++ b/java/fory-core/src/test/java/org/apache/fory/xlang/RustXlangTest.java
@@ -130,6 +130,12 @@ public class RustXlangTest extends XlangTestBase {
super.testCrossLanguageSerializer();
}
+ @Override
+ @Test(groups = "xlang", dataProvider = "enableCodegenParallel")
+ public void testExactFieldTags(boolean enableCodegen) throws
java.io.IOException {
+ super.testExactFieldTags(enableCodegen);
+ }
+
@Test(groups = "xlang", dataProvider = "enableCodegenParallel")
public void testSimpleStruct(boolean enableCodegen) throws
java.io.IOException {
super.testSimpleStruct(enableCodegen);
diff --git
a/java/fory-core/src/test/java/org/apache/fory/xlang/ScalaXlangTest.java
b/java/fory-core/src/test/java/org/apache/fory/xlang/ScalaXlangTest.java
index 67aa5c1de..d62c858f0 100644
--- a/java/fory-core/src/test/java/org/apache/fory/xlang/ScalaXlangTest.java
+++ b/java/fory-core/src/test/java/org/apache/fory/xlang/ScalaXlangTest.java
@@ -239,6 +239,12 @@ public class ScalaXlangTest extends XlangTestBase {
super.testCrossLanguageSerializer();
}
+ @Override
+ @Test(groups = "xlang", dataProvider = "enableCodegenParallel")
+ public void testExactFieldTags(boolean enableCodegen) throws IOException {
+ super.testExactFieldTags(enableCodegen);
+ }
+
@Test(groups = "xlang", dataProvider = "enableCodegenParallel")
public void testSimpleStruct(boolean enableCodegen) throws IOException {
super.testSimpleStruct(enableCodegen);
diff --git
a/java/fory-core/src/test/java/org/apache/fory/xlang/SwiftXlangTest.java
b/java/fory-core/src/test/java/org/apache/fory/xlang/SwiftXlangTest.java
index 5f64af5a6..e29c2b851 100644
--- a/java/fory-core/src/test/java/org/apache/fory/xlang/SwiftXlangTest.java
+++ b/java/fory-core/src/test/java/org/apache/fory/xlang/SwiftXlangTest.java
@@ -207,6 +207,12 @@ public class SwiftXlangTest extends XlangTestBase {
super.testCrossLanguageSerializer();
}
+ @Override
+ @Test(groups = "xlang", dataProvider = "enableCodegenParallel")
+ public void testExactFieldTags(boolean enableCodegen) throws
java.io.IOException {
+ super.testExactFieldTags(enableCodegen);
+ }
+
@Test(groups = "xlang", dataProvider = "enableCodegenParallel")
public void testSimpleStruct(boolean enableCodegen) throws
java.io.IOException {
super.testSimpleStruct(enableCodegen);
diff --git
a/java/fory-core/src/test/java/org/apache/fory/xlang/XlangTestBase.java
b/java/fory-core/src/test/java/org/apache/fory/xlang/XlangTestBase.java
index cbd0a41d1..fff9c46fe 100644
--- a/java/fory-core/src/test/java/org/apache/fory/xlang/XlangTestBase.java
+++ b/java/fory-core/src/test/java/org/apache/fory/xlang/XlangTestBase.java
@@ -654,6 +654,40 @@ public abstract class XlangTestBase extends ForyTestBase {
int last; // Changed from Integer to int to match Rust
}
+ @Data
+ @ForyStruct
+ static class ExactFieldTags {
+ @ForyField(id = 15)
+ int first;
+
+ @ForyField(id = 65551)
+ int second;
+
+ @ForyField(id = 536870911)
+ int last;
+ }
+
+ @Test(
+ groups = {"xlang", "field-tag"},
+ dataProvider = "enableCodegenParallel")
+ public void testExactFieldTags(boolean enableCodegen) throws IOException {
+ Fory fory =
+
Fory.builder().withXlang(true).withCompatible(false).withCodegen(enableCodegen).build();
+ fory.register(ExactFieldTags.class, 104);
+ ExactFieldTags value = new ExactFieldTags();
+ value.first = 39;
+ value.second = 40;
+ value.last = 41;
+ serDeCheck(fory, value);
+
+ MemoryBuffer buffer = MemoryUtils.buffer(64);
+ fory.serialize(buffer, value);
+ ExecutionContext ctx =
+ prepareExecution("test_exact_field_tags", buffer.getBytes(0,
buffer.writerIndex()));
+ runPeer(ctx);
+ Assert.assertEquals(fory.deserialize(readBuffer(ctx.dataFile())), value);
+ }
+
@Test(groups = "xlang", dataProvider = "enableCodegenParallel")
public void testSimpleStruct(boolean enableCodegen) throws
java.io.IOException {
String caseName = "test_simple_struct";
diff --git a/javascript/test/crossLanguage.test.ts
b/javascript/test/crossLanguage.test.ts
index 32f7729b5..6b700c7e9 100644
--- a/javascript/test/crossLanguage.test.ts
+++ b/javascript/test/crossLanguage.test.ts
@@ -364,6 +364,27 @@ describe("bool", () => {
writeToFile(serializedData as Buffer);
});
+ test("test_exact_field_tags", () => {
+ const fory = new Fory({
+ compatible: false,
+ });
+
+ @Type.struct(104, {
+ first: Type.int32().setId(15),
+ second: Type.int32().setId(65551),
+ last: Type.int32().setId(536870911),
+ })
+ class ExactFieldTags {
+ first: number = 0;
+ second: number = 0;
+ last: number = 0;
+ }
+ fory.register(ExactFieldTags);
+
+ const value = fory.deserialize(content) as ExactFieldTags;
+ expect(value).toEqual({ first: 39, second: 40, last: 41 });
+ writeToFile(fory.serialize(value) as Buffer);
+ });
test("test_named_simple_struct", () => {
// Same as test_simple_struct but with named registration
const fory = new Fory({
diff --git
a/kotlin/fory-kotlin-ksp/src/test/kotlin/org/apache/fory/kotlin/ksp/ProcessorValidationTest.kt
b/kotlin/fory-kotlin-ksp/src/test/kotlin/org/apache/fory/kotlin/ksp/ProcessorValidationTest.kt
index 0f5ba639a..8d705bcae 100644
---
a/kotlin/fory-kotlin-ksp/src/test/kotlin/org/apache/fory/kotlin/ksp/ProcessorValidationTest.kt
+++
b/kotlin/fory-kotlin-ksp/src/test/kotlin/org/apache/fory/kotlin/ksp/ProcessorValidationTest.kt
@@ -80,10 +80,8 @@ class ProcessorValidationTest {
}
@Test
- fun validatesFieldIds() {
+ fun rejectsInvalidFieldIds() {
assertNull(fieldIdError(-1, requireFieldId = false))
- assertNull(fieldIdError(0, requireFieldId = true))
- assertNull(fieldIdError(536_870_911, requireFieldId = true))
assertNotNull(fieldIdError(-1, requireFieldId = true))
assertNotNull(fieldIdError(536_870_912, requireFieldId = true))
}
diff --git
a/kotlin/fory-kotlin-tests/src/main/kotlin/org/apache/fory/kotlin/xlang/KotlinXlangPeer.kt
b/kotlin/fory-kotlin-tests/src/main/kotlin/org/apache/fory/kotlin/xlang/KotlinXlangPeer.kt
index 05c7e66b3..92e190411 100644
---
a/kotlin/fory-kotlin-tests/src/main/kotlin/org/apache/fory/kotlin/xlang/KotlinXlangPeer.kt
+++
b/kotlin/fory-kotlin-tests/src/main/kotlin/org/apache/fory/kotlin/xlang/KotlinXlangPeer.kt
@@ -65,6 +65,9 @@ constructor(
@ForyField(id = 3) val score: @VarInt Long,
)
+@ForyStruct
+public data class KotlinMaximumFieldTag constructor(@ForyField(id =
536_870_911) val value: Int)
+
@ForyStruct
internal data class KotlinInternalUser
constructor(
@@ -305,6 +308,7 @@ private fun staticSerializerRoundTrip(dataFile: String) {
val fory = newFory()
fory.register<KotlinUser>("kotlin.KotlinUser")
+ fory.register<KotlinMaximumFieldTag>("kotlin.KotlinMaximumFieldTag")
fory.register<KotlinInternalUser>("kotlin.KotlinInternalUser")
fory.register<KotlinConcreteCollections>("kotlin.KotlinConcreteCollections")
fory.register<KotlinUnsignedCollections>("kotlin.KotlinUnsignedCollections")
@@ -333,6 +337,14 @@ private fun staticSerializerRoundTrip(dataFile: String) {
check(descriptors[0].typeRef.typeExtMeta.typeId() == Types.UINT32)
check(descriptors[2].typeRef.typeExtMeta.typeId() == Types.VARINT64)
+ val maximumTag = KotlinMaximumFieldTag(42)
+ check(
+ fory.deserialize(fory.serialize(maximumTag),
KotlinMaximumFieldTag::class.java) == maximumTag
+ )
+ val maximumTagSerializer =
fory.getSerializer(KotlinMaximumFieldTag::class.java)
+ check(maximumTagSerializer is StaticGeneratedStructSerializer<*>)
+ check(maximumTagSerializer.generatedDescriptors.single().foryFieldId ==
536_870_911)
+
val internalUser = KotlinInternalUser(id = UInt.MAX_VALUE, name =
"internal-static")
check(
fory.deserialize(fory.serialize(internalUser),
KotlinInternalUser::class.java) == internalUser
diff --git a/python/pyfory/field.py b/python/pyfory/field.py
index e723ac548..69357c437 100644
--- a/python/pyfory/field.py
+++ b/python/pyfory/field.py
@@ -51,7 +51,7 @@ class ForyFieldMeta:
Fory field metadata extracted from field.metadata.
Attributes:
- id: Field tag ID. -1 is the internal sentinel for no configured ID;
>=0 means use tag ID.
+ id: Field tag ID in [0, 2^29), or -1 as the internal no-ID sentinel.
nullable: Whether null flag is written. Default False.
ref: Whether reference tracking is enabled for this field. Default
False.
ignore: Whether to ignore this field during serialization. Default
False.
@@ -129,8 +129,9 @@ def field(
Args:
id: Field tag ID (optional).
- omitted: Use field name with meta string encoding
- - >=0: Use numeric tag ID (more compact, stable across renames)
- Must be unique within the class. Negative configured IDs are
invalid.
+ - 0 <= id < 2^29: Use numeric tag ID (more compact, stable across
renames)
+ Must be unique within the class. Negative configured IDs and values
+ greater than or equal to 2^29 are invalid.
nullable: Whether to write null flag for this field.
- False (default): Skip null flag, field cannot be None
diff --git a/python/pyfory/tests/xlang_test_main.py
b/python/pyfory/tests/xlang_test_main.py
index 49d16d2a3..e56f08fe4 100644
--- a/python/pyfory/tests/xlang_test_main.py
+++ b/python/pyfory/tests/xlang_test_main.py
@@ -114,6 +114,13 @@ class SimpleStruct:
last: pyfory.Int32 = pyfory.field(536870911, default=0)
+@dataclass
+class ExactFieldTags:
+ first: pyfory.Int32 = pyfory.field(15, default=0)
+ second: pyfory.Int32 = pyfory.field(65551, default=0)
+ last: pyfory.Int32 = pyfory.field(536870911, default=0)
+
+
@pyfory.dataclass
class EvolvingOverrideStruct:
f1: str = ""
@@ -501,6 +508,20 @@ def test_simple_struct():
f.write(new_bytes)
+def test_exact_field_tags():
+ """Round-trip the complete field-tag boundary set in schema-consistent
mode."""
+ data_file = get_data_file()
+ with open(data_file, "rb") as f:
+ data_bytes = f.read()
+
+ fory = pyfory.Fory(xlang=True, compatible=False)
+ fory.register_type(ExactFieldTags, type_id=104)
+ value = fory.deserialize(data_bytes)
+ assert value == ExactFieldTags(first=39, second=40, last=41)
+ with open(data_file, "wb") as f:
+ f.write(fory.serialize(value))
+
+
def test_named_simple_struct():
"""Test named simple struct serialization."""
data_file = get_data_file()
diff --git a/rust/fory-core/src/meta/type_meta.rs
b/rust/fory-core/src/meta/type_meta.rs
index cbba622c8..f30d7032b 100644
--- a/rust/fory-core/src/meta/type_meta.rs
+++ b/rust/fory-core/src/meta/type_meta.rs
@@ -424,7 +424,7 @@ impl FieldType {
}
}
-#[derive(Debug, PartialEq, Eq, Clone)]
+#[derive(Debug, Clone)]
pub struct FieldInfo {
/// Protocol field tag, or `-1` when the field is identified by name.
pub field_id: i32,
@@ -435,6 +435,17 @@ pub struct FieldInfo {
pub field_type: FieldType,
}
+impl PartialEq for FieldInfo {
+ fn eq(&self, other: &Self) -> bool {
+ // matched_field_id is derived local dispatch state, not protocol
metadata identity.
+ self.field_id == other.field_id
+ && self.field_name == other.field_name
+ && self.field_type == other.field_type
+ }
+}
+
+impl Eq for FieldInfo {}
+
impl FieldInfo {
pub fn new(field_name: &str, field_type: FieldType) -> FieldInfo {
FieldInfo {
@@ -1476,9 +1487,11 @@ mod tests {
let remote_type = FieldType::new(crate::type_id::INT16, false, vec![]);
let local_fields = [FieldInfo::new("value", local_type)];
let mut remote_fields = [FieldInfo::new("value", remote_type)];
+ let wire_field = remote_fields[0].clone();
match_remote_fields(&local_fields, &mut remote_fields).unwrap();
+ assert_eq!(remote_fields[0], wire_field);
assert_eq!(remote_fields[0].field_id, -1);
assert_eq!(remote_fields[0].matched_field_id, 1);
}
diff --git a/rust/tests/tests/test_cross_language.rs
b/rust/tests/tests/test_cross_language.rs
index b94921ba4..45ee5b1d9 100644
--- a/rust/tests/tests/test_cross_language.rs
+++ b/rust/tests/tests/test_cross_language.rs
@@ -72,6 +72,16 @@ struct SimpleStruct {
last: i32,
}
+#[derive(ForyStruct, Debug, PartialEq)]
+struct ExactFieldTags {
+ #[fory(id = 15)]
+ first: i32,
+ #[fory(id = 65551)]
+ second: i32,
+ #[fory(id = 536870911)]
+ last: i32,
+}
+
#[derive(ForyStruct, Debug, PartialEq, Default)]
struct EvolvingOverrideStruct {
f1: String,
@@ -448,6 +458,24 @@ fn test_simple_struct() {
fs::write(&data_file_path, new_bytes).unwrap();
}
+#[test]
+#[ignore]
+fn test_exact_field_tags() {
+ let data_file_path = get_data_file();
+ let bytes = fs::read(&data_file_path).unwrap();
+ let mut fory = Fory::builder().compatible(false).xlang(true).build();
+ fory.register::<ExactFieldTags>(104).unwrap();
+
+ let expected = ExactFieldTags {
+ first: 39,
+ second: 40,
+ last: 41,
+ };
+ let value: ExactFieldTags = fory.deserialize(&bytes).unwrap();
+ assert_eq!(value, expected);
+ fs::write(&data_file_path, fory.serialize(&value).unwrap()).unwrap();
+}
+
#[test]
#[ignore]
fn test_named_simple_struct() {
diff --git
a/scala/fory-scala/src/test/scala-3/org/apache/fory/serializer/scala/ScalaXlangPeer.scala
b/scala/fory-scala/src/test/scala-3/org/apache/fory/serializer/scala/ScalaXlangPeer.scala
index 9b223a66d..0ed3f6a18 100644
---
a/scala/fory-scala/src/test/scala-3/org/apache/fory/serializer/scala/ScalaXlangPeer.scala
+++
b/scala/fory-scala/src/test/scala-3/org/apache/fory/serializer/scala/ScalaXlangPeer.scala
@@ -94,6 +94,13 @@ final case class SimpleStruct(
@ForyField(id = 536870911) last: Int)
derives ForySerializer
+@ForyStruct
+final case class ExactFieldTags(
+ @ForyField(id = 15) first: Int,
+ @ForyField(id = 65551) second: Int,
+ @ForyField(id = 536870911) last: Int)
+ derives ForySerializer
+
@ForyStruct(evolution = Evolution.ENABLED)
final case class EvolvingOverrideStruct(f1: String) derives ForySerializer
@@ -370,6 +377,7 @@ object ScalaXlangPeer {
case "test_string_serializer" => roundTripValues(dataFile, newFory())
case "test_cross_language_serializer" => roundTripValues(dataFile,
crossLanguageFory())
case "test_simple_struct" => roundTripValues(dataFile,
simpleStructFory(false))
+ case "test_exact_field_tags" => roundTripValues(dataFile,
exactFieldTagsFory())
case "test_named_simple_struct" => roundTripValues(dataFile,
simpleStructFory(true))
case "test_struct_evolving_override" => roundTripValues(dataFile,
evolvingOverrideFory())
case "test_list" | "test_map" | "test_item" => roundTripValues(dataFile,
itemFory())
@@ -582,6 +590,12 @@ object ScalaXlangPeer {
fory
}
+ private def exactFieldTagsFory(): Fory = {
+ val fory = newFory(compatible = false)
+ registerStruct(fory, classOf[ExactFieldTags], 104L)
+ fory
+ }
+
private def evolvingOverrideFory(): Fory = {
val fory = newFory()
registerStruct(fory, classOf[EvolvingOverrideStruct], "test",
"evolving_yes")
diff --git a/swift/Tests/ForyXlangTests/main.swift
b/swift/Tests/ForyXlangTests/main.swift
index ccdf521f4..db503d40a 100644
--- a/swift/Tests/ForyXlangTests/main.swift
+++ b/swift/Tests/ForyXlangTests/main.swift
@@ -59,6 +59,16 @@ private struct SimpleStruct {
var last: Int32 = 0
}
+@ForyStruct
+private struct ExactFieldTags {
+ @ForyField(id: 15)
+ var first: Int32 = 0
+ @ForyField(id: 65551)
+ var second: Int32 = 0
+ @ForyField(id: 536870911)
+ var last: Int32 = 0
+}
+
@ForyStruct
private struct EvolvingOverrideStruct {
var f1: String = ""
@@ -695,6 +705,12 @@ private func handleSimpleStruct(_ bytes: [UInt8]) throws
-> [UInt8] {
return try roundTripSingle(bytes, fory: fory, as: SimpleStruct.self)
}
+private func handleExactFieldTags(_ bytes: [UInt8]) throws -> [UInt8] {
+ let fory = Fory(config: .init(trackRef: false, compatible: false))
+ try fory.register(ExactFieldTags.self, id: 104)
+ return try roundTripSingle(bytes, fory: fory, as: ExactFieldTags.self)
+}
+
private func handleNamedSimpleStruct(_ bytes: [UInt8]) throws -> [UInt8] {
let fory = Fory(config: .init(trackRef: false, compatible: true))
try fory.register(PeerColor.self, name: "demo.color")
@@ -1168,6 +1184,8 @@ private func rewritePayload(caseName: String, bytes:
[UInt8]) throws -> [UInt8]
return try handleCrossLanguageSerializer(bytes)
case "test_simple_struct":
return try handleSimpleStruct(bytes)
+ case "test_exact_field_tags":
+ return try handleExactFieldTags(bytes)
case "test_named_simple_struct":
return try handleNamedSimpleStruct(bytes)
case "test_struct_evolving_override":
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
