jonyoder opened a new pull request, #3086:
URL: https://github.com/apache/fory/pull/3086
## What does this PR do?
Fixes reference tracking bug that causes deserialization to fail with hash
mismatch errors when more than 127 objects are serialized with reference
tracking enabled.
## Root Cause
The check `if int8(refID) < NotNullValueFlag` in multiple files causes
integer overflow when `refID >= 128`:
```go
// BEFORE (buggy):
if int8(refID) < NotNullValueFlag {
```
- `int8(128)` = `-128` (two's complement overflow)
- `-128 < -1` (NotNullValueFlag) = `TRUE` (incorrectly!)
This causes the code to incorrectly enter the "reference found" branch,
corrupting buffer positions and causing hash mismatches.
## Fix
Replace int8 cast with int32 comparison:
```go
// AFTER (fixed):
if refID < int32(NotNullValueFlag) {
```
Since `refID` is already `int32` (returned from `TryPreserveRefId`), and the
flag constants are small negative int8 values that correctly sign-extend to
int32, this comparison is type-safe and avoids truncation.
## Files Changed
- `array.go` (1 occurrence)
- `map.go` (3 occurrences)
- `pointer.go` (2 occurrences)
- `reader.go` (7 occurrences)
- `serializer.go` (1 occurrence)
- `set.go` (4 occurrences)
- `slice.go` (1 occurrence)
- `slice_dyn.go` (4 occurrences)
- `struct.go` (2 occurrences)
## Testing
Added regression test `TestRefTrackingLargeCount` in `ref_resolver_test.go`
that verifies serialization/deserialization works correctly with 127, 128, and
200 items.
All existing tests pass.
## Related Issues
Closes #3085
## Checklist
- [x] I have read the [contributing guidelines](../CONTRIBUTING.md)
- [x] I have added tests that prove my fix is effective
- [x] All new and existing tests pass locally with my changes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
