gscher opened a new pull request, #4030:
URL: https://github.com/apache/avro/pull/4030
## What is the purpose of the change
This pull request makes resolving the generated class of a named schema in
`ObjectCreator.FindType` independent of the number of types loaded in the
process, fixing AVRO-4357. Before, every name not found in the entry assembly
enumerated all types of all loaded assemblies, which takes seconds and
gigabytes of allocations in large applications on the first deserialization of
a schema.
- Names with a namespace are looked up directly with `Assembly.GetType` in
every loaded assembly.
- Names without a namespace, and names not found directly, fall back to the
existing enumeration.
- `UnMangle(name)` is computed once per name instead of once per type.
- The resolved type is the same as before: if several assemblies contain the
type, the one found last in the loaded assemblies wins.
- One difference: an assembly whose `GetTypes()` throws was skipped entirely
before. The direct lookup can still find a loadable type in it, so names that
failed to resolve before can resolve now.
This touches the same method as #3966 (AVRO-3893), which avoids the per-call
closure in `FindType`. The two changes are independent and should combine
cleanly; I'm happy to rebase onto it if it lands first.
## Verifying this change
This change added tests and can be verified as follows:
- `ObjectCreatorTests` gains tests for resolving a full name from a
non-entry assembly, for reserved words in the namespace (mangled and
unmangled), and for the last loaded assembly winning when several assemblies
contain the same full name. All new tests pass before and after the change.
- `Avro.benchmark` gains `ObjectCreatorBenchmarks`, which resolves 50 named
types with a fresh `ObjectCreator` while 0 to 250,000 additional types are
loaded. The benchmark can be removed if it shouldn't land in main.
Run the benchmark with `dotnet run -c Release -f net10.0 -- --filter
'*ObjectCreatorBenchmarks*'` in `lang/csharp/src/apache/benchmark`, once with
`ObjectCreator.cs` from `main` and once with this change.
Before (`ObjectCreator.cs` from `main`):
```
BenchmarkDotNet v0.15.8, macOS Tahoe 26.6.2 (25G83) [Darwin 25.6.0]
Apple M2 Max, 1 CPU, 12 logical and 12 physical cores
.NET SDK 10.0.401
[Host] : .NET 10.0.12 (10.0.12, 10.0.1226.42308), Arm64 RyuJIT
armv8.0-a
Job-HDDYAJ : .NET 10.0.12 (10.0.12, 10.0.1226.42308), Arm64 RyuJIT
armv8.0-a
IterationCount=10 LaunchCount=1 WarmupCount=2
| Method | AdditionalLoadedTypes | Mean | Error |
StdDev | Gen0 | Gen1 | Allocated |
|------------------------ |----------------------
|-----------:|---------:|---------:|------------:|-----------:|-----------:|
| ResolveNamedSchemaTypes | 0 | 286.9 ms | 14.46 ms |
8.60 ms | 36500.0000 | 1000.0000 | 292.12 MB |
| ResolveNamedSchemaTypes | 25000 | 1,017.6 ms | 79.68 ms |
52.70 ms | 117000.0000 | 3000.0000 | 939.97 MB |
| ResolveNamedSchemaTypes | 100000 | 2,720.9 ms | 97.21 ms |
57.85 ms | 339000.0000 | 10000.0000 | 2707.81 MB |
| ResolveNamedSchemaTypes | 250000 | 5,970.5 ms | 33.21 ms |
21.96 ms | 782000.0000 | 23000.0000 | 6243.94 MB |
Outliers
ObjectCreatorBenchmarks.ResolveNamedSchemaTypes: IterationCount=10,
LaunchCount=1, WarmupCount=2 -> 1 outlier was removed (311.73 ms)
ObjectCreatorBenchmarks.ResolveNamedSchemaTypes: IterationCount=10,
LaunchCount=1, WarmupCount=2 -> 1 outlier was removed (3.02 s)
```
After (this change):
```
BenchmarkDotNet v0.15.8, macOS Tahoe 26.6.2 (25G83) [Darwin 25.6.0]
Apple M2 Max, 1 CPU, 12 logical and 12 physical cores
.NET SDK 10.0.401
[Host] : .NET 10.0.12 (10.0.12, 10.0.1226.42308), Arm64 RyuJIT
armv8.0-a
Job-HDDYAJ : .NET 10.0.12 (10.0.12, 10.0.1226.42308), Arm64 RyuJIT
armv8.0-a
IterationCount=10 LaunchCount=1 WarmupCount=2
| Method | AdditionalLoadedTypes | Mean | Error |
StdDev | Gen0 | Gen1 | Allocated |
|------------------------ |----------------------
|-----------:|----------:|----------:|---------:|--------:|-----------:|
| ResolveNamedSchemaTypes | 0 | 592.1 us | 12.87 us |
7.66 us | 78.1250 | 1.9531 | 641.41 KB |
| ResolveNamedSchemaTypes | 25000 | 986.0 us | 25.57 us |
15.22 us | 126.9531 | 1.9531 | 1041.8 KB |
| ResolveNamedSchemaTypes | 100000 | 2,156.7 us | 35.80 us |
23.68 us | 273.4375 | 7.8125 | 2242.98 KB |
| ResolveNamedSchemaTypes | 250000 | 4,446.9 us | 293.64 us |
194.23 us | 562.5000 | 15.6250 | 4645.32 KB |
Outliers
ObjectCreatorBenchmarks.ResolveNamedSchemaTypes: IterationCount=10,
LaunchCount=1, WarmupCount=2 -> 1 outlier was removed (651.12 us)
ObjectCreatorBenchmarks.ResolveNamedSchemaTypes: IterationCount=10,
LaunchCount=1, WarmupCount=2 -> 1 outlier was removed (1.05 ms)
```
## Documentation
- Does this pull request introduce a new feature? No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
