[
https://issues.apache.org/jira/browse/AVRO-4357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated AVRO-4357:
---------------------------------
Labels: pull-request-available (was: )
> [C#] ObjectCreator enumerates all types of all loaded assemblies for every
> schema name
> --------------------------------------------------------------------------------------
>
> Key: AVRO-4357
> URL: https://issues.apache.org/jira/browse/AVRO-4357
> Project: Apache Avro
> Issue Type: Improvement
> Components: csharp
> Affects Versions: 1.12.2
> Reporter: Gerald Schermann
> Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {{ObjectCreator.FindType}} resolves the generated class of a named schema
> type by name. If the class is not in the entry assembly, the usual case for
> generated contracts, it enumerates all types of all loaded assemblies,
> without stopping at a match and allocating an unmangled copy of the name per
> type. This runs once per named schema type and process, so the first
> deserialization of a schema costs (named types in the schema) x (loaded
> types).
> We hit this in a large .NET 10 service consuming Kafka with Confluent.Kafka
> 2.15.0 and {{AvroDeserializer<T>}} (Confluent.SchemaRegistry.Serdes.Avro
> 2.15.0) on Apache.Avro 1.12.2: the first message after every process start
> took about 5 seconds to deserialize, the following ones milliseconds.
> We benchmarked time and allocations to resolve (the generated) classes of 50
> named schema types with an empty type cache (a new {{{}ObjectCreator{}}}),
> depending on how many types the process has loaded (BenchmarkDotNet, .NET 10,
> Apple M2 Max):
> ||Types loaded in the process||Time to resolve all 50||Allocated||
> |5,938|286.9 ms|292 MB|
> |30,938|1,017.6 ms|940 MB|
> |105,938|2,720.9 ms|2,708 MB|
> |255,938|5,970.5 ms|6,244 MB|
> h3. Proposed fix
> Look up names with a namespace directly with {{Assembly.GetType(name,
> false)}} in each loaded assembly, and keep the existing enumeration as
> fallback. This resolves the same types as before, including the last match
> winning across assemblies, and brings the 255,938-type case down from 5,970.5
> ms and 6,244 MB allocated to 4.4 ms and 4.5 MB.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
