Ewen Cheslack-Postava created KAFKA-5330:
--------------------------------------------
Summary: Use per-task converters in Connect
Key: KAFKA-5330
URL: https://issues.apache.org/jira/browse/KAFKA-5330
Project: Kafka
Issue Type: Improvement
Components: KafkaConnect
Affects Versions: 0.11.0.0
Reporter: Ewen Cheslack-Postava
Because Connect started with a worker-wide model of data formats, we currently
allocate a single Converter per worker and only allocate an independent one
when the user overrides the converter.
This can lead to performance problems when the worker-level default converter
is used by a large number of tasks because converters need to be threadsafe to
support this model and they may spend a lot of time just on synchronization.
We could, instead, simply allocate one converter per task. There is some
overhead involved, but generally it shouldn't be that large. For example,
Confluent's Avro converters will each have their own schema cache and have to
make their on calls to the schema registry API, but these are relatively small,
likely inconsequential compared to any normal overhead we would already have
for creating and managing each task.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
