Andrew Flower created KAFKA-21570:
-------------------------------------

             Summary: Task-level connector-type JMX attribute returns a 
ConnectorType enum, breaking remote JMX clients
                 Key: KAFKA-21570
                 URL: https://issues.apache.org/jira/browse/KAFKA-21570
             Project: Kafka
          Issue Type: Bug
          Components: connect
    Affects Versions: 4.2.1, 4.1.0
            Reporter: Andrew Flower


KAFKA-18988 (4.1.0) added a {{connector-type}} attribute to 
{{kafka.connect:type=connector-task-metrics}}. Its value is the 
{{org.apache.kafka.connect.runtime.rest.entities.ConnectorType}} enum rather 
than a String:

{code:java}
// WorkerTask.java (4.1, 4.2 and trunk, line 468)
metricGroup.addValueMetric(registry.taskConnectorType, now -> 
pluginsMetadata.connectorType());
{code}

Over remote JMX (RMI), attribute values are Java-serialized, so any client 
without the Connect runtime on its classpath fails to read it:

{noformat}
java.rmi.UnmarshalException: error unmarshalling return; nested exception is:
    java.lang.ClassNotFoundException: 
org.apache.kafka.connect.runtime.rest.entities.ConnectorType (no security 
manager: RMI class loader disabled)
{noformat}

Every other Connect metric is a number or a String. The connector-level 
{{connector-type}} metric already returns a String ("source" / "sink"), and 
{{JmxReporter}} advertises every attribute as {{double}}.

*Impact*
Generic remote JMX collectors don't ship Kafka's jars. The Datadog agent's JMX 
check treats the {{UnmarshalException}} as a broken connection and drops all 
metrics for the worker, not just this attribute. We lost every 
{{kafka.connect}} and JVM metric after upgrading an embedded Connect worker 
from 4.0.0 to 4.2.1, and on a Connect cluster running a Confluent Platform 8.x 
image. Confluent support reproduced it on CP 8.4.1; CP 8.0.x is not affected.

*Steps to reproduce*
# Run a Connect 4.1+ worker with remote JMX enabled and at least one connector 
task.
# Connect with a JMX client that doesn't have Kafka on its classpath (e.g. 
jmxterm or JConsole).
# Read {{connector-type}} on 
{{kafka.connect:type=connector-task-metrics,connector=<name>,task=<id>}}.

*Workaround*
Exclude the attribute in the JMX collector, or collect only named attributes.

*Fix*
Return {{ConnectorType.toString()}}, which yields "source", "sink" or 
"unknown", matching the connector-level metric: 
https://github.com/apache/kafka/pull/24249
{code:java}
metricGroup.addValueMetric(registry.taskConnectorType, now -> 
pluginsMetadata.connectorType().toString());
{code}



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to