david-streamlio opened a new issue, #61:
URL: https://github.com/apache/pulsar-connectors/issues/61
## Description
The Debezium SQL Server source (`DebeziumMsSqlSource`) fails during `open()`
with a `NullPointerException` inside Debezium's JMX metric-name construction:
```
java.lang.NullPointerException
at io.debezium.metrics.Metrics ... Sanitizer.jmxSanitize(null)
via SchemaHistoryMetrics
```
This was found while adding integration test coverage for the module (#43 /
#60), and it reproduces against a real SQL Server container with the documented
configuration. It is not a test-only problem.
## Root cause
`AbstractKafkaConnectSource.open()` only calls `connector.taskConfigs(1)`
when the config contains the adaptor's own `kafkaConnectorSourceClass` key
([AbstractKafkaConnectSource.java#L144-L170](https://github.com/apache/pulsar-connectors/blob/master/kafka-connect-adaptor/src/main/java/org/apache/pulsar/io/kafka/connect/AbstractKafkaConnectSource.java#L144)):
```java
if (config.get(CONNECTOR_CLASS) != null) { // CONNECTOR_CLASS =
"kafkaConnectorSourceClass"
...
List<Map<String, String>> configs = connector.taskConfigs(1);
taskConfig = configs.get(0);
} else {
// backward-compat path: instantiate the task directly from task.class
sourceTask =
Class.forName(stringConfig.get(TaskConfig.TASK_CLASS_CONFIG))...;
taskConfig = stringConfig; // <-- raw user config, nothing injected
}
```
The Debezium sources set Debezium's own `connector.class` and `task.class`
keys (see `DebeziumMsSqlSource.setDbConnectorClass/setDbConnectorTask`),
**not** `kafkaConnectorSourceClass`. So every Debezium source takes the `else`
branch and `SqlServerConnector.taskConfigs()` is never invoked.
That matters for SQL Server specifically because it is a **multi-partition**
connector: `taskConfigs()` is what assigns each task its `task.id`, and
`SqlServerConnectorConfig` requires it. Starting the task directly leaves
`task.id` unset, and the null propagates into the metrics layer.
Single-partition connectors (MySQL, Postgres) don't hit this, which is why
the existing tests pass.
## Workaround
Setting `task.id=0` explicitly in the connector config lets the source
start. #60 does this in its integration test with an explanatory comment so the
test can land, but the underlying gap should be fixed so users don't have to.
## Suggested fix
Route Debezium sources through `taskConfigs()` (i.e. have `DebeziumSource`
populate `kafkaConnectorSourceClass`, or invoke `taskConfigs()` in the
backward-compat branch when a connector class is resolvable), so
multi-partition connectors are configured the way Kafka Connect configures
them. A regression test for SQL Server exists in #60 once merged; removing the
explicit `task.id` there would validate the fix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
