[
https://issues.apache.org/jira/browse/PLC4X-322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17437878#comment-17437878
]
Łukasz Dywicki commented on PLC4X-322:
--------------------------------------
By looking at sources I think we can't blame kafka-connector itself. Looking at
[org.apache.plc4x.kafka.Plc4xSourceTask|https://github.com/apache/plc4x/blob/v0.9.0/plc4j/integrations/apache-kafka/src/main/java/org/apache/plc4x/kafka/Plc4xSourceTask.java]
I see few places which might result in issues.
Because kafka connector wraps scraper utility we have issue might be buried in
second component. Given ephemeral nature of scraper itself we probably didn't
spot issues with connection management in it. And it seem not to stop
connections properly [at least
here|https://github.com/apache/plc4x/blob/v0.9.0/plc4j/tools/scraper/src/main/java/org/apache/plc4x/java/scraper/triggeredscraper/TriggeredScraperImpl.java#L269L283],
at least now I can't locate valid piece of code which should do that. Other
thing that (I don't know kafka connector apis), kafka related tasks lifecycle
is a bit of mystery to me, yet I see we had a fix for issue which sounds
exactely as yours:
https://github.com/apache/plc4x/commit/824c791c868665162e40ab2471974dd944f43bc3
(see fix calls scaper.stop(), but later we have no proper connection pool
shutdown).
> Modbus - Kafka does not close the connections
> ---------------------------------------------
>
> Key: PLC4X-322
> URL: https://issues.apache.org/jira/browse/PLC4X-322
> Project: Apache PLC4X
> Issue Type: Bug
> Components: Integration-Kafka-Connect
> Affects Versions: 0.9.0
> Environment: WSL2 (Ubuntu 20.04)
> Reporter: Fernando
> Priority: Major
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> Hello,
> I have been testing the connector and Kafka connects for a few weeks now. To
> do these tests, I try to ingest measurements from a sensor using Modbus, and
> this measurement ingestion is done correctly. The problem arises when I
> decide to remove the connector, as this connection is not closed by default
> with my sensor, which leads me to serious problems (due to the limitation of
> devices that read by Modbus, limited to 4 users per manufacture). The only
> way I can close these connections is to restart the kafka connect container.
> Is there any option or way to force these connections to close? Additionally,
> here are the steps to replicate this error:
> 1. Initially, I don't have any connector launched, so I don't have any active
> connection: active: 0, waiting: 0
> 2. Then, I launch the connector using Kafka Rest API. At this moment, we're
> importing data into the kafka cluster. Using curl, we can see that the
> connector is working: curl -X 'GET' http://localhost:18083/connectors/ ->
> ["modbus-office"]. We can see right now that we only have 1 active
> connection: active: 1, waiting: 0
> 3. I delete the kafka connector using the rest API: curl -X 'DELETE'
> http://localhost:18083/connectors/modbus-office. So, now, using the previous
> command from the previous point, we don't see any active connector, but,
> analysing the active connections from the sensor, it can be seen that there
> is one active connection: active: 1, waiting: 0.
> This is what is giving me problems, as in theory, there should not be any
> active connections at the moment. We have made a proxy to limit the number of
> active connections, but so far, we have not been able to close it manually
> using the REST API of Kafka or some configuration of the connector.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
