Hey Yeikel, Thanks for your question!
It appears that we only expose the already-concatenated data to the logs: https://github.com/apache/kafka/blob/e8c70fce26626ed2ab90f2728a45f6e55e907ec1/connect/runtime/src/main/java/org/apache/kafka/connect/util/LoggingContext.java#L186-L204 instead of letting the logger do the concatenation. It looks like the discussion thread included mention of parsing, and the current format is intended to be "easy" to parse. I don't see any mention of providing connector id as a separate field that can be configured in the log config, so I don't think they considered such a solution. I think you'll need to stick to regex parsing the connector.context field in the logs. Hope this helps! Greg On Sat, Feb 10, 2024 at 9:58 PM Yeikel Santana <em...@yeikel.com> wrote: > > Hi team, > > > > In KIP-449[1], we introduced a "connector context" that allows us to get > worker context in the logs such as "[my-connector|task-0|offsets]" > > > > Are you aware of any built-in mechanism to extract only the connector name > from these logs? > > > > In an ideal world, I would like to build a log that contains just the > connector name excluding any task or offset information. ie: > connector=my-connector-name. The final goal is to get stats per connector > name where the task or offset information is not that important. The > additional metadata gets in the way as it pollutes the log entry > > > > I imagine that I can write a regular expression to do this, but I do wish to > reinvent the wheel if there are better options. > > > > Thank you! > > > > [1] > https://cwiki.apache.org/confluence/display/KAFKA/KIP-449%3A+Add+connector+contexts+to+Connect+worker+logs
