cwtrex commented on pull request #7965: URL: https://github.com/apache/kafka/pull/7965#issuecomment-1021635467
> > suggest using grok patterns > > I knew I'd seen this somewhere before, but finally found it again > > https://github.com/streamthoughts/kafka-connect-transform-grok So I was able to build the solution presented here, but after some hair pulling, I did end up using the grok pattern solution since it performed better overall to include being able to parse curly brackets correctly. I did note that the grok SMT had not been updated in over a year. Hopefully it is just because they consider it stable rather than it becoming abandonware. Regardless, thank you for sharing @OneCricketeer It was very much appreciated. I definitely appreciated the workflow as well. The grok solution provides a method where depending on the message you may need to declare it as a string using the Cast SMT, but you are then able to directly work with the message to form the struct and appropriate fields. With the solution presented here, I had to use Cast to declare the message and string, then apply HoistField to make a simple struct for all of the message field, then I could use this SMT to try and work with the struct itself. The grok method was definitely more efficient and the inclusion of the usual + additional patterns was a very nice addition. The note I would add that was not immediately obvious to me was that while I could place this SMT into the Kafka plugins directly, I received very odd errors with the Grok SMT. It took me far longer than I care to admit to realize that the Grok SMT and the 2 supporting jar files needed to be placed into their own subfolder within the Kafka plugins directly in order for the jar files to be loaded properly. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
