HI Philip, thanks for the reply !
This issue looks like it might match my problem indeed. I'll look into it. Jehan On 16 October 2017 at 20:53, Philip Schmitt <philip.schm...@outlook.com> wrote: > Hi Jehan, > > I've run into the same issue last week and also got a "class could not be > found" error. > > Konstantine Karantasis helpfully pointed me towards > https://issues.apache.org/jira/projects/KAFKA/issues/KAFKA-6007 > > To test this, I simply copied my SMT jar to the folder of the connector I > was using and adjusted the plugin.path property. > > I haven't fully tested tested it, but in my very quick initial test, it > seemed to have move past that "class could not be found" issue. > > Regards, > Philip > ________________________________ > From: jeh...@gmail.com <jeh...@gmail.com> on behalf of Jehan Bruggeman < > jehanbrugge...@gmail.com> > Sent: Monday, October 16, 2017 8:17 AM > To: users@kafka.apache.org > Subject: Re: Custom converter with Kafka Connect ? > > Hi Randall, > > thanks for your reply. I'm not sure about this; what detail could I add > that would help you figure it out ? > > Concerning the classpath: as described in my original email, I'm pretty > sure the jars are correctly added to the classpath since the classes in the > jar are recognized by Kafka Connect when it starts (they are mentionned in > the logs, at least). > > (It's easier to read here, where I asked the same question: > https://stackoverflow.com/questions/46712095/using-a- > custom-converter-with-kafka-connect > ) > > thanks for your help ! > > Jehan > > On 13 October 2017 at 16:07, Randall Hauch <rha...@gmail.com> wrote: > > > On Tue, Oct 10, 2017 at 8:31 AM, Jehan Bruggeman < > jehanbrugge...@gmail.com > > > > > wrote: > > > > > Hello, > > > > > > I'm trying to use a custom converter with Kafka Connect and I cannot > seem > > > to get it right. I'm hoping someone has experience with this and could > > help > > > me figure it out ! > > > > > > > > > Initial situation > > > ================ > > > > > > - my custom converter's class path is 'custom.CustomStringConverter'. > > > > > > - to avoid any mistakes, my custom converter is currently just a > > copy/paste > > > of the pre-existing StringConverter (of course, this will change when > > I'll > > > get it to work). > > > https://github.com/apache/kafka/blob/trunk/connect/api/ > > > src/main/java/org/apache/kafka/connect/storage/StringConverter.java > > > > > > - I have a kafka connect cluster of 3 nodes, The nodes are running > > > confluent's official docker images ( confluentinc/cp-kafka-connect: > 3.3.0 > > > ). > > > > > > - Each node is configured to load a jar with my converter in it (using > a > > > docker volume). > > > > > > > Can you explain this in more detail? Make sure that you add the JAR to > the > > classpath. > > > > > > > > > > > > > > > > What happens ? > > > ================ > > > > > > When the connectors start, they correctly load the jars and find the > > custom > > > converter. Indeed, this is what I see in the logs : > > > > > > [2017-10-10 13:06:46,274] INFO Registered loader: > > > PluginClassLoader{pluginLocation=file:/opt/custom-connectors/custom- > > > converter-1.0-SNAPSHOT.jar} > > > (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:199) > > > [2017-10-10 13:06:46,274] INFO Added plugin > > 'custom.CustomStringConverter' > > > (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:132) > > > [...] > > > [2017-10-10 13:07:43,454] INFO Added aliases 'CustomStringConverter' > and > > > 'CustomString' to plugin 'custom.CustomStringConverter' > > > (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:293) > > > > > > I then POST a JSON config to one of the connector nodes to create my > > > connector : > > > > > > { > > > "name": "hdfsSinkCustom", > > > "config": { > > > "topics": "yellow", > > > "tasks.max": "1", > > > "key.converter": "org.apache.kafka.connect. > storage.StringConverter", > > > "value.converter": "custom.CustomStringConverter", > > > "connector.class": "io.confluent.connect.hdfs.HdfsSinkConnector", > > > "hdfs.url": "hdfs://hdfs-namenode:8020/hdfs-sink", > > > "topics.dir": "yellow_storage", > > > "flush.size": "1", > > > "rotate.interval.ms": "1000" > > > } > > > } > > > > > > And receive the following reply : > > > > > > { > > > "error_code": 400, > > > "message": "Connector configuration is invalid and contains the > > > following 1 error(s):\nInvalid value custom.CustomStringConverter for > > > configuration value.converter: Class custom.CustomStringConverter could > > not > > > be found.\nYou can also find the above list of errors at the endpoint > > > `/{connectorType}/config/validate`" > > > } > > > > > > ================ > > > > > > If I try running Kafka Connect stadnalone, the error message is the > same. > > > > > > Has anybody faced this already ? What am I missing ? > > > > > > Many thanks to anybody reading this ! > > > > > > Jehan > > > > > >
