Re: Kafka Connect with HdfsSinkConnector on DC/OS ..

2017-08-03 Thread Debasish Ghosh
Yes, looks like the hdfs.url value that I gave is not the proper one. But I cannot find how to get the appropriate URL to supply here. If I ask dcos hdfs for the core-site.xml I get the following .. $ dcos hdfs endpoints core-site.xml fs.default.name hdfs://hdfs

Re: Kafka Connect with HdfsSinkConnector on DC/OS ..

2017-08-03 Thread Stephen Durfey
Looking around some more it could be a misconfiguration of the running hadoop cluster and what you're trying to point to. So, double check the IP address and port number for the hadoop cluster (in the hadoop cluster config and make sure the port is open, reachable, and listening). If its all runnin

Re: Kafka Connect with HdfsSinkConnector on DC/OS ..

2017-08-03 Thread Debasish Ghosh
Thanks for the heads up. In fact I am using both HDFS and Kafka Connect from Mesosphere repository. Hence expected some compatibility .. will take a look to see if they include different versions of protobuf. regards. On Thu, Aug 3, 2017 at 7:24 PM, Stephen Durfey wrote: > That sounds like eith

Re: Kafka Connect with HdfsSinkConnector on DC/OS ..

2017-08-03 Thread Stephen Durfey
That sounds like either a protobuf dependency compatibility issue between what is on the classpath of kafka connect and the hadoop cluster you are trying to write to (e.g. you're on a newer version of protobuf than your cluster, or vice versa), or a wire incompatilibty of the communcation protocol