The bug I was referring to was only in trunk for just a while. Thus, your issue must be related to something else, even though the response statuses are similar.
Let me know if you want to share a bigger and more detailed (DEBUG level at least) snapshot of the parts of the logs that might be related to this failure. Cheers, Konstantine On Wed, Dec 7, 2016 at 11:15 AM, Phillip Mann <pm...@trulia.com> wrote: > Hello Konstantine, > > > > Thanks for your reply. > > > > I am using Confluent 3.0.1 installed on my machine and our cluster. > However, our AWS cluster has Confluent 3.1.1 installed so I will test with > 3.1.1 client and cluster and see if this resolves the issue. Additionally, > I’ll use the debug levels if this does not resolve my issue. > > > > If not, I’ll explore the trunk repo but I would prefer to use stable > versions of CP / Kafka that can be accessed with Maven. > > > > Thanks again. > > > > Phillip > > > > > Hi Phillip, > > > > > > may I ask which Kafka version did you use? > > > > > > trunk repo in Apache Kafka contained briefly a bug in Connect framework > > > (during the past week) that produced failures similar to the one you > > > describe (only in distributed mode). A fix has been pushed since > yesterday. > > > > > > 3) Some useful step-by-step information is provided in the quickstart > guide > > > here: > > > https://kafka.apache.org/quickstart#quickstart_kafkaconnect > > > > > > as well as in the documentation of Confluent: > > > http://docs.confluent.io/3.1.0/connect/quickstart.html# > > > > > > Alternatively, you might want to follow the quickstart guide of one of > the > > > open source connectors, here: > > > http://docs.confluent.io/3.1.0/connect/connectors.html > > > > > > 2) From what you mention above, it seems more like that you're hitting > this > > > temporary bug. But again that depends on which Kafka version you've been > > > using. > > > > > > 1) Generating logs, in one of the debug levels (e.g. DEBUG, TRACE) is > > > usually a useful source of information. > > > Alternatively you may chose to run Connect in debug mode by setting the > > > environment variable KAFKA_DEBUG and attaching a remote debugger to it > > > (such as IntelliJ's remote debugging capability). With respect to live > > > debugging, we are planning to post a step-by-step guide for Kafka and > Kafka > > > Connect soon. > > > > > > Regards, > > > Konstantine > > > > > >> On Tue, Dec 6, 2016 at 11:22 AM, Phillip Mann <pm...@trulia.com> wrote: > > >> > > >> I am working on migrating from Camus to Kafka Connect. I am working on > the > > >> implementation of Kafka Connect and specifically focused on distributed > > >> mode. I am able to start a worker successfully on my local machine > which I > > >> assume communicates with my Kafka cluster. I am further able to run two > GET > > >> commands such as / and /connector-plugins which return the correct JSON. > > >> However, when I try to POST a command to create a connector, I receive a > > >> 500 error and a time out. Specifically, I use this command to POST for > > >> testing: > > >> > > >> curl -X POST -H "Content-Type: application/json" --data '{"name": > > >> "local-file-sink", "config": {"connector.class":" > FileStreamSinkConnector", > > >> "tasks.max":"1", "file":"test.sink.txt", "topics":"myTopic" }}' > > >> localhost:8083/connectors > > >> > > >> and eventually I get this response: > > >> > > >> {"error_code": 500, "message": "Request timed out"} > > >> > > >> I am lost as to what is going on. The logs from my Kafka Connect > > >> distributed worker show this: > > >> > > >> [2016-12-05 14:34:32,436] INFO 0:0:0:0:0:0:0:1 - - [05/Dec/2016:22:34:32 > > >> +0000] "GET /connector-plugins HTTP/1.1" 200 315 2 > > >> (org.apache.kafka.connect.runtime.rest.RestServer:60) > > >> [2016-12-05 15:05:25,422] INFO 0:0:0:0:0:0:0:1 - - [05/Dec/2016:23:05:25 > > >> +0000] "GET /connector-plugins HTTP/1.1" 200 315 3 > > >> (org.apache.kafka.connect.runtime.rest.RestServer:60) > > >> [2016-12-05 15:05:28,389] INFO 0:0:0:0:0:0:0:1 - - [05/Dec/2016:23:05:28 > > >> +0000] "GET /connector-plugins HTTP/1.1" 200 315 2 > > >> (org.apache.kafka.connect.runtime.rest.RestServer:60) > > >> [2016-12-05 15:07:38,644] INFO 0:0:0:0:0:0:0:1 - - [05/Dec/2016:23:06:08 > > >> +0000] "GET /connectors HTTP/1.1" 500 48 90003 > (org.apache.kafka.connect. > > >> runtime.rest.RestServer:60) > > >> [2016-12-05 15:07:44,450] INFO 0:0:0:0:0:0:0:1 - - [05/Dec/2016:23:07:44 > > >> +0000] "GET /connector-plugins HTTP/1.1" 200 315 1 > > >> (org.apache.kafka.connect.runtime.rest.RestServer:60) > > >> [2016-12-05 15:13:06,703] INFO 0:0:0:0:0:0:0:1 - - [05/Dec/2016:23:11:36 > > >> +0000] "POST /connectors HTTP/1.1" 500 48 90003 > (org.apache.kafka.connect. > > >> runtime.rest.RestServer:60) > > >> [2016-12-05 15:15:38,506] INFO 0:0:0:0:0:0:0:1 - - [05/Dec/2016:23:14:08 > > >> +0000] "POST /connectors HTTP/1.1" 500 48 90005 > (org.apache.kafka.connect. > > >> runtime.rest.RestServer:60) > > >> > > >> where you can see the error codes and the commands. > > >> > > >> I guess my main questions and issues are: > > >> > > >> 1. How can I better debug Kafka Connect so I can try and fix this? > > >> 2. Is there anything that I'm doing that is glaringly wrong? > > >> 3. Is there any step-by-step documentation or blog posts on getting a > > >> Kafka Connect distributed worker and connector to run? I have not really > > >> seen anything or even best practices kinds of documentation? Maybe I am > > >> just way too early of an adopter. > > >> > > >> I look forward to hearing back from the community and thank you for your > > >> help! > > >> > > >> >
