Ali It is very possible you're being bitten by a bug we have with PutKafka. It is fixed in this branch which we're in the release motions for right now.
https://github.com/apache/nifi/tree/support/nifi-0.6.x Thanks Joe On Wed, Apr 6, 2016 at 10:07 AM, Ali Karray <[email protected]> wrote: > I'm using the last version 0.6 of Nifi and the 0.9 kafka version > When I use plain csv files, all is working. > > Thanks, > Ali > > > On Wed, Apr 6, 2016 at 3:23 PM, Bryan Bende <[email protected]> wrote: >> >> Hello, >> >> What version of Apache NiFi are you using? >> >> Also, lets take Avro and kafka-avro-console-consumer out of the mix for a >> minute, what happens if you used PutKafka to send some plain text to Kafka, >> and then tried to consumer from that topic with GetKafka? >> You mentioned this didn't work, but what happened? was there an error >> message? did it just receive 0 messages from Kafka? >> >> Thanks, >> >> Bryan >> >> On Wed, Apr 6, 2016 at 6:10 AM, Ali Karray <[email protected]> wrote: >>> >>> Hi, >>> >>> I'm exploring integration possibility of nifi processors with kafka. >>> I use PutKafka to puch avro generated flow from ExecuteSQL. >>> I verify that the topic created in Kafka is alimented with data using >>> kafka-console-consumer bash command. >>> Then I try to use getKafka processor to pull all the Data stored in >>> the kafka topic and store all that data in a local File using the >>> PutFile processor. I realize that the getKafka processor don't pull >>> the topic content. >>> >>> I verify if all is ok with kafka-avro-console-consumer (confluent >>> command), I get this exception : >>> org.apache.kafka.common.errors.SerializationException: Error >>> deserializing Avro message for id -1 >>> Caused by: org.apache.kafka.common.errors.SerializationException: >>> Unknown magic byte! >>> >>> I tested also getKafka from a basic topic format and it doesn't work for >>> me. >>> >>> So I have some questions : >>> How can I deal with avro format using getKafka anf putKafka processor ? >>> Any idea to resolve getKafka processor problem to pull content from a >>> kafka Topic ? >>> >>> >>> Thank you in advance >>> >>> A. Karray >> >>
