As I understood correctly, you're turning on tcpflow and send a command to Redis, right? Then maybe Redis protocol could be captured by tcpflow and published to Kafka. Please check your tcpflow setting.
2015-03-11 21:10 GMT+09:00 Mithun Kalan <[email protected]>: > Hi > I've got something weird going on. > > [my setup is: I get TCP data using tcpflow, publish to kafka using > kafkacat, storm picks it up with kafkaspout to process and publishes to > redis using jedis. They all run off a single machine with a single > zookeeper] > > Storm breaks a few minutes into the run because the messages coming from > kafka do not conform to the format that is expected. > Digging deeper, i found that the culprit messages in kafka look like the > ones in redis. > > How is this possible? i never told storm to publish into kafka. Kafka > should never have those messages. I cant find a pattern either. its not a > specific message or a timing issue. > > *The lines in kafka should have been:* > 2015-02-02 14:03 catergoy: A price: 123 > 2015-02-02 14:04 catergoy: A price: 124 > 2015-02-02 14:05 catergoy: A price: 122 > > *the lines in redis are * > 14:04 change +1 > 14:05 change -2 > > *but i get this * > 2015-02-02 14:03 catergoy: A price: 123 > 2015-02-02 14:04 catergoy: A price: 124 > *3 > $7 > message > $8 > $76 > 14:04 change +1 > 2015-02-02 14:05 catergoy: A price: 122 > > > i suspect its a because kafka and storm share the same zookeeper. I will > try to separate them to see if it works. > but has anyone had a similar issue? how did you solve it? > > Thank you > > Mithun Kalan > > -- Name : 임 정택 Blog : http://www.heartsavior.net / http://dev.heartsavior.net Twitter : http://twitter.com/heartsavior LinkedIn : http://www.linkedin.com/in/heartsavior
