Alright... So to repeat (and I think you get this much)... Kafka is in principle very simple; Put any byte array into a topic in one end, and readers can get that byte array from that topic somewhere else. Just a simple message queue, at an abstract level. And the Kafka client libraries for doing this is dead easy to use, producer.send() and consumer.poll() respectively.
So, I suspect that you are talking about some higher level than base Kafka, possibly masquerading the Kafka transport as some other transport abstraction or a component implementing some protocol, such as MQTT [1], that behind the end point feeds Kafka. But I could be completely off-track and some really clever ideas are at play here, that I don't realize yet. So, yes, please invite people to discuss further. Niclas [1] MQTT feeding Kafka is something I will implement myself in the not too distant future On Wed, Mar 21, 2018 at 4:50 PM, Christofer Dutz <christofer.d...@c-ware.de> wrote: > Hi Niclas, > > have to admit I don't know in detail either ... it was part of a talk a > colleague of mine held at a Kafka meetup in our office. Also I'm not into > Kafka half as deep as you ... from a user point of view it looked as if > such a Kafka Connect adapter would be a good idea. I'm trying to convince > that colleague to contribute ... perhaps he could register here and shed > some light on the topic ... I'll continue the convincing (even if another > codecentric participant wouldn't increase company diversity) > > Chris > > > Am 13.03.18, 16:41 schrieb "hedh...@gmail.com im Auftrag von Niclas > Hedhman" <hedh...@gmail.com im Auftrag von nic...@hedhman.org>: > > Sorry, don't know what that is or what it really is, compared to what I > wrote... > > On Tue, Mar 13, 2018 at 8:50 PM, Christofer Dutz < > christofer.d...@c-ware.de> > wrote: > > > Argh ... I wanted to propose a Kafka Connect and not a Kafka Steam > adapter > > ( ... > > > > By the way my currently most used setup is actually using PLC4X for > PLC > > communication and using Edgents Kafka Connector to publish to Kafka. > > > > A Kafka Connect adapter would allow to directly connect it to Kafka. > > > > Chris > > > > > > Am 13.03.18, 00:02 schrieb "hedh...@gmail.com im Auftrag von Niclas > > Hedhman" <hedh...@gmail.com im Auftrag von nic...@hedhman.org>: > > > > I have some Kafka experience, but not used Kafka Streams. But I > think > > the > > most straight forward approach would still be to have a Kafka > Producer > > on > > the PLC4X side simply writing to one or more Kafka topics, which > is > > relatively simple. > > > > You need to 'configure' during the instance creation; > > > > Properties properties = new Properties(); > > properties.put( ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, > cli.kafka ); > > properties.put( ProducerConfig.LINGER_MS_CONFIG, 1 ); > > properties.put( ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, > > StringSerializer.class.getName() ); > > properties.put( ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, > > StringSerializer.class.getName() ); > > KafkaProducer<String, String> producer = new KafkaProducer<>( > > properties ); > > > > And once you have the producer, you can push a message to the > topic > > with or > > without a key, like this; > > > > producer.send( new ProducerRecord<>( "your-topic", value ) ); > > > > > > > > I am pretty sure Kafka Streams can take over from the topic > queue. > > > > > > Cheers > > Niclas > > > > On Mon, Mar 12, 2018 at 10:38 PM, Christofer Dutz < > > christofer.d...@c-ware.de > > > wrote: > > > > > Hi, > > > > > > last week I attended the Kafka meetup of a colleague of mine > and in > > his > > > talk he introduced Kafka Streams and to me it sounded like a > > „Clustered > > > Edgent“. > > > > > > My second thought was: would it not be cool to implement such > a Kafka > > > Stream Connector? > > > > > > Anyone here got the knowhow to do that? Shouldn’t be that > difficult. > > > > > > Chris > > > > > > > > > > > -- > > Niclas Hedhman, Software Developer > > http://polygene.apache.org - New Energy for Java > > > > > > > > > -- > Niclas Hedhman, Software Developer > http://polygene.apache.org - New Energy for Java > > > -- Niclas Hedhman, Software Developer http://polygene.apache.org - New Energy for Java
