Re: Creating a Kafka Stream connector for PLC4X?

2018-03-24 Thread Christofer Dutz
Hi Niclas, Thanks for your explanation ... helps a lot :-) I wasn't suggesting that we immediately jump on this. Much more I thought this would be a great thing for someone involved with Kafka or at least a lot of Kafka knowledge to jump in. After all we're trying to get as many new people on

Re: Creating a Kafka Stream connector for PLC4X?

2018-03-23 Thread Niclas Hedhman
Kafka Connect is the ability to run a "source" and/or "sink" of streaming data on the Kafka instance itself. It defines the APIs so people can include the component (and its configuration) from the command-line. Part of the Kafka Connect component are Key and Value serialization, and in Kafka

Re: Creating a Kafka Stream connector for PLC4X?

2018-03-22 Thread Niclas Hedhman
Chris, I came across a presentation by coincidence that mentioned "Kafka Connect", so I will research this over the weekend and see what it is, and how it fits with PLC4X. The presenter only mentioned that a "Kafka Connect" component is enable by configuration (in Kafka was implied but not sure)

Re: Creating a Kafka Stream connector for PLC4X?

2018-03-21 Thread Niclas Hedhman
Alright... So to repeat (and I think you get this much)... Kafka is in principle very simple; Put any byte array into a topic in one end, and readers can get that byte array from that topic somewhere else. Just a simple message queue, at an abstract level. And the Kafka client libraries for doing

Re: Creating a Kafka Stream connector for PLC4X?

2018-03-21 Thread Christofer Dutz
Hi Niclas, have to admit I don't know in detail either ... it was part of a talk a colleague of mine held at a Kafka meetup in our office. Also I'm not into Kafka half as deep as you ... from a user point of view it looked as if such a Kafka Connect adapter would be a good idea. I'm trying to

Re: Creating a Kafka Stream connector for PLC4X?

2018-03-13 Thread Niclas Hedhman
Sorry, don't know what that is or what it really is, compared to what I wrote... On Tue, Mar 13, 2018 at 8:50 PM, Christofer Dutz wrote: > Argh ... I wanted to propose a Kafka Connect and not a Kafka Steam adapter > ( ... > > By the way my currently most used setup is

Re: Creating a Kafka Stream connector for PLC4X?

2018-03-12 Thread Niclas Hedhman
I have some Kafka experience, but not used Kafka Streams. But I think the most straight forward approach would still be to have a Kafka Producer on the PLC4X side simply writing to one or more Kafka topics, which is relatively simple. You need to 'configure' during the instance creation;

Creating a Kafka Stream connector for PLC4X?

2018-03-12 Thread Christofer Dutz
Hi, last week I attended the Kafka meetup of a colleague of mine and in his talk he introduced Kafka Streams and to me it sounded like a „Clustered Edgent“. My second thought was: would it not be cool to implement such a Kafka Stream Connector? Anyone here got the knowhow to do that?