Re: Any one using oracle event hub - dedicated ?

2018-07-25 Thread Robin Moffatt
Compared to other managed solutions, or compared to running it yourself? On Wed, 25 Jul 2018 at 12:44, Vaibhav Jain wrote: > Hi There, Any pros/cons ? Regards, Vaibhav -- -- Robin Moffatt | Developer Advocate | ro...@confluent.io | @rmoff

Re: Exposing Kafka on WAN

2018-08-31 Thread Robin Moffatt
Shameless plug for an article I wrote on understanding Kafka listeners better, which might help in this case: https://rmoff.net/2018/08/02/kafka-listeners-explained/ On Thu, 30 Aug 2018 at 14:14, Andrew Otto wrote: > The trouble is that the producer and consumer clients need to discover the

Re: how to use SMT to transform a filed value in kafka connect?

2018-07-10 Thread Robin Moffatt
inc/ksql/issues/620> ( https://github.com/confluentinc/ksql/issues/620) -- Robin Moffatt | Developer Advocate | ro...@confluent.io | @rmoff On Tue, 10 Jul 2018 at 03:48, 刘波 <1091643...@qq.com> wrote: > Hi all, > I'm new to kafak. I'm using kafka to build a ETL program. >

Re: Add company name on the Powered by page

2018-11-02 Thread Robin Moffatt
Hi Rohan, Please contact a...@confluent.io and she can arrange this. thanks, Robin. -- Robin Moffatt | Developer Advocate | ro...@confluent.io | @rmoff On Fri, 26 Oct 2018 at 18:26, Rohan Rasane wrote: > Hi, > What is the email address, if I want to add ServiceNow to the p

Re: Magic byte error when trying to consume Avro data with Kafka Connect

2018-12-07 Thread Robin Moffatt
ned -- Robin Moffatt | Developer Advocate | ro...@confluent.io | @rmoff On Thu, 6 Dec 2018 at 18:30, Marcos Juarez wrote: > We're trying to use Kafka Connect to pull down data from Kafka, but we're > having issues with the Avro deserialization. > > When we attempt to consume data

Re: Problem with Kafka Connector

2018-12-06 Thread Robin Moffatt
If the properties are not available per-connector, then you will have to set them on the worker and have independent Kafka Connect clusters delineated by connector requirements. So long as you configure the ports not to clash, there's no reason these can't exist on the same host. -- Robin

Re: Kafka data log timestamp

2019-01-10 Thread Robin Moffatt
{"uid":2,"name":"Nick","locale":"en_US","address_city":"Palo Alto","elite":"G"} Timestamp: 1520618381093Partition: 0Offset: 1 -- -- Robin Moffatt | Developer Advocate | ro...@confluent.io | @rmoff On Thu,

Re: Kafka servers starting and stopping abnormally

2019-01-14 Thread Robin Moffatt
It would help if you provide a bit more detail. For example, you mention an OS upgrade - but what OS? from what version to what version? Do you see any errors in the broker logs at the time at which they shut down? -- Robin Moffatt | Developer Advocate | ro...@confluent.io | @rmoff On Mon

Re: Need info

2018-09-14 Thread Robin Moffatt
-connect-s3/). There are also options for pulling from the mainframe, depending on specific platform and system. -- Robin Moffatt | Developer Advocate | ro...@confluent.io | @rmoff On Wed, 12 Sep 2018 at 08:32, Chanchal Chatterji < chanchal.chatte...@infosys.com> wrote: > Hi, > >

Re: How to config the servers when Kafka cluster is behind a NAT?

2018-09-17 Thread Robin Moffatt
This should help: https://rmoff.net/2018/08/02/kafka-listeners-explained/ -- Robin Moffatt | Developer Advocate | ro...@confluent.io | @rmoff On Mon, 17 Sep 2018 at 12:18, XinYi Long wrote: > Hello, Guys, > > I have a Kafka cluster, which is behind a NAT. > > I ha

Re: Kafka Connect Skip Messages

2019-01-25 Thread Robin Moffatt
-transformations-not-transformations-youre-looking/ You might also want to look at Kafka Streams for applying logic to a stream before you ingest it with Kafka Connect. -- Robin Moffatt | Developer Advocate | ro...@confluent.io | @rmoff On Fri, 25 Jan 2019 at 13:29, Sönke Liebau wrote: >

Re: Streaming Data

2019-04-10 Thread Robin Moffatt
/streams/interactivequeries/kafkamusic/KafkaMusicExample.java Depending on the processing you want to do KSQL could also be useful, writing the results of queries (e.g. spotting exceptions in the data) to a Kafka topic that can be queried by your app. -- Robin Moffatt | Developer Advocate | ro

Re: Kafka Connect - HDFS or FileStream

2019-05-12 Thread Robin Moffatt
that can be done in a better way. -- Robin Moffatt | Developer Advocate | ro...@confluent.io | @rmoff On Mon, 13 May 2019 at 01:35, Vinay Jain wrote: > Hi > > I would like to consume a Topic and save the AVRO messages in local > directory in the AVRO file formats. As per Kafka Connect

Re: quick start does not work...

2019-05-23 Thread Robin Moffatt
Hi Tom, Can you link to which quickstart instructions you're following? thanks. -- Robin Moffatt | Developer Advocate | ro...@confluent.io | @rmoff On Thu, 23 May 2019 at 07:35, Tom Kwong wrote: > Hi, > I’m getting the following error (NoNode for /brokers) on a fresh new >

Re: Kafka Connect issues when running in Docker

2019-07-19 Thread Robin Moffatt
Do you get any output from *docker logs*? And does it work if you don't use authentication? How about if you try one of the dockerised Kafka Connect examples here? https://github.com/confluentinc/demo-scene/tree/master/kafka-connect-zero-to-hero -- Robin Moffatt | Senior Developer Advocate

Re: Install kafka-connect-storage-cloud

2019-11-12 Thread Robin Moffatt
API after starting the worker. See https://rmoff.dev/berlin19-kafka-connect for examples and discussion -- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff On Tue, 12 Nov 2019 at 16:40, Miguel Silvestre wrote: > Hi, > > I'm new to kafka (really newbie) and I

Re: help for regarding my question

2019-11-15 Thread Robin Moffatt
We can try, but you'll have to tell us what the problem is :) This is a good checklist for asking a good question: http://catb.org/~esr/faqs/smart-questions.html#beprecise -- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff On Thu, 14 Nov 2019 at 05:04, prashanth sri

Re: Powered By: add BookingSync and Smily

2019-12-06 Thread Robin Moffatt
Hi Karol, Thanks for submitting these - I'll arrange for a PR to be created to add them to the page. -- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff On Fri, 6 Dec 2019 at 08:50, Karol Galanciak wrote: > Hi, > > Could you please add BookingSy

Re: Which container should you use when deploying on docker ?

2019-12-16 Thread Robin Moffatt
<https://hub.docker.com/r/confluentinc/cp-kafka/> -- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff On Sat, 14 Dec 2019 at 04:18, Yu Watanabe wrote: > Hello. > > I would like to ask question related to kafka on docker engine. > Which container shoul

Re: Monitor Kafka connect jobs

2019-10-25 Thread Robin Moffatt
| sort Which returns: source | source-debezium-mysql-02 | RUNNING | RUNNING | io.debezium.connector.mysql.MySqlConnector -- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff On Thu, 24 Oct 2019 at 15:52, KhajaAsmath Mohammed wrote: > Hi, > > > We are using

Re: kafka connect incoming port

2019-11-26 Thread Robin Moffatt
Kafka Connect workers listen on the port defined in *rest.port* in the worker configuration - typically 8083. -- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff On Mon, 25 Nov 2019 at 18:41, Johnson, Paul wrote: > Hello, > > I am working on setting u

Re: Ksql with schema registry

2019-10-07 Thread Robin Moffatt
You can specify VALUE_FORMAT='AVRO' to use Avro serialisation and the Schema Registry. See docs for more details https://docs.confluent.io/current/ksql/docs/installation/server-config/avro-schema.html -- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff On Sat, 5 Oct

Re: Some questions about copyright

2020-02-24 Thread Robin Moffatt
Hi Michael, You can find the ASF trademark policies here: https://www.apache.org/foundation/marks/#books -- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff On Sun, 23 Feb 2020 at 19:47, Michi Bertschi wrote: > Hello > I'm Michael > > > I am w

Re: User interface for kafka: Management/administration

2020-01-23 Thread Robin Moffatt
list of projects to look at: https://dev.to/guyshiloizy/kafka-administration-and-monitoring-ui-tools-hf4 -- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff On Thu, 23 Jan 2020 at 04:10, Sunil CHAUDHARI wrote: > Hi all, > Please help me to get some user int

Re: connect to bitnami kafka cluster from djang app-engine

2020-02-07 Thread Robin Moffatt
The error you get puts you on the right lines: > Is your advertised.listeners (called advertised.host.name before Kafka 9) correct and resolvable? This article explains why and how to fix it: https://rmoff.net/2018/08/02/kafka-listeners-explained/ -- Robin Moffatt | Senior Develo

Re: Kafka compatibility metrics

2020-02-18 Thread Robin Moffatt
Hi, Can you be a bit more specific about what you're looking for? thanks, Robin. -- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff On Tue, 18 Feb 2020 at 00:28, Ajith Kumar Rajendran wrote: > Hi team, > > Need a kafka compatibility metrics details &g

Re: ingesting web server logs, or log4j log files from a boss server

2020-01-15 Thread Robin Moffatt
/crunch19-zero-to-hero-kafka-connect -- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff On Wed, 15 Jan 2020 at 03:43, George wrote: > Hi all. > > Please advise, a real noob here still, unpacking how the stack still > works... > > if I have a mySQL serve

Re: Public IPs for brokers

2020-01-15 Thread Robin Moffatt
If you use a load balancer bear in mind the importance of the advertised.listeners setting: https://rmoff.net/2018/08/02/kafka-listeners-explained/ -- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff On Wed, 15 Jan 2020 at 08:15, Himanshu Shukla wrote

Re: ingesting web server logs, or log4j log files from a boss server

2020-01-15 Thread Robin Moffatt
If spooldir doesn't suit, there's also https://github.com/streamthoughts/kafka-connect-file-pulse to check out. Also bear in mind tools like filebeat from Elastic support Kafka as a target. -- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff On Wed, 15 Jan 2020 at 12

Re: Any available cassandra source connector for kafka

2020-01-15 Thread Robin Moffatt
Pretty sure WePay are working on this: https://github.com/debezium/debezium-incubator -- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff On Wed, 15 Jan 2020 at 13:55, Sachin Mittal wrote: > Hi, > I was looking for any cassandra source connector to read dat

Re: Stand alone connector SW deployment on a web server

2020-01-16 Thread Robin Moffatt
Not sure what you mean by "SQ", but Kafka Connect workers can indeed be deployed on their own, connecting to a remote Kafka cluster. Standalone would make sense in that case, yes. -- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff On Thu, 16 Jan 2020 at 03:

Re: Free Kafka Stream Data

2020-01-08 Thread Robin Moffatt
://www.confluent.io/blog/easy-ways-generate-test-data-kafka/ -- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff On Wed, 8 Jan 2020 at 20:42, cool girl wrote: > Hi , > > I am trying to learn Kafka. Is there any free API which I can use like > twitter? I created twitter accou

Re: Adding new connector Jars dynamically in Kafka Connect

2020-04-14 Thread Robin Moffatt
No. You have you restart the worker. -- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff On Tue, 7 Apr 2020 at 18:24, nitin agarwal wrote: > Hi, > > I have a use case where new connectors will keep on adding to existing > running Kafka Connect cluster.

Re: Sybase Connection

2020-04-20 Thread Robin Moffatt
if Sybase supports JDBC then you can probably use the JDBC Source connector for Kafka Connect Ref: https://www.confluent.io/blog/kafka-connect-deep-dive-jdbc-source-connector/ -- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff On Mon, 20 Apr 2020 at 17:31, Jonatan

New blog post: A quick and dirty way to monitor data arriving on Kafka

2020-04-17 Thread Robin Moffatt
I thought I'd share here a short blog I wrote up recently: https://rmoff.net/2020/04/16/a-quick-and-dirty-way-to-monitor-data-arriving-on-kafka/ Enjoy :) -- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff

Confluent Virtual User Group (VUG)

2020-04-17 Thread Robin Moffatt
( https://airtable.com/shrzKAKFMcfIgFVK9/iCal?timeZone=Europe%2FLondon=en-gb) or an online view here (https://cnfl.io/online-community-meetups) -- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff

Re: How to disable OPTIONS method in confluent-control-center

2020-03-31 Thread Robin Moffatt
You probably want the Confluent Platform mailing list for this: https://groups.google.com/forum/#!forum/confluent-platform (or Confluent Platform slack group: http://cnfl.io/slack with the #control-center channel). Or if you have a Confluent support contract, contact support :) -- Robin

Re: JDBC source connector to Kafka topic

2020-04-29 Thread Robin Moffatt
How are you running Kafka? Do you mean when you shut it down you have to reconfigure the connector? -- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff On Wed, 29 Apr 2020 at 20:03, vishnu murali wrote: > Hi guys > I am trying that JDBC source connector to ge

Re: Connector For MirrorMaker

2020-05-01 Thread Robin Moffatt
Are you talking about MirrorMaker 2? That runs as a connector. If not, perhaps you can clarify your question a bit as to what it is you are looking for. -- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff On Fri, 1 May 2020 at 13:57, vishnu murali wrote: > Hi G

Re: JDBC Sink Connector

2020-05-07 Thread Robin Moffatt
://www.youtube.com/watch?v=b-3qN_tlYR4=627s -- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff On Thu, 7 May 2020 at 06:48, vishnu murali wrote: > Hey Guys, > > i am working on JDBC Sink Conneector to take data from kafka topic to > mysql. > > i am having

Re: Write to database directly by referencing schema registry, no jdbc sink connector

2020-05-11 Thread Robin Moffatt
> wirite to target database. I want to use self-written java code instead of kafka jdbc sink connector. Out of interest, why do you want to do this? Why not use the JDBC sink connector (or a fork of it if you need to amend its functionality)? -- Robin Moffatt | Senior Developer Advoc

Re: JDBC SINK SCHEMA

2020-05-11 Thread Robin Moffatt
You can use Apache Kafka as you are currently using, and just deploy Schema Registry alongside it. -- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff On Sat, 9 May 2020 at 02:16, Chris Toomey wrote: > You have to either 1) use one of the Confluent serializ

Re: JDBC Sink Connector

2020-05-11 Thread Robin Moffatt
Schema Registry and its serde libraries are part of Confluent Platform, licensed under Confluent Community Licence ( https://www.confluent.io/confluent-community-license-faq/) -- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff On Fri, 8 May 2020 at 13:39, vishnu

Re: JDBC SINK SCHEMA

2020-05-11 Thread Robin Moffatt
components as is. When you configure Schema Registry you point it at your existing Apache Kafka cluster ( https://docs.confluent.io/current/schema-registry/installation/config.html#schemaregistry-config ) -- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff On Mon, 11 May 2020

Re: Problem in Docker

2020-05-15 Thread Robin Moffatt
Check this out: https://rmoff.dev/fix-jdbc-driver-video (specifically here for Docker instructions https://www.youtube.com/watch?v=vI_L9irU9Pc=665s) -- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff On Fri, 15 May 2020 at 08:16, vishnu murali wrote: > Hi i

Re: Jdbc Source Connector Config

2020-05-13 Thread Robin Moffatt
-thereabouts-part-3/ - https://www.confluent.io/blog/kafka-connect-deep-dive-jdbc-source-connector/ - http://rmoff.dev/ksldn19-kafka-connect -- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff On Wed, 13 May 2020 at 10:34, vishnu murali wrote: > Hi Guys, > > i

Re: JDBC source connector

2020-05-14 Thread Robin Moffatt
If you just want it once then delete the connector once it's processed all the data -- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff On Thu, 14 May 2020 at 16:14, vishnu murali wrote: > Thanks Liam > > But I am asking like assume I am having 10.

Re: I'm trying to connect to a kafka broker running in AWS EKS (from outside the EKS cluster).

2020-03-19 Thread Robin Moffatt
You need to make sure you've configured your listeners & advertised listeners correctly. This should help: https://rmoff.net/2018/08/02/kafka-listeners-explained/ -- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff On Thu, 19 Mar 2020 at 01:49, Dan Hill w

Re: Connect - Membership Protocol

2020-03-18 Thread Robin Moffatt
-kafka-connect-workers/ -- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff On Tue, 17 Mar 2020 at 17:58, mandeep gandhi wrote: > Gentle reminder. > > + users. > > On Mon, Mar 16, 2020 at 11:27 PM mandeep gandhi > wrote: > > > Hi, > &

Re: Kafka Connect Connector Tasks Uneven Division

2020-05-20 Thread Robin Moffatt
available. -- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff On Wed, 20 May 2020 at 16:39, Deepak Raghav wrote: > Hi Robin > > Replying to your query i.e > > One thing I'd ask at this point is though if it makes any difference where > the tasks execut

Re: Persist Kafka Topics and ksqldb

2020-05-19 Thread Robin Moffatt
You need to externalise your container data stores. Here's an example Docker Compose that does that: https://github.com/confluentinc/demo-scene/blob/master/wifi-fun/docker-compose.yml <https://github.com/confluentinc/demo-scene/blob/master/wifi-fun/docker-compose.yml#L10> -- Robin M

Re: Exception in SFTP CSV SOURCE

2020-05-19 Thread Robin Moffatt
Hi Vishnu, I think there is a problem with your email client, it's just sent a duplicate of each of your emails from yesterday? thanks, Robin. On Tue, 19 May 2020 at 16:44, vishnu murali wrote: > Hi Guys > > By Trying SFTP CSV SOURCE i am getting this exception by using this > configuration.

Re: Kafka Connect Connector Tasks Uneven Division

2020-05-21 Thread Robin Moffatt
an actual one. If this is an actual problem that you're encountering and need a solution to then since the task allocation is not deterministic it sounds like you need to deploy separate worker clusters based on the workload patterns that you are seeing and machine resources available. -- Robin Moffatt | Sen

Re: User ID / Password based authentication for Kafka Connect Source connector REST Interface

2020-05-22 Thread Robin Moffatt
See here: https://docs.confluent.io/current/security/basic-auth.html#basic-auth-kconnect -- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff On Fri, 22 May 2020 at 15:42, wrote: > Hi Team, > > I have configured a Kafka Connect source connector on Distribut

Re: KSQL Enrichment issue with converting decimal to string column

2020-09-08 Thread Robin Moffatt
Why is quantity a STRING? Should it be a numeric? -- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff On Tue, 8 Sep 2020 at 08:52, Rainer Schamm wrote: > Hi all > > we are struggling a bit with a fairly simple stream that is supposed to > map columns

Re: Amazon MSK Feeback

2020-09-07 Thread Robin Moffatt
, Robin. -- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff On Mon, 7 Sep 2020 at 10:00, Himanshu Shukla wrote: > Hi, > We are planning to go for amazon MSK instead of having our own self > launched cluster on bare EC2 machines. > > It would be very help

Re: [VOTE] KIP-657: Add Customized Kafka Streams Logo

2020-08-19 Thread Robin Moffatt
I echo what Michael says here. Another consideration is that logos are often shrunk (when used on slides) and need to work at lower resolution (think: printing swag, stitching socks, etc) and so whatever logo we come up with needs to not be too fiddly in the level of detail - something that I

Re: Kafka Connect Connector Tasks Uneven Division

2020-05-26 Thread Robin Moffatt
The KIP for the current rebalancing protocol is probably a good reference: https://cwiki.apache.org/confluence/display/KAFKA/KIP-415:+Incremental+Cooperative+Rebalancing+in+Kafka+Connect -- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff On Tue, 26 May 2020 at 14:25

Re: How to manually start ingesting in kafka source connector ?

2020-05-28 Thread Robin Moffatt
When you create the connector, it will start. -- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff On Thu, 28 May 2020 at 04:12, Yu Watanabe wrote: > Dear community . > > I would like to ask question related to source connector in kafka > co

Re: How to manually start ingesting in kafka source connector ?

2020-05-28 Thread Robin Moffatt
gt; > Thanks, > Yu > > On Thu, May 28, 2020 at 6:01 PM Robin Moffatt wrote: > > > > When you create the connector, it will start. > > > > > > -- > > > > Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff > > &g

Re: Kafka Connect Connector Tasks Uneven Division

2020-05-20 Thread Robin Moffatt
to you (if so please describe why), or if you're just interested in the theory behind the rebalancing protocol? -- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff On Wed, 20 May 2020 at 06:46, Deepak Raghav wrote: > Hi > > Please, can anybod

Re: Kafka Connect Connector Tasks Uneven Division

2020-05-20 Thread Robin Moffatt
difference where the tasks execute? The point of a cluster is that Kafka Connect manages the workload allocation. If you need workload separation and guaranteed execution locality I would suggest separate Kafka Connect distributed clusters. -- Robin Moffatt | Senior Developer Advocate | ro

Re: Problem in reading From JDBC SOURCE

2020-07-02 Thread Robin Moffatt
Check out this article where it covers decimal handling: https://www.confluent.io/blog/kafka-connect-deep-dive-jdbc-source-connector/#bytes-decimals-numerics -- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff On Thu, 2 Jul 2020 at 13:54, vishnu murali wrote: >

Re: handle multiple requests

2020-06-10 Thread Robin Moffatt
You might also find this resource useful in general: https://www.confluent.io/whitepaper/comparing-confluent-platform-with-traditional-messaging-middleware/ -- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff ‪On Wed, 10 Jun 2020 at 16:00, ‫נתי אלמגור‬‎ wrote

Re: Kafka - entity-default Vs entity-name

2020-06-12 Thread Robin Moffatt
If you're going to cross-post, it's useful to include where else you've asked this before, so that people can see what else has been suggested and answered to avoid duplicating their effort :) https://stackoverflow.com/questions/62343035/kafka-entity-default-vs-entity-name -- Robin Moffatt

Re: MONGODB KAFKA CONNECT

2020-07-28 Thread Robin Moffatt
For MongoDB connectors see https://www.confluent.io/hub/#mongodb -- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff On Mon, 27 Jul 2020 at 18:19, Milka Wafula wrote: > Hello, > > Please how may we connect mongoDB with kafka &

Announcing #TwelveDaysOfSMT

2020-12-09 Thread Robin Moffatt
/ Reply here with any feedback, any particular SMT you'd like me to explore, or scenarios to solve :) -- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff

Re: Announcing #TwelveDaysOfSMT

2020-12-10 Thread Robin Moffatt
Here's the second instalment of  #TwelveDaysOfSMT   Day 2: ValueToKey and ExtractField : https://rmoff.net/2020/12/09/twelve-days-of-smt-day-2-valuetokey-and-extractfield/ -- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff On Wed, 9 Dec 2020 at 09:07, Robin

Re: Announcing #TwelveDaysOfSMT

2020-12-14 Thread Robin Moffatt
 #TwelveDaysOfSMT   Day 5️⃣: MaskField https://rmoff.net/2020/12/14/twelve-days-of-smt-day-5-maskfield/ -- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff On Fri, 11 Dec 2020 at 17:14, Robin Moffatt wrote: >  #TwelveDaysOfSMT   > > Day 4️⃣: Re

Re: Announcing #TwelveDaysOfSMT

2020-12-10 Thread Robin Moffatt
Here's the third instalment of  #TwelveDaysOfSMT   Flatten :https://rmoff.net/2020/12/10/twelve-days-of-smt-day-3-flatten/ -- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff On Thu, 10 Dec 2020 at 09:49, Robin Moffatt wrote: > Here's the second instalm

Re: Announcing #TwelveDaysOfSMT

2020-12-11 Thread Robin Moffatt
 #TwelveDaysOfSMT   Day 4️⃣: RegexRouter https://rmoff.net/2020/12/11/twelve-days-of-smt-day-4-regexrouter/ -- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff On Thu, 10 Dec 2020 at 17:56, Robin Moffatt wrote: > Here's the third instalm

Re: Announcing #TwelveDaysOfSMT

2020-12-15 Thread Robin Moffatt
/ thanks, Robin. -- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff On Mon, 14 Dec 2020 at 16:30, Ryanne Dolan wrote: > Robin, I don't think this is an appropriate use of the user group. > > Ryanne > > On Mon, Dec 14, 2020 at 10:20 AM Robin

Re: [ANNOUNCE] New Kafka PMC Member: Konstantine Karantasis

2021-06-22 Thread Robin Moffatt
Congratulations Konstantine! -- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff On Mon, 21 Jun 2021 at 16:28, Mickael Maison wrote: > Hi, > > It's my pleasure to announce that Konstantine Karantasis is now a > member of the Kafka PMC. > > Ko

Re: Mirror Maker2 Event filter capability from topic before replication

2021-05-07 Thread Robin Moffatt
There's also this https://forum.confluent.io/t/kafka-connect-jmespath-expressive-content-based-record-filtering/1104 which is worth checking out - I came across it after recording that video unfortunately but it looks like a useful predicate implementation. -- Robin Moffatt | Senior Developer

Re: print.partition=true not working

2021-02-10 Thread Robin Moffatt
Headers: %h\n' -- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff On Wed, 10 Feb 2021 at 17:59, Rovere Lorenzo wrote: > Hi > > > > We are on Kafka version: 2.2.1-kafka-4.1.0 > > We have some issues when trying to dump some messages from a topi

Re: Kafka Connect Distributed Mode Doubt

2021-03-23 Thread Robin Moffatt
still have a properties file to configure the worker itself. -- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff On Tue, 23 Mar 2021 at 02:02, Himanshu Shukla wrote: > Hi All, > > I am having these below understanding regarding Kafka connect. > > 1

Re: Confluence Control Center On Windows 10

2021-03-26 Thread Robin Moffatt
Satendra - there was a new blog published just today about running Confluent Control Center on Windows: https://www.confluent.io/blog/set-up-and-run-kafka-on-windows-and-wsl-2 -- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff On Fri, 26 Mar 2021 at 10:07, Robin

Re: Confluence Control Center On Windows 10

2021-03-26 Thread Robin Moffatt
You can use WSL2: https://www.confluent.io/blog/set-up-and-run-kafka-on-windows-linux-wsl-2 -- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff On Fri, 26 Mar 2021 at 05:09, Satendra Negi wrote: > Hello Guys, > > Is there any way to run the confluent kafk

Re: Filter Kafka Messages

2021-03-10 Thread Robin Moffatt
of it here: https://forum.confluent.io/t/kafka-connect-jmespath-expressive-content-based-record-filtering/1104 -- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff On Wed, 10 Mar 2021 at 09:55, raman gugnani wrote: > HI Tom, > > For which connector class conne

Re: Kafka Connectors output to topic.

2021-03-11 Thread Robin Moffatt
For what it's worth, that sounds less like a Kafka Connector and more like a Kafka Streams app. -- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff On Thu, 11 Mar 2021 at 18:27, Nick Siviglia wrote: > Hi Everyone, > > I'd like to create a Kafka connector tha

Re: [ANNOUNCE] New committer: Tom Bentley

2021-03-15 Thread Robin Moffatt
Congratulations Tom! -- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff On Mon, 15 Mar 2021 at 18:00, Mickael Maison wrote: > Hi all, > > The PMC for Apache Kafka has invited Tom Bentley as a committer, and > we are excited to announce that he accep

Re: Docker images

2021-03-16 Thread Robin Moffatt
There is a similar discussion about Kafka Connect images here: https://issues.apache.org/jira/browse/KAFKA-9774 -- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff On Tue, 16 Mar 2021 at 16:04, Francois Papon wrote: > Hi, > > It would be nice to push an

Loading delimited data into Apache Kafka - quick & dirty (but effective)

2021-03-01 Thread Robin Moffatt
I wrote a blog about loading data into Apache Kafka using kafkacat. It's kinda hacky, but it works and was an interesting journey through some other bash tools too :D https://rmoff.net/2021/02/26/loading-delimited-data-into-kafka-quick-dirty-but-effective/ -- Robin Moffatt | Senior Developer

Re: [ANNOUNCE] New Kafka PMC Member: Randall Hauch

2021-04-19 Thread Robin Moffatt
Great news, congratulations Randall! -- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff On Sat, 17 Apr 2021 at 00:45, Matthias J. Sax wrote: > Hi, > > It's my pleasure to announce that Randall Hauch in now a member of the > Kafka PMC. > > Randal

Re: Kafka Security Questions

2021-02-16 Thread Robin Moffatt
You can read about the security available in Apache Kafka here: https://kafka.apache.org/documentation/#security_ssl -- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff On Tue, 16 Feb 2021 at 15:18, Jones, Isaac wrote: > Hello, > > I have a couple important

Kafka Summit is this week

2021-09-13 Thread Robin Moffatt
Kafka Summit Americas 2021 starts tomorrow - register for free at https://www.myeventi.events/kafka21/na/ and tune in for tons of great Apache Kafka content! -- Robin Moffatt | Staff Developer Advocate | ro...@confluent.io | @rmoff

Kafka Summit London 2022 - Call for Papers closes soon

2021-12-07 Thread Robin Moffatt
feel free to contact me directly - ro...@confluent.io. thanks, Robin Moffatt P.S. Here’s a little-known fact about CfPs: at this stage, you don’t need to have written the talk itself - just the abstract. If your talk is accepted then you need to write it :) [1] https://www.kafka-summit.org/ [2

Re: using cluster Kafka with a F5 Vip

2022-02-22 Thread Robin Moffatt
I *think* it should work - you just need to get your advertised.listeners set up correctly. I wrote a blog about this that should help you understand the config more: www.confluent.io/blog/kafka-client-cannot-connect-to-broker-on-aws-on-docker-etc/ -- Robin Moffatt | Staff Developer Advocate

Re: Kafka Connect - offset.storage.topic reuse across clusters

2022-03-30 Thread Robin Moffatt
Hi Jordan, Is there a good reason for wanting to do this? I can think of multiple reasons why you shouldn't do this even if technically it works in some cases. Or it's just curiosity as to whether you can/should? thanks, Robin. -- Robin Moffatt | Principal Developer Advocate | ro

[ANNOUNCE] Call for Speakers is open for Current 2022: The Next Generation of Kafka Summit

2022-05-24 Thread Robin Moffatt
aker-at-current-2022-the-next-kafka-summit/. If you have any questions about submitting I would be pleased to answer them - you can contact me directly at ro...@confluent.io. The call for speakers closes at June 26, 23:59 CT. Thanks, Robin Moffatt Program Committee Chair

Re: Kafka Question

2022-05-03 Thread Robin Moffatt
-transformations -- Robin Moffatt | Principal Developer Advocate | ro...@confluent.io | @rmoff On Tue, 3 May 2022 at 03:09, Liam Clarke-Hutchinson wrote: > Hi Emily, > > Nope, Kafka doesn't have that capability built in, it's just a distributed > log that's great for streaming events. Howe