Re: Interactive runner that uses flink runner can read kafka messages?

2024-03-07 Thread Jaehyeon Kim
Hello, Thanks for your reply. Does that mean every single current and future I/O connector that is supported via the multi-language pipelines framework requires its own ipython magic? It'd be a lot more convenient if key connectors are supported natively as I have really good user experience with

Re: Interactive runner that uses flink runner can read kafka messages?

2024-03-07 Thread Ning Kang via user
Python SDK's ReadFromKafka is an external transform implemented in Java, which is similar to SqlTransform. InteractiveRunner doesn't support it. That being said, if you want to implement an interactive interaction with external transforms, you may follow the workaround for SQL (

Interactive runner that uses flink runner can read kafka messages?

2024-03-06 Thread Jaehyeon Kim
Hello, I'm playing with the interactive runner on a notebook and the flink runner is used as the underlying runner. I wonder if it can read messages from Kafka. I checked the example notebook