Hello, I'm doing POC with KafkaIO and spark runner on Azure Databricks. I'm trying to create a simple streaming app with Apache Beam, where it reads data from an Azure event hub and produces messages into another Azure event hub.
I'm creating and running spark jobs on Azure Databricks. The problem is the consumer (uses SparkRunner) is not able to read data from Event hub (queue). There is no activity and no errors on the Spark cluster. I would appreciate it if anyone could help to fix this issue. Thank you Utkarsh
