Are you getting any output? Streaming jobs typically run forever, and keep processing data as it comes in the input. If a streaming job is working well, it will typically generate output at a certain cadence
From: KhajaAsmath Mohammed <[email protected]> Date: Tuesday, October 20, 2020 at 1:23 PM To: "user @spark" <[email protected]> Subject: [EXTERNAL] Spark Structured streaming - Kakfa - slowness with query 0 CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe. Hi, I have started using spark structured streaming for reading data from kaka and the job is very slow. Number of output rows keeps increasing in query 0 and the job is running forever. any suggestions for this please? [cid:[email protected]] Thanks, Asmath
