Re: Kafka spark structure streaming out of memory issue

2020-08-13 Thread Srinivas V
It depends on how much memory is available and how much data you are processing. Please provide data size and cluster details to help. On Fri, Aug 14, 2020 at 12:54 AM km.santanu wrote: > Hi > I am using Kafka stateless structure streaming.i have enabled watermark as > 1 > hour.after long

Kafka spark structure streaming out of memory issue

2020-08-13 Thread km.santanu
Hi I am using Kafka stateless structure streaming.i have enabled watermark as 1 hour.after long running about 2 hour my job is terminating automatically.check point has been enabled. I am doing average on input data. Can you please suggest how to avoid out of memory error -- Sent from: