Spark job crashing - Spark Structured Streaming with Kafka

[Sachit Murarka]

Hi All,

My spark job is crashing (Structured stream) . Can anyone help please. I am
using spark 3.0.1 with kubernetes.

[ERROR] - StreamingQueryException Exception in  query.awaitTermination()
  File "/opt/spark/python/lib/pyspark.zip/pyspark/sql/streaming.py", line
103, in awaitTermination
    return self._jsq.awaitTermination()
  File "/opt/spark/python/lib/py4j-0.10.9-src.zip/py4j/java_gateway.py",
line 1304, in __call__
    return_value = get_return_value(
  File "/opt/spark/python/lib/pyspark.zip/pyspark/sql/utils.py", line 134,
in deco
    raise_from(converted)

Kind Regards,
Sachit Murarka