If it helps, below is the same query progress report that I am able to
fetch from streaming query
{
"id" : "f2cb24d4-622e-4355-b315-8e440f01a90c",
"runId" : "6f3834ff-10a9-4f57-ae71-8a434ee519ce",
"name" : "query_name_1",
"timestamp" : "2019-02-27T06:06:58.500Z",
"batchId" : 3725,
Hi Experts,
I have a structured streaming query running on spark 2.3 over yarn cluster,
with below features:
- Reading JSON messages from Kafka topic with:
- maxOffsetsPerTrigger as 5000
- trigger interval of my writeStream task is 500ms.
- streaming dataset is defined as
