e.
>>>
>>> As now with batch programming, we can do it with
>>>
>>>> df.rollup(col1,col2).count
>>>
>>>
>>> But if I try to use it with spark structured streaming state, will it
>>> store the state of all the groups as
oups as well?
>> I hope I was able to make my point clear.
>>
>> Regards
>> Amit Joshi
>>
>> On Wed, Jun 16, 2021 at 11:36 PM Mich Talebzadeh <
>> mich.talebza...@gmail.com> wrote:
>>
>>>
>>>
>>> Hi,
>>>
>
;
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>> any loss, damage or destruction of data or any other property which may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be
reciate if someone could give some pointers in the question below.
>>
>> -- Forwarded message -----
>> From: Amit Joshi
>> Date: Tue, Jun 15, 2021 at 12:19 PM
>> Subject: [Spark]Does Rollups work with spark structured streaming with
>> state.
>&
16:37, Amit Joshi wrote:
> Appreciate if someone could give some pointers in the question below.
>
> -- Forwarded message -
> From: Amit Joshi
> Date: Tue, Jun 15, 2021 at 12:19 PM
> Subject: [Spark]Does Rollups work with spark structured streaming with
> state.
Appreciate if someone could give some pointers in the question below.
-- Forwarded message -
From: Amit Joshi
Date: Tue, Jun 15, 2021 at 12:19 PM
Subject: [Spark]Does Rollups work with spark structured streaming with
state.
To: spark-user
Hi Spark-Users,
Hope you are all
Hi Spark-Users,
Hope you are all doing well.
Recently I was looking into rollup operations in spark.
As we know state based aggregation is supported in spark structured
streaming.
I was wondering if rollup operations are also supported?
Like the state of previous aggregation on the rollups are sa
