Re: Accessing side input in event_time timer callback function

[Sahil Modak]

Also to confirm, I am seeing the same exception as mentioned in the
bug : java.lang.UnsupportedOperationException:
Attempt to deliver a timer to a DoFn, but timers are not supported in
Dataflow.
Due to this the timers in my code are not expiring when they are supposed to

On Wed, Jan 5, 2022 at 3:02 PM Sahil Modak 
wrote:

> Hi Luke,
>
> Please find answers below:
> 1) Number of keys in main input : this is indefinite, currently we are
> observing around ~2 million keys
> 2) How often (if ever) does the side input change : this will stay
> constant most of the time, will change once a month in the worse case
> 3) How big the side input is : this is a file of size 3kb
>
> Regards,
> Sahil
>
> On Sat, Jan 1, 2022 at 2:14 AM Luke Cwik  wrote:
>
>> This is a missing feature[1]. It looks like there was some partial
>> progress over the past two years but it is still incomplete.
>>
>> Can you provide more details about:
>> 1) Number of keys in main input
>> 2) How often (if ever) does the side input change
>> 3) How big the side input is
>>
>> 1: https://issues.apache.org/jira/browse/BEAM-6855
>> 
>>
>>
>> On Thu, Dec 30, 2021 at 2:37 AM Sahil Modak 
>> wrote:
>>
>>> Hi,
>>>
>>> We are using beam's EVENT_TIME based timers in a DoFn that operates on a
>>> KV pair in a global window.
>>>
>>> We are also providing this DoFn with a side input, however we are unable
>>> to access this side input in the callback function provided for the
>>> EVENT_TIME based timers.
>>>
>>> Is there way to access this side input in the callback function of the
>>> EVENT_TIME timers ? If not, what would be the right way to do this?
>>>
>>> Thanks,
>>> Sahil
>>>
>>>
>>>

Re: Accessing side input in event_time timer callback function

[Sahil Modak]

Hi Luke,

Please find answers below:
1) Number of keys in main input : this is indefinite, currently we are
observing around ~2 million keys
2) How often (if ever) does the side input change : this will stay constant
most of the time, will change once a month in the worse case
3) How big the side input is : this is a file of size 3kb

Regards,
Sahil

On Sat, Jan 1, 2022 at 2:14 AM Luke Cwik  wrote:

> This is a missing feature[1]. It looks like there was some partial
> progress over the past two years but it is still incomplete.
>
> Can you provide more details about:
> 1) Number of keys in main input
> 2) How often (if ever) does the side input change
> 3) How big the side input is
>
> 1: https://issues.apache.org/jira/browse/BEAM-6855
> 
>
>
> On Thu, Dec 30, 2021 at 2:37 AM Sahil Modak 
> wrote:
>
>> Hi,
>>
>> We are using beam's EVENT_TIME based timers in a DoFn that operates on a
>> KV pair in a global window.
>>
>> We are also providing this DoFn with a side input, however we are unable
>> to access this side input in the callback function provided for the
>> EVENT_TIME based timers.
>>
>> Is there way to access this side input in the callback function of the
>> EVENT_TIME timers ? If not, what would be the right way to do this?
>>
>> Thanks,
>> Sahil
>>
>>
>>