Re: Re: Operator state in New Source API

2021-12-23 Thread Yun Gao
it should be not possible to use operator state in the SplitEnumerator. Best, Yun --Original Mail -- Sender:Krzysztof Chmielewski Send Date:Thu Dec 23 19:11:55 2021 Recipients:user Subject:Re: Operator state in New Source API Thank you both, yes seems

Re: Operator state in New Source API

2021-12-23 Thread Krzysztof Chmielewski
t; > Best, > Yun > > --Original Mail -- > *Sender:*Krzysztof Chmielewski > *Send Date:*Thu Dec 23 06:33:07 2021 > *Recipients:*user > *Subject:*Operator state in New Source API > >> Hi, >> Is it possible to use managed operator state like MapStat

Re: Operator state in New Source API

2021-12-22 Thread Yun Gao
Hi Krzysztof, If I understand right, I think managed operator state might not help here since currently Flink only support in-memory operator state. Is it possible currently we first have a customized SplitEnumerator to skip the processed files in some other way? For example, if these files

Re: Operator state in New Source API

2021-12-22 Thread Yun Tang
/#using-operator-state Best, Yun Tang From: Krzysztof Chmielewski Sent: Thursday, December 23, 2021 6:32 To: user Subject: Operator state in New Source API Hi, Is it possible to use managed operator state like MapState in an implementation of new unified source

Operator state in New Source API

2021-12-22 Thread Krzysztof Chmielewski
Hi, Is it possible to use managed operator state like MapState in an implementation of new unified source interface [1]. I'm especially interested with using Managed State in SplitEnumerator implementation. I have a use case that is a variation of File Source where I will have a great number of