Hi Gordon, Ah so when it said "all registered state" that means all state keys defined in the "module.yaml", not all state for all function instances. So the expiration has always been _per_ instance then and not across all instances of a function.
Thanks for the heads up, that sounds like a good change! I definitely like the idea of putting more configuration into the SDK so that there's not two sources that have to be kept up to date. Would be neat if eventually the SDK just hosts some "/spec" endpoint that serves a list of functions and all their configuration options to Statefun on boot. Btw, I ended up also making a Scala replica of my Haskell library to use at work (some of my examples in the microsite are a bit out of date, need to revisit that): https://github.com/BlueChipFinancial/flink-statefun4s I know it seems weird to not use an embedded function, but it keeps us from having to deal with mismatched Scala versions since Flink is still on 2.12, and generally reduces friction using stuff in the Scala Cats ecosystem. Thanks, Tim On Wed, Feb 24, 2021 at 11:49 AM Tzu-Li (Gordon) Tai <tzuli...@apache.org> wrote: > Hi Timothy, > > Starting from StateFun 2.2.x, in the module.yaml file, you can set for > each individual state of a function an "expireMode" field, which values can > be either "after-invoke" or "after-write". For example: > > ``` > - function: > meta: > ... > spec: > states: > - name: state-1 > expireMode: after-write > expireAfter: 1min > - name: state-2 > expireMode: after-invoke > expireAfter: 5sec > ``` > > In earlier versions, expireMode can not be individually set for each > state. This is more flexible with 2.2.x. > > As a side note which is somewhat related, all state related configurations > will be removed from the module.yaml, instead to be defined by the language > SDKs starting from StateFun 3.0. > This opens up even more flexibility, such as zero-downtime upgrades of > remote functions which allows adding / removing state declarations without > restarting the StateFun cluster. > We're planning to reach out to the language SDK developers we know of > (which includes you for the Haskell SDK ;) ) soon on a briefing of this > change, as there is a change in the remote invocation protocol and will > require existing SDKs to be updated in order to work with StateFun 3.0. > > Cheers, > Gordon > > On Wed, Feb 24, 2021 at 11:00 PM Timothy Bess <tdbga...@gmail.com> wrote: > >> Hey, >> >> I noticed that the Flink Statefun 2.1.0 release notes had this snippet >> with regards to TTL: >> >> Note: The state expiration mode for remote functions is currently >>> restricted to AFTER_READ_AND_WRITE, and the actual TTL being set is the >>> longest duration across all registered state, not for each individual state >>> entry. This is planned to be improved in upcoming releases (FLINK-17954). >>> >> >> I noticed that the Ticket and PR for this have been closed with a >> reference to commit "289c30e8cdb54d2504ee47a57858a1d179f9a540". Does this >> mean that if I upgrade to 2.2.2 and set an expiration in my modules.yaml it >> is now "per function id" rather than across instances of said function? >> >> Thanks, >> >> Tim >> >
