Those are understandable. I am more interested in a few things ( and may be more that could be added )
* As far as I can understand JM is the SPOF. Does HA become a necessity ? * If there are 2 or more JM could we theoretically have a LB fronting them ? Thus it is a peer to peer access ( Cassandra ) or a master slave setup for JM HA specifically for Queryable access ( For flink jobs it is master slave ) * Do we replicate state to other TMs for read optimization ( specifically to avoid Hot Node issues ) ? * If the job goes down it seems the state is not accessible. What plans to we have to "separate concerns" for Queryable state. We consider Queryable State significant a feature Flink provides and would do the necessary leg work if there are certain gaps in it being trully considered a Highly Available Key Value store. Regards. On Mon, Mar 19, 2018 at 5:53 AM, Fabian Hueske <fhue...@gmail.com> wrote: > Hi Vishal, > > In general, Queryable State should be ready to use. > There are a few things to consider though: > > - State queries are not synchronized with the application code, i.e., they > can happen at the same time. Therefore, the Flink application should not > modify objects that have been put into or read from the state if you are > not using the RocksDBStatebackend (which creates copies by deserialization). > - State will be rolled back after a failure. Hence, you can read writes > that are not "committed by a checkpoint". > > @Kostas, did I forget something? > > Best, Fabian > > > > 2018-03-18 16:50 GMT+01:00 Vishal Santoshi <vishal.santo...@gmail.com>: > >> To be more precise, is anything thing similar to >> https://engineering.linkedin.com/blog/2018/03/air-traffic >> -controller--member-first-notifications-at-linkedin . done in Samza, can >> be replicated with production level guarantees with Flink Queryable state ( >> as it stands currently version 1.5 ) ? >> >> On Fri, Mar 16, 2018 at 5:10 PM, Vishal Santoshi < >> vishal.santo...@gmail.com> wrote: >> >>> We are making few decisions on use cases where Queryable state is a >>> natural fit https://ci.apache.org/projects/flink/flink-docs-release-1.4/ >>> dev/stream/state/queryable_state.html >>> >>> Is Queryable state production ready ? We will go to 1.5 flnk if that >>> helps to make the case for the usage. >>> >> >> >
