Hi, Nice to hear that you are trying out StateFun!
It is by design that function state is attached to each HTTP invocation request, as defined by StateFun's remote invocation request-reply protocol. This decision was made with typical application cloud-native architectures in mind - having function deployments be stateless and require no session dependencies between the StateFun runtime and the functions services allows the functions to scale out very easily. There are some discussions on potentially adding a bi-directional protocol in the future so that state can be lazily fetched on demand instead of every invocation, but that is still in very early stages of discussion. Could you briefly describe what the state access pattern in your application looks like? Maybe this can provide some insight for us in figuring out how a more advanced / efficient protocol should be designed in future releases. On Thu, Oct 8, 2020, 6:20 PM Clements, Danial C <danial.cleme...@optum.com> wrote: > Hi, > > > > In doing some testing with Flink stateful functions in Python and I’ve > gotten a small POC working. One of our key requirements for our stream > processors is that they be written in python due to the skillset of our > team. Given that the Python DataStreams api seems to be under development > in Flink 1.12, we’ve implemented our business logic as a stateful function > using the remote pattern. In some testing, it seems the state object is > getting serialized and sent along with each HTTP request > One clarification here: StateFun does not serialize or deserialize state, everything is maintained and provided to functions as byte arrays. Serialization / deserialization happens in user code (i.e. the functions). Cheers, Gordon > and given that we’re storing quite a bit of data in this state, this seems > to contribute to the latency of the application in a linear fashion. Is > there any way around this? Is there a way to store the state local to the > python application? > > > > Thanks, > > Dan > > > This e-mail, including attachments, may include confidential and/or > proprietary information, and may be used only by the person or entity > to which it is addressed. If the reader of this e-mail is not the intended > recipient or his or her authorized agent, the reader is hereby notified > that any dissemination, distribution or copying of this e-mail is > prohibited. If you have received this e-mail in error, please notify the > sender by replying to this message and delete this e-mail immediately. >
