Tzu-Li (Gordon) Tai created FLINK-20265:
-------------------------------------------
Summary: Extend invocation protocol to allow functions to indicate
incomplete state context
Key: FLINK-20265
URL: https://issues.apache.org/jira/browse/FLINK-20265
Project: Flink
Issue Type: Sub-task
Components: Stateful Functions
Reporter: Tzu-Li (Gordon) Tai
Assignee: Tzu-Li (Gordon) Tai
Fix For: statefun-2.3.0
Currently, users declare the states a function will access with a module YAML
definition file. The modules are loaded once when starting a StateFun cluster,
meaning that the state specifications remain static throughout the cluster's
execution lifetime.
We propose that state specifications should be declared by the function
themselves via the language SDKs, instead of being declared in the module YAMLs.
The state specifications, now living in the functions, can be made discoverable
by the StateFun runtime through the invocation request-reply protocol.
Brief simplified sketch of the extended protocol:
- StateFun dispatches an invocation request, with states {A, B}.
- Function receives request, but since it requires {A, B, C, D}, it responds
with a IncompleteInvocationContext response indicating that state values for C,
D is missing
- StateFun receives this response, and registers new Flink state handles for
{C, D}.
- Then, a new invocation request with the same input messages, but "patched"
with new states to contain all values for {A, B, C, D} is resent to the
function.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
