[
https://issues.apache.org/jira/browse/FLINK-20265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tzu-Li (Gordon) Tai updated FLINK-20265:
----------------------------------------
Description:
Currently, users declare the states a function will access with a module YAML
definition file. The modules are loaded once when starting a StateFun cluster,
meaning that the state specifications remain static throughout the cluster's
execution lifetime.
We propose that state specifications should be declared by the function
themselves via the language SDKs, instead of being declared in the module YAMLs.
The state specifications, now living in the functions, can be made discoverable
by the StateFun runtime through the invocation request-reply protocol.
Brief simplified sketch of the extended protocol:
- StateFun dispatches an invocation request, with states [A, B].
- Function receives request, but since it requires states [A, B, C, D], it
responds with a {{IncompleteInvocationContext}} response that indicates state
values for [C, D] is missing.
- StateFun receives this response, and registers new Flink state handles for
[C, D].
- Finally, a new invocation request with the same input messages, but "patched"
with new states to contain all values for [A, B, C, D] is resent to the
function.
was:
Currently, users declare the states a function will access with a module YAML
definition file. The modules are loaded once when starting a StateFun cluster,
meaning that the state specifications remain static throughout the cluster's
execution lifetime.
We propose that state specifications should be declared by the function
themselves via the language SDKs, instead of being declared in the module YAMLs.
The state specifications, now living in the functions, can be made discoverable
by the StateFun runtime through the invocation request-reply protocol.
Brief simplified sketch of the extended protocol:
- StateFun dispatches an invocation request, with states [A, B].
- Function receives request, but since it requires [A, B, C, D], it responds
with a {{IncompleteInvocationContext}} response that indicates state values for
[C, D] is missing.
- StateFun receives this response, and registers new Flink state handles for
[C, D].
- Finally, a new invocation request with the same input messages, but "patched"
with new states to contain all values for [A, B, C, D] is resent to the
function.
> Extend invocation protocol to allow functions to indicate incomplete state
> context
> ----------------------------------------------------------------------------------
>
> Key: FLINK-20265
> URL: https://issues.apache.org/jira/browse/FLINK-20265
> Project: Flink
> Issue Type: Sub-task
> Components: Stateful Functions
> Reporter: Tzu-Li (Gordon) Tai
> Assignee: Tzu-Li (Gordon) Tai
> Priority: Major
> Fix For: statefun-2.3.0
>
>
> Currently, users declare the states a function will access with a module YAML
> definition file. The modules are loaded once when starting a StateFun
> cluster, meaning that the state specifications remain static throughout the
> cluster's execution lifetime.
> We propose that state specifications should be declared by the function
> themselves via the language SDKs, instead of being declared in the module
> YAMLs.
> The state specifications, now living in the functions, can be made
> discoverable by the StateFun runtime through the invocation request-reply
> protocol.
> Brief simplified sketch of the extended protocol:
> - StateFun dispatches an invocation request, with states [A, B].
> - Function receives request, but since it requires states [A, B, C, D], it
> responds with a {{IncompleteInvocationContext}} response that indicates state
> values for [C, D] is missing.
> - StateFun receives this response, and registers new Flink state handles for
> [C, D].
> - Finally, a new invocation request with the same input messages, but
> "patched" with new states to contain all values for [A, B, C, D] is resent to
> the function.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
