tzulitai opened a new pull request #193: URL: https://github.com/apache/flink-statefun/pull/193
The end result of this PR is: - Remote functions may now specify the typename (of format `<namespace>/<name>`, e.g. `com.foo.bar/my-type`) of the types of their state. This is specified via a returned `PersistedValueSpec` from the functions. - On the runtime side, we use this information to detect any mismatches of the type across registrations. - The typename is also persisted in snapshots via Flink's serializer snapshots, so that the mismatch detection can also be done across StateFun cluster restores. To allow incremental development without lock-stepping also updating the SDKs at the same time, the current implementation leaves room for the type to be left unset by SDKs, but eventually by the time we release, all SDKs should be updated to always set the state type typename when responding with a state spec. --- ## Changes - 4fe1efc Some preliminary changes for cleaning up `StateBinder` interface and logic. - d50c1bd Introduces the `RemotePersistedValue` state class in the SDK. A `RemotePersistedValue` directly maps to a single registered state value in remote functions. In contrast to the old `PersistedValue` class, instead of accepting a `Class<T>` as the type of the state, `RemotePersistedValue` accepts a `TypeName` to represent the state type, and always just work against bytes. - 5978253 Implements the `TypeInformation`, `TypeSerializer`, `TypeSerializerSnapshot` that will be used specifically for `RemotePersistedValue` states. It is essentially a primitive byte array serializer, with the addition of a `TypeName` configuration that's sole purpose is to be persisted in the serializer snapshot. - 49efb7a and cc7ff2f Wires everything together to enable the usage of the new `RemotePersistedValue` construct in functions. - 7655b5c Extends the request-reply protocol to actually contain state types in `PersistedValueSpec` - 9a04e9a And finally, the final result of using the new `RemotePersistedValue` in the `PersistedRemoteFunctionValues` class. Essentially, this replaces the original usages of `PersistedValue<byte[]>` with `RemotePersistedValue`, and using the new information to throw meaningful exceptions where a type mismatch is found. --- ## Testing the change Various UT was added to cover the new functionality around registering a `RemotePersistedValue`. Also, a new UT for `PersistedRemoteFunctionsValues` was added to cover the contract that an exception is thrown if the typename of state types was changed across multiple registrations. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
