Github user StephanEwen commented on the pull request:
https://github.com/apache/flink/pull/1239#issuecomment-146581025
Thanks for the fast and good feedback.
Concerning the removal of non-partitioned operator state:
- As per prior discussion offline, I wanted to consolidate this to a core
feature set, since it is hard to remove such features once they are released
and endorsed
- For simple states, let us add an annotation `@State` (with possible
checkpointer/serializer `@State(checkpointer=new BloomFilterCheckpointer())`)
which serves the same purpose and would be more lightweight even. I would put
that on the list for the next release.
Multiple key/value states per operator: Will re-add that (simple effort),
it is in fact very useful
Closing the state backend: Will add that, good idea
Concerning context info during checkpointing:
- You should have access to the RuntimeContext when drawing the snapshot,
which means you can access the task index. Let us also add the JobVertexID to
the RuntimeContext and these values stay deterministic across restarts (state
handles are stored under JobVertexID + task index).
- I was thinking about giving access to the state backend in the
`Checkpointed` interface methods, but let's make a design for this for the next
release.
With addressing these issues, any objections against adding this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
