[
https://issues.apache.org/jira/browse/BEAM-91?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15341996#comment-15341996
]
Matt Pouttu-Clarke commented on BEAM-91:
----------------------------------------
I've solved this problem successfully several times since around 2007. It
requires implementing data versioning and treating data much like you would
treat code in github. You could also call it streaming with parallel universe
support, as some consumers may not want or need your redactions, while others
may have critical need of them (much like in the source code world some users
do not want immediate "upgrades"). Also, please note that it is just as
important to support redacting structural changes as it is to support redacting
data changes. I have mature and battle tested designs in this area if there's
interest.
> Retractions
> -----------
>
> Key: BEAM-91
> URL: https://issues.apache.org/jira/browse/BEAM-91
> Project: Beam
> Issue Type: New Feature
> Components: beam-model
> Reporter: Tyler Akidau
> Assignee: Frances Perry
> Original Estimate: 672h
> Remaining Estimate: 672h
>
> We still haven't added retractions to Beam, even though they're a core part
> of the model. We should document all the necessary aspects (uncombine,
> reverting DoFn output with DoOvers, sink integration, source-level
> retractions, etc), and then implement them.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
