damccorm commented on PR #17782:
URL: https://github.com/apache/beam/pull/17782#issuecomment-1142469943
I'm actually starting to question whether this is a case of me using this
wrong. The runner _should_ never call split with 0.0, so we're pretty much just
looking at self-checkpoints. In that case, is it _ever_ valid to self
checkpoint without first calling `TryClaim` (@jrmccluskey). I had been trying
to do something like:
```
i := rt.GetRestriction().(offsetrange.Restriction).Start
if !isDataAvailable(i) {
return sdf.ResumeProcessingIn(5 * time.Minute)
}
for rt.TryClaim(i) {
// ...
i++
if !isDataAvailable(i) {
return sdf.ResumeProcessingIn(5 * time.Minute)
}
}
```
But I'm not sure if that's an ok thing to do, or if I should be calling:
```
i := rt.GetRestriction().(offsetrange.Restriction).Start
for rt.TryClaim(i) {
if !isDataAvailable(i) {
return sdf.ResumeProcessingIn(5 * time.Minute)
}
// ...
i++
}
```
The latter actually _seems_ cleaner, but it also leads to questions about
what we should be doing if we fail on the `TryClaim` call
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
