github-actions[bot] commented on code in PR #41507:
URL: https://github.com/apache/beam/pull/41507#discussion_r4645034496
##########
sdks/java/io/amqp/src/main/java/org/apache/beam/sdk/io/amqp/AmqpIO.java:
##########
@@ -172,30 +182,195 @@ public PCollection<Message> expand(PBegin input) {
}
}
- private static class AmqpCheckpointMark implements
UnboundedSource.CheckpointMark, Serializable {
+ @VisibleForTesting
+ static class AmqpCheckpointMark implements UnboundedSource.CheckpointMark,
Serializable {
- private transient Messenger messenger;
+ // Pinned to the value computed before readerId and sequence were added,
so checkpoints written
+ // by older versions can still be decoded.
+ private static final long serialVersionUID = 6903044759462496471L;
+
+ private final @Nullable String readerId;
+ private final long sequence;
+ private transient @Nullable Messenger messenger;
private transient List<Tracker> trackers = new ArrayList<>();
+ private transient @Nullable PendingCheckpoints pendingCheckpoints;
+ private transient @Nullable AtomicInteger activeCheckpoints;
+ private transient @Nullable Runnable onCheckpointFinalized;
- public AmqpCheckpointMark() {}
+ public AmqpCheckpointMark() {
+ this(null, 0, null, new ArrayList<>(), null, null);
+ }
+
+ private AmqpCheckpointMark(
+ @Nullable PendingCheckpoints pendingCheckpoints,
+ long sequence,
+ @Nullable Messenger messenger,
+ List<Tracker> trackers,
+ @Nullable AtomicInteger activeCheckpoints,
+ @Nullable Runnable onCheckpointFinalized) {
+ this.pendingCheckpoints = pendingCheckpoints;
+ this.readerId = pendingCheckpoints == null ? null :
pendingCheckpoints.readerId;
+ this.sequence = sequence;
+ this.messenger = messenger;
+ this.trackers = trackers;
+ this.activeCheckpoints = activeCheckpoints;
+ this.onCheckpointFinalized = onCheckpointFinalized;
+ }
@Override
public void finalizeCheckpoint() {
- for (Tracker tracker : trackers) {
- // flag as not cumulative
- messenger.accept(tracker, 0);
+ PendingCheckpoints pending = pendingCheckpoints;
+ if (pending == null && readerId != null) {
+ pending = PendingCheckpoints.LIVE.getIfPresent(readerId);
}
+ if (pending != null) {
+ for (AmqpCheckpointMark checkpoint : pending.removeUpTo(sequence)) {
+ checkpoint.acknowledge();
+ }
+ return;
+ }
+ if (readerId != null) {
+ return;
+ }
+ acknowledge();
+ }
+
+ private synchronized void acknowledge() {
Review Comment:
**🟠High**: `acknowledge()` can run concurrently with
`abandonPendingCheckpointsAndStopMessenger()` (when the close timeout task
fires at the same time `finalizeCheckpoint()` is executing on another thread).
Because `pending.removeUpTo(sequence)` removes the checkpoint from
`PendingCheckpoints` before calling `checkpoint.acknowledge()`, a concurrent
`checkpointPreparer.abandonPendingCheckpoints()` call in
`abandonPendingCheckpointsAndStopMessenger()` will not see that checkpoint or
acquire its monitor, and will immediately proceed to call `messenger.stop()`
while `acknowledge()` is calling `messenger.accept(tracker, 0)`. Qpid Proton's
`Messenger` (`MessengerImpl`) is not thread-safe, so concurrent `accept` and
`stop` calls can corrupt internal state or throw.
Consider guarding `messenger.accept(...)` and `messenger.stop()` with a
shared lock (e.g., `closeLock` or synchronizing on the `Messenger` instance)
and checking that the reader/messenger has not already been stopped before
calling `accept`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]