github-actions[bot] commented on code in PR #41504:
URL: https://github.com/apache/beam/pull/41504#discussion_r4644972439
##########
sdks/java/io/solace/src/main/java/org/apache/beam/sdk/io/solace/read/UnboundedSolaceReader.java:
##########
@@ -197,15 +234,44 @@ public void close() {
}
}
+ private void abandonPendingCheckpointsOnCloseTimeout() {
+ List<PendingCheckpoint> abandoned;
+ synchronized (lock) {
+ abandoned = new ArrayList<>(pendingCheckpoints.values());
+ pendingCheckpoints.clear();
+ }
+ if (!abandoned.isEmpty()) {
+ int totalMessages = 0;
+ for (PendingCheckpoint cp : abandoned) {
+ totalMessages += cp.messages.size();
+ }
+ LOG.info(
+ "SolaceIO.Read: Reader {} closes with {} checkpoint(s) ({}
message(s)) not finalized"
+ + " within ackDeadline {}.",
+ readerUuid,
+ abandoned.size(),
+ totalMessages,
+ ackDeadline);
+ }
+ }
+
void finalizeCheckpoint(long checkpointId) {
+ if (resourcesClosed.get() || ackExecutor.isShutdown()) {
+ return;
+ }
+ // Keep the cached session alive while acknowledging pending checkpoints.
+ sessionServiceCache.getIfPresent(readerUuid);
+
List<BytesXMLMessage> messagesToAck = new ArrayList<>();
+ boolean shouldCloseAfterAck;
synchronized (lock) {
SortedMap<Long, PendingCheckpoint> toAck =
pendingCheckpoints.headMap(checkpointId, true);
for (PendingCheckpoint cp : toAck.values()) {
Review Comment:
**🟡 Medium**: If two threads call `finalizeCheckpoint` concurrently (for
example, `UnboundedSolaceSource.createReader` finalizing the checkpoint on
resume at the same time as the runner's `BundleFinalizer` callback for the same
or an earlier checkpoint), the second thread will see
`pendingCheckpoints.isEmpty()` inside the `synchronized (lock)` block while the
first thread is still executing `msg.ackMessage()` asynchronously on
`ackExecutor`.
The second thread will immediately enter the `finally` block with
`shouldCloseAfterAck == true` and call `closeResources()`, which calls
`sessionServiceCache.invalidate(readerUuid)` (closing the Solace session) while
the first thread's `ackExecutor` tasks are still in flight, causing
`msg.ackMessage()` to fail with `IllegalStateException` (session closed).
To prevent closing the session while acknowledgements are still in flight,
track in-flight ack operations (or only set `shouldCloseAfterAck` when
`!messagesToAck.isEmpty()` / when the last in-flight `finalizeCheckpoint`
completes).
##########
sdks/java/io/solace/src/main/java/org/apache/beam/sdk/io/solace/read/UnboundedSolaceReader.java:
##########
@@ -184,6 +189,38 @@ public boolean advance() {
@Override
public void close() {
+ closed.set(true);
+ boolean hasPending;
+ synchronized (lock) {
+ hasPending = !pendingCheckpoints.isEmpty();
Review Comment:
**🟠High**: `ActiveReadersRegistry` uses `.weakValues()`. Once the runner
calls `reader.close()` and drops its strong reference to the
`UnboundedSolaceReader` (e.g., on cache eviction or split), the only remaining
strong reference to `this` is the lambda scheduled in `cleanUpThread`
(`closeTimeoutFuture`).
If `close()` is called multiple times, or if GC runs when
`closeTimeoutFuture` is not holding `this`, or if `ActiveReadersRegistry` is
queried under memory pressure before `closeTimeoutFuture` is scheduled, the
weak reference in `ActiveReadersRegistry` can be cleared prematurely.
Additionally, if `receivedMessages` is non-empty when `close()` is called
(messages were read via `start()`/`advance()` but never checkpointed) and
`pendingCheckpoints` is empty, `receivedMessages` should be cleared on
`close()`.
Ensure `receivedMessages.clear()` is called on `close()`, and consider
keeping a strong reference in `ActiveReadersRegistry` (since
`ActiveReadersRegistry.unregister(readerUuid)` is now guaranteed to be called
in `closeResources()` either upon finalization or after `ackDeadline`).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]