adutra commented on code in PR #4648:
URL: https://github.com/apache/polaris/pull/4648#discussion_r3395637278
##########
runtime/service/src/main/java/org/apache/polaris/service/events/PolarisEventListeners.java:
##########
@@ -136,4 +165,55 @@ private void deliverEvent(
public boolean hasListeners(PolarisEventType eventType) {
return eventTypesWithListeners.contains(eventType);
}
+
+ private static class ListenerExecutor implements Executor {
+
+ private final Executor delegate;
+ private final Queue<Runnable> queue;
+ private final AtomicBoolean draining = new AtomicBoolean(false);
+
+ ListenerExecutor(Executor delegate, int capacity) {
+ this.delegate = delegate;
+ this.queue =
+ capacity == -1 ? new ConcurrentLinkedQueue<>() : new
ArrayBlockingQueue<>(capacity);
+ }
+
+ @Override
+ public void execute(@NonNull Runnable task) {
+ if (!queue.offer(task)) {
+ throw new RejectedExecutionException("Event listener queue is full");
+ }
+ if (draining.compareAndSet(false, true)) {
+ try {
+ delegate.execute(this::drain);
+ } catch (Throwable t) {
+ draining.set(false);
+ throw t;
+ }
+ }
+ }
+
+ private void drain() {
+ try {
+ Runnable task;
+ while ((task = queue.poll()) != null) {
+ task.run();
+ }
+ } finally {
+ draining.set(false);
+ // A producer may have enqueued a task between the last poll() and the
set(false) above,
+ // and seen draining=true so it did not schedule a drain. Catch that
here.
+ if (!queue.isEmpty() && draining.compareAndSet(false, true)) {
+ try {
+ // re-schedule for fairness
+ delegate.execute(this::drain);
+ } catch (Exception e) {
Review Comment:
And now `VertxAsyncExec`. Switching to this component comes with a few
caveats:
- It's shared with other parts of the application (NoSQL, OPA). Unless we
create a separate bean, we'd have to share the underlying executor. But event
delivery work may be quite intense depending on how much activity is received
and how many listeners there are (and how slow they can be).
- Using `VertxAsyncExec.schedulePeriodic()` is tempting, but would move
event delivery from a push model (deliver as soon as an event arrives) to a
pure polling mechanism. I think we should stick with
`VertxAsyncExce.schedule()`, but with a delay of zero.
Here is a version with `VertxAsyncExec`:
```java
@Override
public void execute(@NonNull Runnable task) {
if (!queue.offer(task)) {
throw new RejectedExecutionException("Event listener queue is full");
}
if (draining.compareAndSet(false, true)) {
asyncExec.schedule(this::drain, Duration.ZERO); // doesn't throw
}
}
private void drain() {
try {
Runnable task;
while ((task = queue.poll()) != null) {
task.run();
}
} finally {
draining.set(false);
if (!queue.isEmpty() && draining.compareAndSet(false, true)) {
asyncExec.schedule(this::drain, Duration.ZERO); // doesn't throw
}
}
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
