chibenwa commented on pull request #831:
URL: https://github.com/apache/james-project/pull/831#issuecomment-1007949523
> in other words, removal applies on enqueue order and not on dequeue order.
Yes.
Imagine for instance a given sender uses my SaaS instance to send SPAM. I
spot the issue, dis-activate his account and purge the queue of his messages,
both ready for sending and scheduled.
Also, similarily if a given recipient start bouncing, we wmight have a lot
of messages in the queue, both without delays and with delays (retries). If the
situation is hopeless, I might want to remove all emails to that very
recipient, both scheduled and not scheduled.
> Assuming we implement the 2nd option I proposed : duplicate the filter
logic for items consumed from the scheduled topic before they are injected in
the out topic using the schedule topic offset at the time of remove as a
reference for filtering, do you still see
:+1:
>> Second problem: filter removal based on sequence number
> as a fundamental problem that needs to be solved ? I can't think of
anything but I could be missing something.
I do not understand your reply
If the reference to apply deletes is the time of enqueue, as we have delays,
these would thus not be ordered at the time of dequeue. Thus we could not craft
an heuristic relying on an ordered sequence to simplify the deletion problem
(performance versus correctness).
I do not see other solutions than keeping the filters forever (as we do not
know if a message for which filter applies is still in the queue)
- We could define an upper bound to delays, thus allowing keeing the same
strategy
- A smarter solution could be to :
- Upon delete browse the queue to find matching messages
- Build a distributed filter using the enqueueId (unique idenfier for
this email transit) + expected time of dequeue = delay + time of enqueue
- We could cleanup filters (that here would apply for individual
emails) after a threshold after the expected time of dequeue is reached. Like 1
day after the time of the dequeue.
Or a double queuing system could be used:
- one topic for emails marked as deleted (builds the filter)
- one topic for emails effectively deleted (cleans the filter)
- Or we could document the limitation and keep it for later. Thus keeping
the filters forever.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
