[jira] [Commented] (JAMES-3687) Implements Apache Pulsar based Mailqueue
[ https://issues.apache.org/jira/browse/JAMES-3687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17770301#comment-17770301 ] Benoit Tellier commented on JAMES-3687: --- +1 > Implements Apache Pulsar based Mailqueue > > > Key: JAMES-3687 > URL: https://issues.apache.org/jira/browse/JAMES-3687 > Project: James Server > Issue Type: Sub-task > Components: Queue >Reporter: Jean Helou >Priority: Major > Time Spent: 4h 10m > Remaining Estimate: 0h > > An apache pulsar based mailqueue offers a different set of compromises over > the existing mailqueue implementations: > pros: > * pulsar is a distributed queue > * pulsar offers scheduling facilities making it easier to implement delayed > queues > cons: > * being fully distributed some consistency guarantees cannot be honored for > flush and filter since the flushing and filtering commands take time to > propagate in the cluster -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
[jira] [Commented] (JAMES-3687) Implements Apache Pulsar based Mailqueue
[ https://issues.apache.org/jira/browse/JAMES-3687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17770237#comment-17770237 ] Matthieu Baechler commented on JAMES-3687: -- I would like to treat this last issue in another task. There's already https://issues.apache.org/jira/browse/JAMES-3696 to track that if I understand correctly. Could we close this ticket? > Implements Apache Pulsar based Mailqueue > > > Key: JAMES-3687 > URL: https://issues.apache.org/jira/browse/JAMES-3687 > Project: James Server > Issue Type: Sub-task > Components: Queue >Reporter: Jean Helou >Priority: Major > Time Spent: 4h 10m > Remaining Estimate: 0h > > An apache pulsar based mailqueue offers a different set of compromises over > the existing mailqueue implementations: > pros: > * pulsar is a distributed queue > * pulsar offers scheduling facilities making it easier to implement delayed > queues > cons: > * being fully distributed some consistency guarantees cannot be honored for > flush and filter since the flushing and filtering commands take time to > propagate in the cluster -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
[jira] [Commented] (JAMES-3687) Implements Apache Pulsar based Mailqueue
[
https://issues.apache.org/jira/browse/JAMES-3687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17472650#comment-17472650
]
Jean Helou commented on JAMES-3687:
---
all these already have a wait time built in to account for the eventual
consistency of remove
* verride public void awaitRemove() \{ try { Thread.sleep(100); } catch (
* InterruptedException e) \{ throw new RuntimeException(e); } }
However as usual with fixed wait time maybe this is too short. There is no API
to observe applicable delete filters and I'm not sure it's a good idea.
Two options here:
* derive the wait time from ENV defaulting to 100ms, so the 100 can be
overriden for CI
* change all the tests assertions to use Awaitility... and wait for eventual
consistency
> Implements Apache Pulsar based Mailqueue
>
>
> Key: JAMES-3687
> URL: https://issues.apache.org/jira/browse/JAMES-3687
> Project: James Server
> Issue Type: Sub-task
> Components: Queue
>Reporter: Jean Helou
>Priority: Major
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> An apache pulsar based mailqueue offers a different set of compromises over
> the existing mailqueue implementations:
> pros:
> * pulsar is a distributed queue
> * pulsar offers scheduling facilities making it easier to implement delayed
> queues
> cons:
> * being fully distributed some consistency guarantees cannot be honored for
> flush and filter since the flushing and filtering commands take time to
> propagate in the cluster
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org
[jira] [Commented] (JAMES-3687) Implements Apache Pulsar based Mailqueue
[
https://issues.apache.org/jira/browse/JAMES-3687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17471344#comment-17471344
]
Benoit Tellier commented on JAMES-3687:
---
https://ci-builds.apache.org/job/james/job/ApacheJames/job/PR-826/7/
Pulsar mail queue bringed in some unstable tests
{code:java}
Test Result (4 failures / +4)
org.apache.james.queue.pulsar.PulsarMailQueueTest.removeByRecipientShouldRemoveSpecificEmail
org.apache.james.queue.pulsar.PulsarMailQueueTest.removeByRecipientShouldRemoveSpecificEmailWhenMultipleRecipients{List,
MailAddress}[5]
org.apache.james.queue.pulsar.PulsarMailQueueTest.removeByRecipientShouldRemoveSpecificEmailWhenMultipleRecipients{List,
MailAddress}[6]
org.apache.james.queue.pulsar.PulsarMailQueueTest.removeShouldRemoveMailFromStoreWhenFilteredOut
{code}
> Implements Apache Pulsar based Mailqueue
>
>
> Key: JAMES-3687
> URL: https://issues.apache.org/jira/browse/JAMES-3687
> Project: James Server
> Issue Type: Improvement
> Components: Queue
>Reporter: Jean Helou
>Priority: Major
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> An apache pulsar based mailqueue offers a different set of compromises over
> the existing mailqueue implementations:
> pros:
> * pulsar is a distributed queue
> * pulsar offers scheduling facilities making it easier to implement delayed
> queues
> cons:
> * being fully distributed some consistency guarantees cannot be honored for
> flush and filter since the flushing and filtering commands take time to
> propagate in the cluster
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org
[jira] [Commented] (JAMES-3687) Implements Apache Pulsar based Mailqueue
[ https://issues.apache.org/jira/browse/JAMES-3687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17470546#comment-17470546 ] Benoit Tellier commented on JAMES-3687: --- Also I tried putting an ADR for this. Review welcomed... https://github.com/apache/james-project/pull/829 > Implements Apache Pulsar based Mailqueue > > > Key: JAMES-3687 > URL: https://issues.apache.org/jira/browse/JAMES-3687 > Project: James Server > Issue Type: Improvement > Components: Queue >Reporter: Jean Helou >Priority: Major > > An apache pulsar based mailqueue offers a different set of compromises over > the existing mailqueue implementations: > pros: > * pulsar is a distributed queue > * pulsar offers scheduling facilities making it easier to implement delayed > queues > cons: > * being fully distributed some consistency guarantees cannot be honored for > flush and filter since the flushing and filtering commands take time to > propagate in the cluster -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
[jira] [Commented] (JAMES-3687) Implements Apache Pulsar based Mailqueue
[
https://issues.apache.org/jira/browse/JAMES-3687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17470544#comment-17470544
]
Benoit Tellier commented on JAMES-3687:
---
Potential bugs of the current proposal that might need to be addressed:
https://github.com/apache/james-project/pull/808#discussion_r780162174
{code:java}
Clear need to also apply to delayed messages.
We need to make this constraint explicit in the underlying contract tests.
We likely need to clear too the scheduled topic
{code}
Likely super easy to fix.
https://github.com/apache/james-project/pull/808#discussion_r780170243
{code:java}
Clearing expired filters is an optimisation relying on messages being ordered.
Out of order messages, like delayed messages, might make this trivially false.
We could benefit from a test demonstrating the issue. (contract test)
We might have to drop this optimisation.
{code}
Easy to solve.
Impact: ever growing set of deletions...
> Implements Apache Pulsar based Mailqueue
>
>
> Key: JAMES-3687
> URL: https://issues.apache.org/jira/browse/JAMES-3687
> Project: James Server
> Issue Type: Improvement
> Components: Queue
>Reporter: Jean Helou
>Priority: Major
>
> An apache pulsar based mailqueue offers a different set of compromises over
> the existing mailqueue implementations:
> pros:
> * pulsar is a distributed queue
> * pulsar offers scheduling facilities making it easier to implement delayed
> queues
> cons:
> * being fully distributed some consistency guarantees cannot be honored for
> flush and filter since the flushing and filtering commands take time to
> propagate in the cluster
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org
