[
https://issues.apache.org/jira/browse/JAMES-3700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17494080#comment-17494080
]
Jean Helou commented on JAMES-3700:
-----------------------------------
The schema way is really neat but I feel a small word of warning is required.
I strongly suggest a long experiment with at least couple schema update on non
critical data before actually enabling it in production.
I had production crashes in one of the projects where I used pulsar and avro
schemas:
On application startup for some reason my pulsar stack ended up skipping older
versions of the schema and crashed attempting to read older message with the
latest schema. I got pretty deep in the debugging but unfortunately I never got
to the bottom of the issue (as in I didn't have time to build a minimized
reproduction that I could submit as a bug to pulsar). I can't prove that the
issue was in the pulsar client event if it is my personal conviction. It may
have been a misuse (but I couldn't identify where), it may be a bug that was
fixed (it was in an older version of pulsar) but it warrants some caution :)
One of the thing I intended to try was using an external, explicit schema
registry instead of using the pulsar embedded implicit registry because I
suspected that the schema was not consistently distributed in the cluster and
connecting to the wrong node would trigger the issue.
> Dead letter policy for the Pulsar MailQueue
> -------------------------------------------
>
> Key: JAMES-3700
> URL: https://issues.apache.org/jira/browse/JAMES-3700
> Project: James Server
> Issue Type: Sub-task
> Components: pulsar, Queue
> Affects Versions: master
> Reporter: Benoit Tellier
> Priority: Major
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> Currently the Pulsar MailQueue do not come up with a dead-letter policy.
> A bad JSON payload halts the processing.
> This makes the Pulsar MailQeue brittle:
> - The ability to inject a single message with a bad payload can cause an
> entire James cluster to come to a halt.
> - Could be seen as an attack vector
> - But also any changes to the underlying JSON schema for payloads is
> susceptible to cause major downtime.
> We should define a deadletter policy:
> - Given a number of failures delivery of the message would be abandonned
> - And moved to a dead-letter topic for later audit (prevent data loss)
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
