Hi,
Below an update of my work to support out-band event notification in
Cyrus based on RFC 5423.
All event types together with their parameters defined in the RFC are
fully supported. Implementation is done to avoid any I/O overhead.
1) event types
MessageAppend, MessageNew
- parameters: mailboxID
- optional parameters: bodyStructure, messageContent, messages,
messageSize, modseq, service, timestamp, uidnext, vnd.cmu.midset,
vnd.cmu.newMessages
- comment: One notification is send per message. Several
notifications in case of IMAP MULTIAPPEND
MessageExpire
- parameters: mailboxID, uidset
- optional parameters: messages, modseq, service, timestamp, uidnext,
vnd.cmu.midset, vnd.cmu.newMessages
- comment: modseq may be included if referring to a specific message.
uidset if referring to a set of messages
MessageExpunge
- parameters: mailboxID, uidset
- optional parameters: messages, modseq, service, timestamp, uidnext,
vnd.cmu.midset, vnd.cmu.newMessages
- comment: modseq may be included if referring to a specific message.
uidset if referring to a set of messages
QuotaExceed, QuotaWithin
- parameters: diskQuota, diskUsed, mailboxID, maxMessages, messages
- optional parameters: service, timestamp
- comment: include diskQuota and diskUsed if quota STORAGE exceed.
maxMessages and messages if quota MESSAGE exceed. I don't support
quota ANNOTSTORAGE, however we could define proprietary parameters for
that like vnd.cmu.annotationQuota and vnd.cmu.annotationUsed.
QuotaChange
- parameters: diskQuota, mailboxID, maxMessages
- optional parameters: diskUsed, messages, service, timestamp
- comment: diskQuota and maxMessages may be included together
depending the SETQUOTA arguments. QuotaExceed, QuotaWithin and
QuotaChange are not enabled by default.
MessageRead, MessageTrash
- parameters: mailboxID, uidset
- optional parameters: messages, modseq, service, timestamp, uidnext,
vnd.cmu.midset, vnd.cmu.newMessages
- comment: MessageRead and MessageTrash replace FlagsSet for \Seen
and \Deleted flags. MessageRead notification is also send on IMAP
FETCH.
FlagsSet, FlagsClear
- parameters: mailboxID, uidset, flagNames
- optional parameters: messages, modseq, service, timestamp, uidnext,
vnd.cmu.midset, vnd.cmu.newMessages
- comment: support +FLAGS, -FLAGS and FLAGS arguments to IMAP STORE.
May send FLagsSet and FlagsClear for FLAGS. May also send MessageTrash
and MessageRead depending the flags in argument.
Login, Logout
- parameters: serverDomain, serverPort, user
- optional parameters: timestamp, service, clientIP, clientPort
- comment: Login and Logout are not enabled by default
MailboxCreate, MailboxDelete
- parameters: mailboxID
- optional parameters: service, timestamp
MailboxRename
- parameters: mailboxID, oldMailboxID
- optional parameters: service, timestamp
MailboxSubscribe, MailboxUnSubscribe
- parameters: mailboxID, user
- optional parameters: service, timestamp
and additional vnd.cmu.MessageCopy.
- parameters: mailboxID, oldMailboxID, vnd.cmu.oldUidset, uidset
- optional parameters: messages, modseq, service, timestamp, uidnext,
vnd.cmu.midset, vnd.cmu.newMessages
- comment: modseq may be included if referring to a specific message.
uidset if referring to a set of messages
2) configuration
The setting is lighter than our current code in Cyrus 2.3 with I hope
good defaults :
- eventnotifier (NULL) :
Notifyd(8) method to use for "EVENT" notifications which are
based on the RFC 5423. If not set, "EVENT" notifications are
disabled.
- event_groups ("message mailbox") :
Space-separated list of groups of related events to turn on
notification: message", "quota", "flags", "access", "mailbox"
- event_extra_params ("timestamp") :
Space-separated list of extra parameters to add to any
appropriated event : "bodyStructure", "clientAddress", "diskUsed",
"flagNames", "messageContent", "messageSize", "messages", "modseq",
"service", "timestamp", "uidnext", "vnd.cmu.host", "vnd.cmu.midset",
"vnd.cmu.newMessages"
- event_exclude_flags (NULL) :
Don't send event notification for given IMAP flag(s)
- event_exclude_folders (NULL) :
Don't send event notification for given folder(s). Set ALL for
any folder
- event_content_inclusion_mode ("standard") :
The mode in which message content may be included with
MessageAppend and MessageNew. "standard" mode is the default behavior
in which message is included up to a size with the notification. In
"message" mode, the message is included and may be truncated to a
size. In "header" mode, it includes headers truncated to a size. In
"body" mode, it includes body truncated to a size. In "headerbody"
mode, it includes full headers and body truncated to a size
- event_content_size (0) :
Truncate the message content that may be included with
MessageAppend and MessageNew. Set 0 to include the entire message
itself
- event_timestamp_format ("iso8601") :
Format event timestamp in ISO 8601 format or print the number
of seconds since the Epoch (UTC) both with fractions of second
3) message format
Notification format is JSON.
examples:
"{"event":"Login","timestamp":"2011-11-22T17:19:36.849Z","service":"imappublic","serverDomain":"127.0.0.3","serverPort":143,"clientIP":"127.0.0.1","clientPort":56197,"user":"testtmpuser"}"
"{"event":"MessageRead","timestamp":"2011-11-22T17:19:36.854Z","service":"imappublic","mailboxID":"imap:\/\/127.0.0.3\/user.testtmpuser;UIDVALIDITY=1321982376","messages":5,"vnd.cmu.newMessages":2,"uidnext":6,"uidset":"1
2","vnd.cmu.midset":"<cmu-lmtpd-3821-1321982376-0@127.0.0.3>
<cmu-lmtpd-3821-1321982376-1@127.0.0.3>"}"
"{"event":"FlagsSet","timestamp":"2011-11-22T17:19:36.854Z","service":"imappublic","mailboxID":"imap:\/\/127.0.0.3\/user.testtmpuser;UIDVALIDITY=1321982376","messages":5,"vnd.cmu.newMessages":2,"uidnext":6,"uidset":"1
2","vnd.cmu.midset":"<cmu-lmtpd-3821-1321982376-0@127.0.0.3>
<cmu-lmtpd-3821-1321982376-1@127.0.0.3>","flagNames":"\\Draft"}"
"{"event":"FlagsClear","timestamp":"2011-11-22T17:19:36.866Z","service":"imappublic","mailboxID":"imap:\/\/127.0.0.3\/user.testtmpuser;UIDVALIDITY=1321982376\/;UID=1","messages":5,"vnd.cmu.newMessages":3,"uidnext":6,"vnd.cmu.midset":"<cmu-lmtpd-3821-1321982376-0@127.0.0.3>","flagNames":"\\Seen","modseq":8}"
"{"event":"MessageTrash","timestamp":"2011-11-22T17:19:36.866Z","service":"imappublic","mailboxID":"imap:\/\/127.0.0.3\/user.testtmpuser;UIDVALIDITY=1321982376\/;UID=1","messages":5,"vnd.cmu.newMessages":3,"uidnext":6,"vnd.cmu.midset":"<cmu-lmtpd-3821-1321982376-0@127.0.0.3>","modseq":8}"
"{"event":"MessageExpunge","timestamp":"2011-11-22T17:19:36.873Z","service":"imappublic","mailboxID":"imap:\/\/127.0.0.3\/user.testtmpuser;UIDVALIDITY=1321982376\/;UID=1","messages":4,"vnd.cmu.newMessages":2,"uidnext":6,"vnd.cmu.midset":"<cmu-lmtpd-3821-1321982376-0@127.0.0.3>","modseq":9}"
"{"event":"Logout","timestamp":"2011-11-22T17:19:36.880Z","service":"imappublic","serverDomain":"127.0.0.3","serverPort":143,"user":"testtmpuser"}"
4) choices of design
- There is no event notification on internal cyrus behaviors :
rollback on mailbox creation, replication on cyrus slave, deletion of
delayed message, murder proxy, and so on.
- I wonder about the IMAP URL format of the messageID parameter. I
made use of cyrus' internal format. But I should rather use namespace
translation to get the external format.
I encounter some issues to access to namespace structure in some
places in the code (QuotaExceed in quota_db.c, MessageExpunge in
mailbox.c and so on). It may be centralized in mboxevent.c before
sending the event, but namespace dependencies are unknown here. Thus,
code to handle namespace should be rewritten :(
- I chose JSON as default format. It may not be the best choice to
implement in-band event notification like IMAP NOTIFY
I will write some documentation based on informations above.
I rebased the msgevent branch on current master :
https://github.com/worldline-messaging/cyrus-imapd/compare/msgevent
any comments or suggestions are welcome.
Thanks,
Sébastien
--
Atos Worldline
