Hello Andre Borie, Your PoC work is very cool. I am happy that someone out there succeeded in using our IMAP command extension mechanism.
> however I have a question as to which pattern the storage & APNS long-lived client should conform to. The IMAP extension & mailbox listeners (which James has established patterns for) are only *part* of the entire solution. FYI, we (Linagora) developed the JMAP Firebase Cloud Messaging extension. The idea is basically as same as yours: - Exposing application API so users can register their notifications. In our case, it was JMAP FirebaseSubscription extension ( https://github.com/linagora/tmail-backend/blob/master/docs/modules/ROOT/pages/tmail-backend/jmap-extensions/pushWithFirebase.adoc). In your case it was the IMAP command extension. For storing and managing the subscriptions, we wrote an interface for the storage API called `FirebaseSubscriptionRepository` ( https://github.com/linagora/tmail-backend/blob/master/tmail-backend/jmap/extensions/src/main/java/com/linagora/tmail/james/jmap/firebase/FirebaseSubscriptionRepository.java). The idea is that we can have multiple implementations for the storage API e.g. memory one (mostly for test), Cassandra one (using Cassandra database), Postgres one (using Postgres database)... - Pushing the users's data change to the Push gateway using a listener. In our case it was `FirebasePushListener` ( https://github.com/linagora/tmail-backend/blob/master/tmail-backend/jmap/extensions/src/main/scala/com/linagora/tmail/james/jmap/firebase/FirebasePushListener.scala#L30), like your listener. I think your PoC is already good enough, the story is just about how to contribute it to James in a proper way. As Benoit suggested, you can contribute your extension into a new module let's say `apsn-imap`: https://github.com/apache/james-project/tree/master/third-party/apsn-imap. For the storage API, you can write a simple Java interface e.g. `ApnsTokenRepository` in a child module: https://github.com/apache/james-project/tree/master/third-party/apsn-imap/token-storage-api Then you can implement a simple implementation for the storage using memory in another child module: https://github.com/apache/james-project/tree/master/third-party/apsn-imap/token-storage-memory. Or whatever implementation you like IMO, other people who want to have a Cassandra or Postgres implementation can just base on your `ApnsTokenRepository` API. After having the storage API, you can integrate your PushClient and the listener. That is basically my idea on the topic. You can create a ticket at https://github.com/linagora/james-project/issues where we can discuss further details on implementing this if you want. Also please do not hesitate to open a draft pull request so we can give you soon feedback. Thanks for your enthusiasm. Regards, Quan Vào Th 6, 8 thg 12, 2023 vào lúc 08:47 Andre Borie <h...@rjevski.io.invalid> đã viết: > Hello, > > > is there a public document specifying this > > No explicit public specification; the closest we have are patches > published against Dovecot (which was part of macOS Server, their > discontinued server product). > > Other open-source mail server projects have used that as a specification > to develop their own solutions: > > * https://github.com/freswa/dovecot-xaps-plugin > * > https://github.com/cyrusimap/cyrus-imapd/commit/a4470bf0f79de49030b3d66d2b385a1345c1a040 > > > we added exactly one year ago support for IMAP extension > > As you saw previously I am already doing this in my proof-of-concept, > however I have a question as to which pattern the storage & APNS long-lived > client should conform to. The IMAP extension & mailbox listeners (which > James has established patterns for) are only *part* of the entire solution. > > I am very new to this project (and Java in general) so I would appreciate > if someone with more experience could stub out a general layout of the > classes I should implement and which base classes to inherit from for the > storage & long-lived APNS client. I will check out that Firebase extension > you linked but I'd appreciate any other help/opinions. > > > > all code landing in ASF james needs t be ASFV2 > > I'm open to licensing this under whatever license - the PoC I linked to > previously is licensed under whatever James is, I've copied the LICENSE > file verbatim. > > > we can not rely on a proprietary custom external APSN bridge (external > bridge is OK to me though, if FOSS and easy to run...). > > T bridge I used in my PoC is FOSS (MIT-licensed), however I am looking to > remove the dependency on it as I find its data storage model inadequate and > unable to support mailbox delegation, therefore this mailing list thread on > how to reimplement the full functionality within James itself. > > Regards. > > > > On 7 Dec 2023, at 20:13, Benoit TELLIER <btell...@apache.org> wrote: > > > > Hello Andre. > > > > Thanks for letting me discover about this Apple feature. Out of > curiosity, is there a public document specifying this? If so could you > share this link? > > > > Are you aware of other free and OpenSource mail servers implementing > this? > > > > Second, we added exactly one year ago support for IMAP extension: > implementing support for XAPPLEPUSHSERVICE can thus always be done in an > extension. Here is an example of how to do so: > https://github.com/apache/james-project/tree/master/examples/custom-imap > > > > > > > > I speak for myself - this is not something I would contribute by myself > for pleasure > > > > > First question would be whether you'd be interested in including this > in James core, or if this should remain a separate extension? > > > > If there is a publicly available specification and other FOSS mail > server implementation for this, I personnally could welcome such a effort > as a third party module that could be embedded into the IMAP stack. > > Definitvely not in James core. > > > > > If interested in upstreaming, would you be able to outline a > high-level overview of what the code should look like, especially where to > put the APNS client? > > > > > https://github.com/apache/james-project/tree/master/third-party/apsn-imap > ? > > > > We coukld define core concepts there, apsn driver, storage api, imap > module. Then separate maven modules for token storage? > > > > Like > https://github.com/apache/james-project/tree/master/third-party/apsn-memory > > > https://github.com/apache/james-project/tree/master/third-party/apsn-cassandra > > > > Somebody willing to enable the extension in James would... > > ...1 Drop the apsn-imap-xxxx jar on the classpath > > ...2 Register the guice module into extensions.properties to select his > device store. > > ...3 Register the APSN IMAP extension. > > > > > I want to know whether James has a way for extensions to attach > arbitrary data to a mailbox > > > > James extension can store data. Talk speaking about this [1] and (old) > code example [2] > > [1] https://www.youtube.com/watch?v=zr8qpNkL6U4&ab_channel=TheASF > > [2] https://github.com/chibenwa/james-blacklist > > > > > Also, where to put the long-lived APNS client? > > > > Dedicated storage for the extension > > > > > Does James have an existing extension pattern that accounts for > long-lived services like this? > > > > Yes as Tung mentionned with JMAP we did implement WebPush and a Linagora > specific firebase extension. > > We also do have websockets and eventsource up and running with JMAP. > > > > > https://github.com/Rjevski/apache-james-xapsd-registration-extension > > > > So cool! > > > > Congrats! > > > > Definitly happy to welcome this in James (as third-party), and happy to > see you did succeed to set up an IMAP extension by yourself! > > > > Last question: all code landing in ASF james needs t be ASFV2 and we can > not rely on a proprietary custom external APSN bridge (external bridge is > OK to me though, if FOSS and easy to run...). > > > > Best regards, > > > > Benoit > > > > On 07/12/2023 10:33, Tran Tung wrote: > >> Hello, > >> > >> I'm unable to answer all your questions. > >> As I know, James doesn't have a backend repository to store data > similar to your use case. > >> At Linagora, we encountered a similar situation when implementing > Firebase. Consequently, we opted to develop a dedicated repository for > storage. > >> If you want to reference the code, you can visit > https://github.com/linagora/tmail-backend/tree/master/tmail-backend/jmap/extensions/src/main/java/com/linagora/tmail/james/jmap/firebase > >> > >> Hope that help you a bit > >> > >> > >> On 07/12/2023 15:36, Andre Borie wrote: > >>> Hello and hope everyone is well. I would like your help as to how best > to implement Apple Push notification support. > >>> > >>> For a bit of context, Apple iOS does not use IMAP IDLE - instead it > uses a proprietary IMAP command "XAPPLEPUSHSERVICE" combined with Apple's > hosted push notifications service "APNS". > >>> > >>> When an iOS device connects to a supported server advertising the > "XAPPLEPUSHSERVICE" capability, the device issues an XAPPLEPUSHSERVICE IMAP > command with the following contents: > >>> > >>> * account-id: opaque blob that must be quoted in subsequent > notifications > >>> * subtopic: APNS subtopic, must be used when sending subsequent > notifications > >>> * device token: the APNS identifier of the device, must be used to > send subsequent notifications > >>> * list of mailboxes: list of IMAP mailbox names (from the perspective > of the iOS IMAP client) to monitor and send notifications upon changes > >>> > >>> The IMAP server replies with an APNS topic (mostly static value, > derived from the certificate it uses to talk to APNS). Then, when changes > occur in the aforementioned mailboxes, the server must send a notification > via APNS (a persistent connection to an Apple server authenticated by a > client certificate) towards the aforementioned device token & subtopic, > quoting the account ID & client-facing mailbox name. The notification is > relayed by Apple to the target device which prompts it to wake up and fetch > the new messages via IMAP. > >>> > >>> This would require us to: > >>> > >>> * store (device_token, account_id, subtopic, client-specific IMAP > mailbox name) tuples against a given Mailbox > >>> * react to mailbox events and enqueue notifications for each of the > above tuples > >>> * maintain a persistent connection to APNS to dequeue & forward those > notifications to > >>> > >>> First question would be whether you'd be interested in including this > in James core, or if this should remain a separate extension? If included > in core, it would require James to be able to accept another keystore (with > the APNS certificate, created by a helper program) and be able to reload it > on the fly to enable zero-downtime certificate rotations. > >>> > >>> If interested in upstreaming, would you be able to outline a > high-level overview of what the code should look like, especially where to > put the APNS client? > >>> > >>> On the other hand, if it's better to keep it as an extension, I want > to know whether James has a way for extensions to attach arbitrary data to > a mailbox (so I can store the device tokens) without changing the storage > backends? Also, where to put the long-lived APNS client? Does James have an > existing extension pattern that accounts for long-lived services like this? > >>> > >>> For reference, I currently have a proof of concept extension at > https://github.com/Rjevski/apache-james-xapsd-registration-extension > which is an IMAP extension (implementing the custom command) and mailbox > listener - however, the storage of mailbox to device token mappings and > APNS communication is delegated to an external service called over HTTP. > >>> > >>> Regards. > >>> --------------------------------------------------------------------- > >>> To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org > >>> For additional commands, e-mail: server-dev-h...@james.apache.org > >>> > >>> > >> > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org > >> For additional commands, e-mail: server-dev-h...@james.apache.org > >> > >> > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org > > For additional commands, e-mail: server-dev-h...@james.apache.org > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org > For additional commands, e-mail: server-dev-h...@james.apache.org > >
