Re: macOS Notes.app IMAP Syncing Not Working

[Aki Tuomi via dovecot]

 
 
  
   
  
  
   
On 1 June 2019 02:11 Steven Smith via dovecot <
dovecot@dovecot.org> wrote:
   
   

   
   

   
   
Thank you very much for posting this. Looking at the diffs across configurations, I tried all these options, without success:
   
   

   
   
* service dns_client block
   
   
* turn off global separator variable
   
   
* auth_mechanisms login plain gssapi cram-md5
   
   

   
   
My dovecot configuration is the common denominator of multiple Notes.app clients not working, so I’m pretty sure I’ve configured it in a way that macOS Notes.app doesn’t like.
   
   

   
   
The only other difference I see is the ldap passwd and userdb driver.
   
   

   
   
Looks like I’ll have to wait for some inspiration and free time to break out wireshark to dig further.
   
   

   
   
Thanks again,
   
   

   
   
Steve
   
  
  
   
  
  
   https://wiki2.dovecot.org/Debugging/Rawlog might be more useful than wireshark. 
   
  
  
   ---
Aki Tuomi
   
 

Re: macOS Notes.app IMAP Syncing Not Working

[Steven Smith via dovecot]

Thank you very much for posting this. Looking at the diffs across 
configurations, I tried all these options, without success:

* service dns_client block
* turn off global separator variable
* auth_mechanisms login plain gssapi cram-md5

My dovecot configuration is the common denominator of multiple Notes.app 
clients not working, so I’m pretty sure I’ve configured it in a way that macOS 
Notes.app doesn’t like.

The only other difference I see is the ldap passwd and userdb driver.

Looks like I’ll have to wait for some inspiration and free time to break out 
wireshark to dig further.

Thanks again,

Steve

Re: macOS Notes.app IMAP Syncing Not Working

[Ralph Seichter via dovecot]

* Steven Smith via dovecot:

> I’m pretty sure it arises from some subtle dovecot configuration
> setting.

Based on your statement that Dovecot does not log connection attempts by
Notes.app, I am not sure this assumption is correct.

> Would you mind posting your `doveconf -n` so that we can compare to
> the non-working and working examples above?

See attachment (I have only obfuscated directories and deleted some
settings related to authentication).

-Ralph

# 2.3.5.2 (38c8f1daf): /path/to/dovecot.conf
# Pigeonhole version 0.5.5 (2483b085)
# OS: Linux 5.1.5-gentoo x86_64 Gentoo Base System release 2.6 
auth_mechanisms = plain login
auth_username_format = %Ln
lmtp_save_to_detail_mailbox = yes
mail_location = /path/to/maildir
mail_plugins = notify
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character 
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy 
include variables body enotify environment mailbox date index ihave duplicate 
mime foreverypart extracttext vnd.dovecot.debug imapsieve vnd.dovecot.imapsieve
namespace inbox {
  inbox = yes
  location = 
  mailbox Drafts {
special_use = \Drafts
  }
  mailbox Junk {
auto = create
special_use = \Junk
  }
  mailbox Sent {
special_use = \Sent
  }
  mailbox "Sent Messages" {
special_use = \Sent
  }
  mailbox Trash {
auto = create
special_use = \Trash
  }
  prefix = 
  separator = /
}
passdb {
  args = /path/to/dovecot-ldap.conf.ext
  driver = ldap
}
plugin {
  imapsieve_mailbox1_before = file:/path/to/report-spam.sieve
  imapsieve_mailbox1_causes = COPY
  imapsieve_mailbox1_name = Junk
  imapsieve_mailbox2_before = file:/path/to/report-ham.sieve
  imapsieve_mailbox2_causes = COPY
  imapsieve_mailbox2_from = Junk
  imapsieve_mailbox2_name = *
  sieve = /path/to/sieve/%u.sieve
  sieve_extensions = +vnd.dovecot.debug
  sieve_global = /path/to/sieve/global
  sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.environment
  sieve_pipe_bin_dir = /path/to/sievepipe
  sieve_plugins = sieve_imapsieve sieve_extprograms
}
protocols = imap lmtp
service auth {
  unix_listener /path/to/dovecot-auth {
group = postfix
mode = 0660
user = postfix
  }
}
service dict {
  unix_listener dict {
mode = 0666
  }
}
service lmtp {
  unix_listener /path/to/dovecot-lmtp {
group = postfix
mode = 0660
user = postfix
  }
}
userdb {
  args = /path/to/dovecot-ldap.conf.ext
  driver = ldap
}
protocol lmtp {
  mail_plugins = notify sieve
  namespace inbox {
location = 
mailbox Trash {
  autoexpunge = 1 weeks
}
prefix = 
  }
}
protocol lda {
  mail_plugins = notify sieve
}
protocol imap {
  mail_plugins = notify imap_sieve
  namespace inbox {
location = 
mailbox Trash {
  autoexpunge = 1 weeks
}
prefix = 
  }
}

Re: macOS Notes.app IMAP Syncing Not Working

[Steven Smith via dovecot]

It does not appear that macOS Notes.app is authenticating to dovecot. I haven’t 
broken out wireshark yet, but I see in the dovecot logs that a working iOS 
access looks like this:

mail-debug.log (mailbox access):
> May 30 20:16:39 imap(pid 28792 user username): Debug: Mailbox Notes: Mailbox 
> open
ed because: SELECT

mail-info.log (solr indexing of a newly added Note):
> May 30 18:44:31 indexer-worker(pid 26589 user username): Info: Indexed 1 
> messages
 in Notes (UIDs 167..167)

There are no such log entries when I launch macOS Notes.app and toggle account 
activation in System Preferences>Internet Accounts.

It doesn’t look like macOS Notes.app is even trying to talk to dovecot.

Re: macOS Notes.app IMAP Syncing Not Working

[Steven Smith via dovecot]

* Ralph, Steven Smith via dovecot:

> > The issue is that macOS Note.app does not sync with this server.

> I'm syncing macOS Mojave's Notes.app with Dovecot without problems, so
> it can be done. Have you made sure that you activated both mail and
> notes in macOS' Internet Accounts preferences for this particular IMAP
> account? If so, is that IMAP account listed in the navigation bar of
> Notes.app (mine shows iCloud and the IMAP account)?

Thanks, yes, I’ve done all that and more, including:
* Setting the Path Prefix to blank, INBOX, INBOX., and ‘.’
* sqlite3 
~/Library/Containers/com.apple.Notes/Data/Library/Notes/NotesV7.storedata 
'select * from ZACCOUNT;'
* sqlite3 
~/Library/Containers/com.apple.Notes/Data/Library/Notes/NotesV7.storedata 
“update ZACCOUNT set ZUSERNAME='em...@example.com', 
ZHOSTNAME='imap.hostname.com',ZSERVERPATHPREFIX='INBOX.' where Z_PK=5;”
* Variations on the above
Reports of problems syncing Notes.app to a IMAP accounts are widespread.
I’m pretty sure it arises from some subtle dovecot configuration setting.
Would you mind posting your `doveconf -n` so that we can compare to the 
non-working and working examples above?
Steve

Re: macOS Notes.app IMAP Syncing Not Working

[Ralph Seichter via dovecot]

* Steven Smith via dovecot:

> The issue is that macOS Note.app does not sync with this server.

I'm syncing macOS Mojave's Notes.app with Dovecot without problems, so
it can be done. Have you made sure that you activated both mail and
notes in macOS' Internet Accounts preferences for this particular IMAP
account? If so, is that IMAP account listed in the navigation bar of
Notes.app (mine shows iCloud and the IMAP account)?

-Ralph

macOS Notes.app IMAP Syncing Not Working

[Steven Smith via dovecot]

I’m requesting help getting macOS’s Notes.app to sync notes with a new dovecot 
IMAP server.

Thanks to dovecot’s excellent online documentation and help from this list, I 
have a new perfectly functioning IMAP server.

The issue is that macOS Note.app does not sync with this server. However, iOS 
Notes.app *does* sync and work well with this IMAP server, and the differences 
between macOS and iOS Notes apps are well known. (Thanks, Apple.)
https://apple.stackexchange.com/questions/265579/notes-wont-sync-via-imap-in-macos-sierra
 

https://jpmens.net/2015/09/28/experiments-using-imap-for-notes/ 

…

I am migrating from a working Server.app system in which Notes.app syncing does 
work across all devices, and therefore have a working dovecot configuration 
from Server.app, although it does not contain an explicit configuration for the 
“Notes” mailbox.

I would greatly appreciate any help or pointers for these questions:

Does anyone have a working template for dovecot that works with macOS 
Notes.app? I’ve posted doveconf -n of both the new server and the old 
Server.app server below.

Would you all be able to provide advice on TO-DO’s and NOT-TO-DO’s for 
mailbox/name-space reconfiguration as I try to muck around with a nicely 
working, deployed IMAP server? I believe that the solution has something to do 
with name spaces and separators, and I’ve already changed the separator from 
the original ‘/’  to ‘.’ on the working server without success or obvious 
changes.

All the mail boxes on the new IMAP server look like this, *without* a prefix 
‘.’:
> ls -1 /var/mail/tld.domain.mail/username/mdbox/mailboxes/
> Archive/
> Deleted Messages/
> Drafts/
> INBOX/
> Junk/
> Notes/
> Sent/
> Trash/


All the mailboxes on the working Server.app mailserver look like this, *with* a 
prefix ‘.’:
> $ sudo ls -1 /Library/Server/Mail/Data/mail/users/username
> .Archive
> .Deleted Messages
> .Drafts
> .Junk
> .Notes
> .Sent Messages
> cur
> …
> new
> subscriptions
> tmp



Here are the `doveconf -n` of the new server that doesn’t sync macOS Notes.app, 
and the old Server.app IMAP server that does:

New server `dovecot -n` [Note: original separator was set to ‘/’, now changed 
to ‘.’ in config.d/10-mail.conf]:
> # 2.3.6 (): /opt/local/etc/dovecot/dovecot.conf
> # Pigeonhole version 0.5.6 ()
> # OS: Darwin 18.6.0 x86_64  apfs
> # Hostname: newhost.domain.tld
> auth_cache_size = 10 M
> auth_gssapi_hostname = $ALL
> auth_krb5_keytab = /opt/local/etc/dovecot/imap.keytab
> auth_mechanisms = plain gssapi
> auth_realms = newhost.domain.tld
> auth_socket_path = /opt/local/var/run/dovecot/auth-userdb
> auth_username_format = %Ln
> debug_log_path = /opt/local/var/log/mail/mail-debug.log
> disable_plaintext_auth = no
> first_valid_gid = 6
> first_valid_uid = 6
> imap_id_log = *
> imap_id_send = "name" * "version" *
> imap_idle_notify_interval = 29 mins
> imap_urlauth_submit_user = submit
> info_log_path = /opt/local/var/log/mail/mail-info.log
> last_valid_gid = 100
> lda_mailbox_autocreate = yes
> log_path = /opt/local/var/log/mail/mail-err.log
> login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c
> mail_access_groups = mail
> mail_attachment_dir = /private/var/mail/tld.domain.mail/attachments
> mail_attachment_fs = sis posix:mode=0666
> mail_debug = yes
> mail_gid = mail
> mail_home = /private/var/mail/tld.domain.mail
> mail_location = mdbox:/private/var/mail/tld.domain.mail/%Ln/mdbox
> mail_log_prefix = "%s(pid %p user %u): "
> mail_plugins = quota zlib acl fts fts_solr fts_lucene
> mail_privileged_group = mail
> mail_uid = _dovecot
> managesieve_notify_capability = mailto
> managesieve_sieve_capability = fileinto reject envelope encoded-character 
> vacation subaddress comparator-i;ascii-numeric relational regex imap4flags 
> copy include variables body enotify environment mailbox date index ihave 
> duplicate mime foreverypart extracttext imapsieve vnd.dovecot.imapsieve
> mdbox_rotate_size = 200 M
> namespace inbox {
>   inbox = yes
>   location = 
>   mailbox Archive {
> auto = subscribe
> special_use = \Archive
>   }
>   mailbox Drafts {
> auto = subscribe
> special_use = \Drafts
>   }
>   mailbox Junk {
> auto = create
> special_use = \Junk
>   }
>   mailbox Notspam_train {
> auto = create
> special_use = \Junk
>   }
>   mailbox Sent {
> auto = subscribe
> special_use = \Sent
>   }
>   mailbox "Sent Messages" {
> special_use = \Sent
>   }
>   mailbox Spam_train {
> auto = create
> special_use = \Junk
>   }
>   mailbox Trash {
> auto = create
> special_use = \Trash
>   }
>   prefix = 
>   separator = .
> }
> passdb {
>   driver = pam
>   name = pam
> }
> plugin {
>   fts = solr
>   fts_autoindex = yes
>   fts_autoindex_exclude = \Junk
>   fts_solr = url=http://127.0.0.1:8983/solr/dovecot/
>