On 18 Jul 2021, at 00:03, Oswald Buddenhagen <oswald.buddenha...@gmx.de> wrote: > On Sat, Jul 17, 2021 at 09:56:55PM +0100, Sabahattin Gucukoglu via > isync-devel wrote: >> Anyway the problem goes away and literal payloads all have the right sizes, >> and sync completes successfully, if I write a little wrapper script that >> just adds a CAPABILITY request immediately after login. > > would you mind posting traces (-DN) of the various failed and successful > login attempts? i'd like to understand better what is going on before > implementing workarounds.
Have a look at (and previous messages): http://mailman13.u.washington.edu/pipermail/imap-protocol/2021-May/000020.html I didn’t know then that the reason Apple Mail didn’t log unusual behaviour is that it sent CAPABILITY after login, which is how my workaround fixes the problem. It never occurred to me until I simply had to track down the problem (because the proxy was deleting characters I couldn’t replace). Hence, Apple actually did do the right thing (by their own terrible standards) and turn off the fixing, but only once they saw the CAPABILITY request which they presumably could rely on their (and other third-party) clients to do. In the curl transcript you can see that the body literal has extended past the declared size. I’m afraid I can’t really demonstrate mbsync itself failing now that I have synchronised all my mail (it occurred in one particular folder, where IMAP discussions were scattered throughout at random). The rough hack in Tcl that fixes it is below, if it helps. I initially tried just deleting “ XAPPLEPUSHSERVICE” which produced correct results until I came across a message where “ STARTTLS” had been deleted; then I had to find out the secret sauce used by Mail.app itself. Here’s a transcript showing how the capability response is altered in a manual session: sgucukoglu@Onyx:~$ openssl s_client -connect imap.mail.me.com:993 -crlf CONNECTED(00000005) […] * OK [CAPABILITY XAPPLEPUSHSERVICE IMAP4 IMAP4rev1 SASL-IR AUTH=ATOKEN AUTH=PLAIN] (2114B34-a748bbbd8277) ms11p00im-qufv17101901.me.com 0 capability * CAPABILITY XAPPLEPUSHSERVICE IMAP4 IMAP4rev1 SASL-IR AUTH=ATOKEN AUTH=PLAIN 0 OK Completed 1 capability * CAPABILITY XAPPLEPUSHSERVICE IMAP4 IMAP4rev1 SASL-IR AUTH=ATOKEN AUTH=PLAIN 1 OK Completed 2 login sgucukoglu blah-blah-blah * OK [CAPABILITY XAPPLEPUSHSERVICE IMAP4 IMAP4rev1 ACL RIGHTS=tekx QUOTA LITERAL+ NAMESPACE UIDPLUS CHILDREN BINARY UNSELECT SORT CATENATE URLAUTH LANGUAGE ESEARCH ESORT THREAD=ORDEREDSUBJECT THREAD=REFERENCES CONDSTORE ENABLE QRESYNC CONTEXT=SEARCH CONTEXT=SORT WITHIN SASL-IR SEARCHRES METADATA ID LIST-STATUS SPECIAL-USE CREATE-SPECIAL-USE MULTISEARCH SORT=DISPLAY LOGIN-REFERRALS UTF8=ACCEPT URL-PARTIAL STATUS=SIZE XSNIPPET=FUZZY X-SUN-SORT ANNOTATE-EXPERIMENT-1 X-SUN-IMAP XUM1 X-ORCL-AS IDLE AUTH=PLAIN AUTH=ATOKEN AUTH=WSTOKEN AUTH=WETOKEN AUTH=PTOKEN] ms72055.mac.com IMAP4 service (Oracle Communications Messaging Server 8.0.2.5.20200127 64bit (built Jan 27 2020)) 2 OK User sgucukoglu logged in 3 capability * CAPABILITY XAPPLEPUSHSERVICE IMAP4 IMAP4rev1 ACL RIGHTS=tekx QUOTA LITERAL+ NAMESPACE UIDPLUS CHILDREN BINARY UNSELECT SORT CATENATE URLAUTH LANGUAGE ESEARCH ESORT THREAD=ORDEREDSUBJECT THREAD=REFERENCES CONDSTORE ENABLE QRESYNC CONTEXT=SEARCH CONTEXT=SORT WITHIN SASL-IR SEARCHRES METADATA ID LIST-STATUS SPECIAL-USE CREATE-SPECIAL-USE MULTISEARCH SORT=DISPLAY LOGIN-REFERRALS UTF8=ACCEPT URL-PARTIAL STATUS=SIZE XSNIPPET=FUZZY X-SUN-SORT ANNOTATE-EXPERIMENT-1 X-SUN-IMAP XUM1 X-ORCL-AS IDLE 3 OK Completed 4 capability * CAPABILITY IMAP4 IMAP4rev1 ACL RIGHTS=tekx QUOTA LITERAL+ NAMESPACE UIDPLUS CHILDREN BINARY UNSELECT SORT CATENATE URLAUTH LANGUAGE ESEARCH ESORT THREAD=ORDEREDSUBJECT THREAD=REFERENCES CONDSTORE ENABLE QRESYNC CONTEXT=SEARCH CONTEXT=SORT WITHIN SASL-IR SEARCHRES METADATA ID LIST-STATUS SPECIAL-USE CREATE-SPECIAL-USE MULTISEARCH SORT=DISPLAY LOGIN-REFERRALS UTF8=ACCEPT URL-PARTIAL STATUS=SIZE XSNIPPET=FUZZY X-SUN-SORT ANNOTATE-EXPERIMENT-1 X-SUN-IMAP XUM1 X-ORCL-AS IDLE 4 OK Completed 5 logout * BYE LOGOUT received 5 OK Completed closed sgucukoglu@Onyx:~$ Without the workaround script below, CAPABILITY is never issued by mbsync, therefore the bodging is still in effect and I got bogus response errors where the lines were, for instance, the tail-ends of MIME boundary markers or mailing list footers that had extended beyond the literal. Here’s the script (apologies for roughness): #!/opt/local/bin/tclsh set isLoggedIn no set gotTagged no proc patch {i o isReply} { global isLoggedIn gotTagged if {[gets $i line]<0} { if {[eof $i]} exit else return } if {$isReply&&!$isLoggedIn&&[regexp " OK User .+ logged in$" $line]} { set isLoggedIn yes puts $i "myveryspecialtag capability" } if {$isReply&&!$gotTagged&&[string equal $line "myveryspecialtag OK Completed"]} { set gotTagged yes return } puts $o $line } fconfigure stdin -blocking 0 -buffering none -encoding binary -translation crlf fconfigure stdout -blocking 0 -buffering none -encoding binary -translation crlf fconfigure stderr -blocking 0 -buffering none -encoding binary -translation lf package require tls set sock [tls::socket -autoservername yes -cafile /opt/local/share/curl/curl-ca-bundle.crt -tls1.2 no -tls1.1 no -tls1 no -request yes -require yes p72-imap.mail.me.com 993] fconfigure $sock -blocking 0 -buffering none -encoding binary -translation crlf fileevent $sock readable [list patch $sock stdout yes] fileevent stdin readable [list patch stdin $sock no] tls::handshake $sock vwait forever HTH and apologies for the delay. Cheers, Sabahattin _______________________________________________ isync-devel mailing list isync-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/isync-devel
