Greetings, On Friday, 2022-03-18 11:50:33 +0100, I myself wrote:
> Nikos, > > On Thursday, 2022-03-17 19:04:04 +0200, you wrote: > > > ... > > http://mmogilvi.users.sourceforge.net/software/oauthbearer.html > > ... > > Really interesting reading. Thanks for the pointer. And also thanks to > the other responders. Matthew Ogilvie's write-up at the above URL was not only interesting but also extremely helpful. It basically covers four points: 1. Background information on OAuth 2. 2. A how-to about setting up one's Google mail account for OAuth 2 and a Python script (requiring a small configuration file), which gets you your first two OAuth tokens: an "access token" which has to be passed to Google as a password, but which has only a life time of 60 minut- es, and a "refresh token" which is valid until you change your Google password or maybe also until Google decides enough being enough and starts rejecting your refresh requests with "HTTP Error 400: Bad Re- quest" messages which you'll find at the bottom of the Python trace- back from Matthew's script. As long as the refresh token is valid you will have to use this script on a rather regular basis to get new valid access tokens. 3. Gentoo-ready patches for "net-mail/fetchmail" up to at least version 6.4.13 which however will only support IMAP and will not work in dae- mon mode, thus requiring a "cron" job for fetching mail. Matthew also points out that Gentoo offers a still masked "net-mail/ fetchmail" version 7.0.0_alpha9-r1 which supports OAuth 2, POP3, and daemon mode. 4. A description how to tweak "mail-mta/postfix" so it supports OAuth 2. This involves installing a "cyrus-sasl-xoauth2" plugin not available in the Gentoo repository, a complex configuration setup for a full- fledged MTA which above all is dynamically changing (at least, if you want to handle outgoing mail for more than one user), special "sudo- ers" rules for these mailing users, and "cron" jobs to periodically send their mail off to their Google accounts. Following part 2 of this guide I soon had my Google account OAuth-ready. The Python script from Matthew requires a configuration file the path to which has to be explicitly specified in every call. My configuration file resides at "~/.../oauth.cfg" and contains: client_id=... client_secret=... access_token_file=/home/rainer/.../oauth-access-token refresh_token_file=/home/rainer/.../oauth-refresh-token max_age_sec=1800 The first two lines specify the Google project just created, the next two lines define the absolute paths to the two token files to be used, and the last line sets the access token age before which the token will not be renewed. DO NOT try to add empty or comment lines to this file! Regarding the adaption of "fetchmail" in part 3 I opted for installing version 7.0.0_alpha9-r1, since I was used to POP3 and daemon mode [1]. Version 7 is still masked, so to be able to emerge it execute (in the first "echo" command replace "amd64" with YOUR architecture): # mkdir -p /etc/portage/package.accept_keywords # echo '>=net-mail/fetchmail-7.0.0_alpha9-r1 ~amd64' \ >> /etc/portage/package.accept_keywords/Oauth # mkdir -p /etc/portage/package.unmask # echo '>=net-mail/fetchmail-7.0.0_alpha9-r1' \ >> /etc/portage/package.unmask/OAuth [1] If you do not like installing an alpha-version, want to use POP3, but do not insist on daemon mode, I've meanwhile found out that Gen- too also provides a package "net-mail/mpop" from the same author as the MTA I decided to install instead of "mail-mta/postfix". Howev- er, Gentoo-wise this package is almost two years behind due to the lack of a Gentoo maintainer, so you might want to install it direct- ly from https://marlam.de/mpop/ Of course the new "fetchmail" version also introduced new configuration items. Here is my new "~/.fetchmailrc" file (the "authenticate" direct- ive as well as the last three lines are new): set daemon 60 set invisible set no syslog poll pop.gmail.com protocol POP3 service 995 authenticate oauthbearer username "rainer.woi...@gmail.com" dropdelivered fetchall no keep mda "/usr/bin/procmail -pf %F" passwordfile "/home/rainer/.../oauth-access-token" sslmode wrapped sslcertck Even though I run "fetchmail" in daemon mode, it's not running all the time. It's started once when I log in, but my backup script which I use at least once a day will again terminate it before doing anything else, so my mailbox will never change between my last backup and hibernating my laptop. And yes, I almost never shutdown my laptop, but rather suspend or hib- ernate it. Thus I'll possibly need a fresh access token when I manually start "fetchmail", but it's not necessary to repeatedly update it via "cron" while "fetchmail" isn't running at all. The same holds when re- suming from suspension or hibernation: only when "fetchmail" is running the age of the access token should be checked and potentially a new one requested. So I'll have to use Matthew's "oauth.py" script in three ways: 1. Running it the first time or starting over to get both, a new refresh token and a new access token. 2. Unconditionally checking the age of the access token and optionally requesting a new one, regardless of whether or not "fetchmail" is ex- ecuting. 3. Only checking the age of the access token and optionally requesting a new one, if "fetchmail" is really executing. This leads to wrapper script "~/.../oauth.sh" which knows the options it has to pass to script "oauth.py": #! /bin/bash -u # # Script to retrieve new OAuth tokens from Google mail. # # There are two types of OAuth tokens, an "access token" which only has # a life time of 60 minutes and is required for both, fetching mail from # and sending mail via Google mail, and a "refresh token" which is valid # until changing one's Google mail password or until Google decides it # has been used long enough, and is required to get a new "access tok- # en". # # The action performed by this script depends on its first argument: # # "init": Get a (new) refresh token together with a new access token. # This causes an URL to be sent to standard output, which you # have to point your browser to. At Google's mail login page # use your mail address and password to log into your mail ac- # count, agree to accessing your project, copy the authorizat- # ion code then provided, paste it after the text "Enter ver- # ification code:" in the terminal window where this script is # still running, and hit Enter. # # "update": Update the access token, except when it is younger than the # maximum age specified in configuration file "oauth.cfg". # # no arg: Only when "fetchmail" is running update the access token as # if "update" has been specified. dir=$(dirname $(realpath "$0")) # Symlink-free abs path to script dir. cfg="--config_file=$dir/oauth.cfg" # Configuration file option. if [[ "${1-}" = init ]] then "$dir/oauth.py" "$cfg" --obtain_refresh_token_file elif pgrep -u $USER fetchmail > /dev/null || [[ "${1-}" = update ]] then "$dir/oauth.py" "$cfg" --auto_refresh fi Now keeping the above comment regarding "init" in mind I executed: $ ~/.../oauth.sh init to get my first OAuth 2 access tokens. I then created: 1. A new "fetchmail" alias (which will also be run upon login): $ alias fetchmail='~/.../oauth.sh update ; fetchmail' 2. A "crontab" entry for my own userid (don't forget to add yourself to group "cron"): # When "fetchmail" is started, it will request a new "OAuth" access tok- # en from Google as soon as the current token is older than 30 minutes. # And as long as "fetchmail" is running, this "cron" job, too, will re- # new the "OAuth" access token when it is older than 30 minutes. Since # we start this "cron" job three times an hour, this will result in acc- # ess tokens (except for the first one, which will be refreshed after 30 # to 50 minutes) being refreshed every 40 minutes. This fits well with # the maximum life time of 60 minutes for any access token. 11,31,51 * * * * $HOME/.../oauth.sh 3. The hook script "/lib64/elogind/system-sleep/oauth-token.sh" for the "elogind" service: #! /bin/bash -u # # When resuming from hibernation or suspension update Google's OAuth to- # ken for user "rainer". # # This script will be called by "elogind" with exactly two arguments: # # $1: Either "pre" or "post". # $2: Either "hibernate", "hybrid-sleep", "suspend", or "suspend-then- # hibernate". # # The "sleep" command is required to give the host some time connecting # the WiFi before running "oauth.sh", and taking this nap in the back- # ground prevents it from delaying "elogind" while firing up the WiFi: [[ "$1" = pre ]] || (sleep 10 ; sudo -u rainer ~rainer/.../oauth.sh) & Now it's time to just start "fetchmail" via its new alias and to catch up with one's incoming mail ... However, I did not follow Matthew's suggestions in part 4 regarding the configuration of "mail-mta/postfix" because the configuration effort for a full-fledged, system wide MTA as well as adding another "ebuild" file to my local overlay seemed like overkill to me. Up to now I was using "ssmtp" for sending mail, and if I remember correctly, the first "s" in this name was referring to "simple". Its configuration file just con- tained eight lines. Looking around I found Gentoo package "mail-mta/ msmtp", which supports OAuth 2 and also a user local configuration file at "~/.msmtprc". Here are the relevant lines from mine: # Name the only account defined here "default", so it overrides the def- # ault account defined in file "/etc/msmtprc": account default auth oauthbearer domain gmail.com host smtp.gmail.com passwordeval cat ~/.../oauth-access-token port 587 protocol smtp tls on tls_certcheck on tls_starttls on user rainer.woi...@gmail.com Now the only thing left to do is telling one's MUA not to pipe outgoing mail directly to the standard "/usr/sbin/sendmail", but rather to script "~/.../msmtp.sh" instead, so we can refresh the OAuth access token, if necessary, before really sending off the mail. Script "~/.../msmtp.sh" contains: #! /usr/bin/bash -u # # Before sending the mail refresh the access token, if necessary, and # since the mail to be sent is provided on standard input, route stand- # ard input around the call to "oauth.sh": { cat >&3 ; ~/.../oauth.sh update ; } 3>&1 | msmtp "$@" Should you MUA not provide a way for specifying the programme to which outgoing mail is piped, you can simply modify the symbolic link "/usr/ sbin/sendmail" (which has just been set by package "mail-mta/msmtp" to point to "../bin/msmtp"): # cd /usr/sbin # rm sendmail # ln -s ~rainer/.../msmtp.sh sendmail That's all, happy mailing ... :-) Sincerely, Rainer PS: This mail was sent to you using "mail-mta/msmtp" set-up the way de- scribed above ... :-)
