Re: 6.8 and Procmail/Formail: anyone still using them?

[Steve Williams]

Hi,

You are correct!

The contents of my .forward!!

pcengine$ cat .forward
"|/usr/local/bin/procmail -f -"

And yes, all my filtering is defined in my .procmailrc file.

Sorry for any confusion!

Cheers,
Steve W.


On 15/02/2021 11:30 a.m., Austin Hook wrote:


Hi Steve,  I wonder if in your email (below) you have your lines slightly
out of order:



My .procmailrc:
"|/usr/local/bin/procmail -f -"

I am thinking that is your .forward file.


and


Not sure if this is your problem or not.? But I have quite a large .procmailrc
file (200 lines) that makes? a historical archive of every incoming email,

I gather that you put all your procmail filtering directly into your
.procmailrc file.  I think it was usually the case that

Or is this is a reference to a something.rc file in the ~/Procmail
directory?

My .procmailrc file looks like this:

# Turn off extra words in log file set to yes for debugging
VERBOSE=no

# For debugging uncomment this line
LOGABSTRACT=all

# Tell procmail where to store your mail. This changes depending on

#  which Unix mail client you use.
# Pine uses $HOME/mail
# Mutt and Elm use $HOME/Mail
MAILDIR=$HOME/mail  #This directory must exist!!!

# Use a seperate directory to store recipes and logs
PMDIR=$HOME/Procmail

# Tell procmail where to put the log file
LOGFILE=$PMDIR/procmail.log
 
# Add recipe files here

# To add more recipe files just add an INCLUDERC=$PMDIR/filename.rc
#  line for each file
#INCLUDERC=$PMDIR/testing.rc

INCLUDERC=$PMDIR/lists.rc
 
# Finally if the above recipes fail to move your mail put it in your

#  inbox
 
:0:

# Above is a zero (0) not the letter O.
$DEFAULT

Austin

PS: I solved my problem with your help above.  Thanks!  It has to do with
the differences in how the new smtpd process deals with aliases which are
pipes to programs or scripts.  It does not handle complex command line
equivalents with definitions of environment variables preceding the
invocation of the program being called up.  I wonder if it even handles
the typical command lines that have the kind of "if this succeds, then do
that also, but if any invoked process fails do the other" -- the usual
&& or || connectors one often sees in major shell scripts.

That kind of usage in the .forward file is what screws up a lot of custom
scripts I wrote for myself years ago.

I haven't had a chance yet to look to see if procmail still recommends
that kind of .forward file.  Later I will submit a report to misc@ or
ports@

The .forward file used to recommended by procmail to look like this:

"|IFS=' ' && exec /usr/local/bin/procmail -f- || exit 75 #austin"
  
That caused an error "cannot expand alias" ( or something like that --

neatly misleading --- as usual in debugging problems... )  :-)

Still have to find time to study the newer replacement folks are
recommending for procmail.  I gather most everyone else has left procmail
in the dust





On Wed, 27 Jan 2021, Steve Williams wrote:

Hi,

I am using procmail under 6.8 successfully.? I did have problems with it when
upgrading to (I think) 6.4.

If you look for the mail list archives for "OpenBSD 6.4 smtpd local mail
delivery missing "From " when .forward (procmail)"

My .procmailrc:

"|/usr/local/bin/procmail -f -"

Not sure if this is your problem or not.? But I have quite a large .procmailrc
file (200 lines) that makes? a historical archive of every incoming email,
filtering maillist emails, etc.

Thanks,
Steve W.


On 26/01/2021 10:43 a.m., Austin Hook wrote:

Wonder if anyone is still using Procmail/Formail under 6.8 for presorting
incoming mail before it hits one's main inbox.

Also wondering if folks send the remainimg mail, after filtering, to
/var/mail/*user*, or to ~/mbox or to ~mail/mbox.  Any advantage to be had,
or any mere consensus, regardless of advantages?

I also use whitelisting extensively, and any such "From: emailaddresses"
get priority.  Does anyone else?

Myself: Having problems with Procmail/formail, after upgrading from 5.3 to
a new server running 6.8.  Would like to hear of anyone else's experience.

Thanks,

Austin Hook
Milk River, Alberta

Re: 6.8 and Procmail/Formail: anyone still using them?

[Austin Hook]

Hi Steve,  I wonder if in your email (below) you have your lines slightly 
out of order:  


> My .procmailrc:
> "|/usr/local/bin/procmail -f -"

I am thinking that is your .forward file.


and  

> Not sure if this is your problem or not.? But I have quite a large .procmailrc
> file (200 lines) that makes? a historical archive of every incoming email,

I gather that you put all your procmail filtering directly into your 
.procmailrc file.  I think it was usually the case that 

Or is this is a reference to a something.rc file in the ~/Procmail 
directory?

My .procmailrc file looks like this:

# Turn off extra words in log file set to yes for debugging
VERBOSE=no

# For debugging uncomment this line
LOGABSTRACT=all
   
# Tell procmail where to store your mail. This changes depending on 
#  which Unix mail client you use.
# Pine uses $HOME/mail
# Mutt and Elm use $HOME/Mail
MAILDIR=$HOME/mail  #This directory must exist!!!

# Use a seperate directory to store recipes and logs
PMDIR=$HOME/Procmail

# Tell procmail where to put the log file
LOGFILE=$PMDIR/procmail.log

# Add recipe files here
# To add more recipe files just add an INCLUDERC=$PMDIR/filename.rc 
#  line for each file 
#INCLUDERC=$PMDIR/testing.rc

INCLUDERC=$PMDIR/lists.rc

# Finally if the above recipes fail to move your mail put it in your 
#  inbox

:0:
# Above is a zero (0) not the letter O.
$DEFAULT

Austin

PS: I solved my problem with your help above.  Thanks!  It has to do with 
the differences in how the new smtpd process deals with aliases which are 
pipes to programs or scripts.  It does not handle complex command line 
equivalents with definitions of environment variables preceding the 
invocation of the program being called up.  I wonder if it even handles 
the typical command lines that have the kind of "if this succeds, then do 
that also, but if any invoked process fails do the other" -- the usual
&& or || connectors one often sees in major shell scripts.

That kind of usage in the .forward file is what screws up a lot of custom 
scripts I wrote for myself years ago.

I haven't had a chance yet to look to see if procmail still recommends 
that kind of .forward file.  Later I will submit a report to misc@ or 
ports@ 

The .forward file used to recommended by procmail to look like this:

"|IFS=' ' && exec /usr/local/bin/procmail -f- || exit 75 #austin"  
 
That caused an error "cannot expand alias" ( or something like that -- 
neatly misleading --- as usual in debugging problems... )  :-)

Still have to find time to study the newer replacement folks are 
recommending for procmail.  I gather most everyone else has left procmail 
in the dust





On Wed, 27 Jan 2021, Steve Williams wrote:
> Hi,
> 
> I am using procmail under 6.8 successfully.? I did have problems with it when
> upgrading to (I think) 6.4.
> 
> If you look for the mail list archives for "OpenBSD 6.4 smtpd local mail
> delivery missing "From " when .forward (procmail)"
> 
> My .procmailrc:
> 
> "|/usr/local/bin/procmail -f -"
> 
> Not sure if this is your problem or not.? But I have quite a large .procmailrc
> file (200 lines) that makes? a historical archive of every incoming email,
> filtering maillist emails, etc.
> 
> Thanks,
> Steve W.
> 
> 

> On 26/01/2021 10:43 a.m., Austin Hook wrote:
> > Wonder if anyone is still using Procmail/Formail under 6.8 for presorting
> > incoming mail before it hits one's main inbox.
> > 
> > Also wondering if folks send the remainimg mail, after filtering, to
> > /var/mail/*user*, or to ~/mbox or to ~mail/mbox.  Any advantage to be had,
> > or any mere consensus, regardless of advantages?
> > 
> > I also use whitelisting extensively, and any such "From: emailaddresses"
> > get priority.  Does anyone else?
> > 
> > Myself: Having problems with Procmail/formail, after upgrading from 5.3 to
> > a new server running 6.8.  Would like to hear of anyone else's experience.
> > 
> > Thanks,
> > 
> > Austin Hook
> > Milk River, Alberta
> 

Re: 6.8 and Procmail/Formail: anyone still using them?

[Marcus MERIGHI]

aus...@computershop.ca (Austin Hook), 2021.01.30 (Sat) 10:52 (CET):
> Marcus:
> 
> > Regarding procmail beware of this:
> > https://marc.info/?l=openbsd-ports=151256201621939
> 
> Fascinating.  Never caught that discussion before. I gather a specially 
> crafted message could get control of the user's account, or at least it 
> would hard to prove that it couldn't.
> 
> I guess I can appreciate further, that since the mail system processes the 
> .forward file, that means a task with higher privileges than even the 
> user, has to deal with external world possibly garbage or infected input, 
> which could be unfriendly, and if the code base was designed without even 
> such a thought, and is unwieldy -- there was the incentive to do better.
> 
> Oh, but the years of fine tuning the procmail scripts  

oh yeah! and it gets even worse... sooner or later we might have to
switch to sieve[1] scripts. because that is what dovecot and everything
else understands. but it's standardized, as a plus.

[1] https://en.wikipedia.org/wiki/Sieve_(mail_filtering_language)

> > formail is not in the ports tree, afaict.
> I probably just should have said only the package for procmail; formail 
> comes with it.  

I should have used pkg_locate(1) first...

> formail goes into the .forward file, and regularizes any 
> problems with the "From" email address, before handing off to procmail.  
> I guess that's somewhat a security enhancement.  Perhaps not enough.
> 
> Maybe formail isn't always used or was dropped in later version. I see a 
> comment from Steve (clipped below) that he doesn't use it. [Haven't yet 
> checked out his reference or absorbed it's implications yet. May comment 
> further, if I further retry procmail first, before learning fdm.]
> 
> > I use ~/Maildir
> 
> In your case, is that ~/Maildir (a file), or is it ~/Maildir/ a directory?

It is a directory, and a different way to store mail. see 
https://en.wikipedia.org/wiki/Maildir
 
> In my new install, not doing any mail sorting yet, Simple "Mail" seems to 
> put new mail into ~/mbox (the file) if not handled explicitly other than 
> looking at the subject lines -- even though it says (at run time) 
> something about putting it back into the user's mailbox -- which is 
> different wording from the documentation, and slightly confusing.

ha-ha, the second time within a couple of days that mail(1) interactive
use confuses people, including me :-)

> Alpine (not further configured) moves all new mail from /var/mail/*user*/ 
> to ~/mbox (the file), soon as it is invoked.

my gut feeling is that this historic behaviour is going extinct.

Marcus

> On Wed, 27 Jan 2021, Maurice McCarthy wrote:
> 
> > Most use fdm from ports
> > Best
> 
> On Wed, 27 Jan 2021, Marcus MERIGHI wrote:
> 
> > aus...@computershop.ca (Austin Hook), 2021.01.26 (Tue) 18:43 (CET):
> > > Wonder if anyone is still using Procmail/Formail under 6.8 for
> > > presorting incoming mail before it hits one's main inbox.
> > 
> > Regarding procmail beware of this:
> > https://marc.info/?l=openbsd-ports=151256201621939
> > 
> > formail is not in the ports tree, afaict.
> > 
> > I switched from procmail to fdm:
> > 
> > Information for inst:fdm-2.0p0
> > 
> > Comment:
> > fetch, filter and deliver mail
> > 
> > Description:
> > fdm is a simple, lightweight replacement for mail fetch, filter and
> > delivery programs such as fetchmail and procmail. It can fetch using
> > POP3 or IMAP (with SSL) or from stdin, and deliver to a pipe, file,
> > maildir, mbox or SMTP server, based on regexps.
> > 
> > Maintainer: Nicholas Marriott 
> > 
> > 
> > > Also wondering if folks send the remainimg mail, after filtering, to 
> > > /var/mail/*user*, or to ~/mbox or to ~mail/mbox.  Any advantage to be 
> > > had, 
> > > or any mere consensus, regardless of advantages?
> > 
> > I use ~/Maildir
> > 
> > Marcus
> > 
> Date: Wed, 27 Jan 2021 09:04:43 -0700
> From: Steve Williams 
> To: misc@openbsd.org
> Subject: Re: 6.8 and Procmail/Formail: anyone still using them?
> 
> Hi,
> 
> I am using procmail under 6.8 successfully.? I did have problems with it 
> when upgrading to (I think) 6.4.
> 
> If you look for the mail list archives for "OpenBSD 6.4 smtpd local mail 
> delivery missing "From " when .forward (procmail)"
> 
> My .procmailrc:
> 
> "|/usr/local/bin/procmail -f -"
> 
> Not sure if this is your problem or not.? But I have quite a large 
> .procmailrc file (200 lines) that makes? a historical archive of every 
> incoming email, filtering maillist emails, etc.
> 
> Thanks,
> Steve W.
> 
> 

Re: 6.8 and Procmail/Formail: anyone still using them?

[Austin Hook]

Maurice, Marcus, Steve

> Most use fdm from ports
> Best

Thanks for the tips on fdm.  I shall take a look!

Marcus:

> Regarding procmail beware of this:
> https://marc.info/?l=openbsd-ports=151256201621939

Fascinating.  Never caught that discussion before. I gather a specially 
crafted message could get control of the user's account, or at least it 
would hard to prove that it couldn't.

I guess I can appreciate further, that since the mail system processes the 
.forward file, that means a task with higher privileges than even the 
user, has to deal with external world possibly garbage or infected input, 
which could be unfriendly, and if the code base was designed without even 
such a thought, and is unwieldy -- there was the incentive to do better.

Oh, but the years of fine tuning the procmail scripts  

> formail is not in the ports tree, afaict.

I probably just should have said only the package for procmail; formail 
comes with it.  formail goes into the .forward file, and regularizes any 
problems with the "From" email address, before handing off to procmail.  
I guess that's somewhat a security enhancement.  Perhaps not enough.

Maybe formail isn't always used or was dropped in later version. I see a 
comment from Steve (clipped below) that he doesn't use it. [Haven't yet 
checked out his reference or absorbed it's implications yet. May comment 
further, if I further retry procmail first, before learning fdm.]

> I use ~/Maildir
> Marcus

In your case, is that ~/Maildir (a file), or is it ~/Maildir/ a directory?

In my new install, not doing any mail sorting yet, Simple "Mail" seems to 
put new mail into ~/mbox (the file) if not handled explicitly other than 
looking at the subject lines -- even though it says (at run time) 
something about putting it back into the user's mailbox -- which is 
different wording from the documentation, and slightly confusing.

Alpine (not further configured) moves all new mail from /var/mail/*user*/ 
to ~/mbox (the file), soon as it is invoked.

Anyway those are clearly moot details for most folks now.

Thanks again, for the comments all!

Austin




On Wed, 27 Jan 2021, Maurice McCarthy wrote:

> Most use fdm from ports
> Best

On Wed, 27 Jan 2021, Marcus MERIGHI wrote:

> aus...@computershop.ca (Austin Hook), 2021.01.26 (Tue) 18:43 (CET):
> > Wonder if anyone is still using Procmail/Formail under 6.8 for
> > presorting incoming mail before it hits one's main inbox.
> 
> Regarding procmail beware of this:
> https://marc.info/?l=openbsd-ports=151256201621939
> 
> formail is not in the ports tree, afaict.
> 
> I switched from procmail to fdm:
> 
> Information for inst:fdm-2.0p0
> 
> Comment:
> fetch, filter and deliver mail
> 
> Description:
> fdm is a simple, lightweight replacement for mail fetch, filter and
> delivery programs such as fetchmail and procmail. It can fetch using
> POP3 or IMAP (with SSL) or from stdin, and deliver to a pipe, file,
> maildir, mbox or SMTP server, based on regexps.
> 
> Maintainer: Nicholas Marriott 
> 
> 
> > Also wondering if folks send the remainimg mail, after filtering, to 
> > /var/mail/*user*, or to ~/mbox or to ~mail/mbox.  Any advantage to be had, 
> > or any mere consensus, regardless of advantages?
> 
> I use ~/Maildir
> 
> Marcus
> 
Date: Wed, 27 Jan 2021 09:04:43 -0700
From: Steve Williams 
To: misc@openbsd.org
Subject: Re: 6.8 and Procmail/Formail: anyone still using them?

Hi,

I am using procmail under 6.8 successfully.? I did have problems with it 
when upgrading to (I think) 6.4.

If you look for the mail list archives for "OpenBSD 6.4 smtpd local mail 
delivery missing "From " when .forward (procmail)"

My .procmailrc:

"|/usr/local/bin/procmail -f -"

Not sure if this is your problem or not.? But I have quite a large 
.procmailrc file (200 lines) that makes? a historical archive of every 
incoming email, filtering maillist emails, etc.

Thanks,
Steve W.

Re: 6.8 and Procmail/Formail: anyone still using them?

[Steve Williams]

On 26/01/2021 10:43 a.m., Austin Hook wrote:

Wonder if anyone is still using Procmail/Formail under 6.8 for presorting
incoming mail before it hits one's main inbox.

Also wondering if folks send the remainimg mail, after filtering, to
/var/mail/*user*, or to ~/mbox or to ~mail/mbox.  Any advantage to be had,
or any mere consensus, regardless of advantages?

I also use whitelisting extensively, and any such "From: emailaddresses"
get priority.  Does anyone else?

Myself: Having problems with Procmail/formail, after upgrading from 5.3 to
a new server running 6.8.  Would like to hear of anyone else's experience.

Thanks,

Austin Hook
Milk River, Alberta


Hi,

I am using procmail under 6.8 successfully.  I did have problems with it 
when upgrading to (I think) 6.4.


If you look for the mail list archives for "OpenBSD 6.4 smtpd local mail 
delivery missing "From " when .forward (procmail)"


My .procmailrc:

"|/usr/local/bin/procmail -f -"

Not sure if this is your problem or not.  But I have quite a large 
.procmailrc file (200 lines) that makes  a historical archive of every 
incoming email, filtering maillist emails, etc.


Thanks,
Steve W.

Re: 6.8 and Procmail/Formail: anyone still using them?

[Marcus MERIGHI]

aus...@computershop.ca (Austin Hook), 2021.01.26 (Tue) 18:43 (CET):
> Wonder if anyone is still using Procmail/Formail under 6.8 for
> presorting incoming mail before it hits one's main inbox.

Regarding procmail beware of this:
https://marc.info/?l=openbsd-ports=151256201621939

formail is not in the ports tree, afaict.

I switched from procmail to fdm:

Information for inst:fdm-2.0p0

Comment:
fetch, filter and deliver mail

Description:
fdm is a simple, lightweight replacement for mail fetch, filter and
delivery programs such as fetchmail and procmail. It can fetch using
POP3 or IMAP (with SSL) or from stdin, and deliver to a pipe, file,
maildir, mbox or SMTP server, based on regexps.

Maintainer: Nicholas Marriott 


> Also wondering if folks send the remainimg mail, after filtering, to 
> /var/mail/*user*, or to ~/mbox or to ~mail/mbox.  Any advantage to be had, 
> or any mere consensus, regardless of advantages?

I use ~/Maildir

Marcus

6.8 and Procmail/Formail: anyone still using them?

[Austin Hook]

Wonder if anyone is still using Procmail/Formail under 6.8 for presorting 
incoming mail before it hits one's main inbox.

Also wondering if folks send the remainimg mail, after filtering, to 
/var/mail/*user*, or to ~/mbox or to ~mail/mbox.  Any advantage to be had, 
or any mere consensus, regardless of advantages?

I also use whitelisting extensively, and any such "From: emailaddresses" 
get priority.  Does anyone else?

Myself: Having problems with Procmail/formail, after upgrading from 5.3 to 
a new server running 6.8.  Would like to hear of anyone else's experience.

Thanks,

Austin Hook
Milk River, Alberta