Re: [new] mail/rmilter

[Stuart Henderson]

Slightly tweaked:

- skip -O3 in new dirs, pointed out by jca
- rand % x -> arc4random_uniform(x)



rmilter.tgz
Description: application/tar-gz

Re: [new] mail/rmilter

[Stuart Henderson]

On 2016/11/19 12:41, Johan Huldtgren wrote:
> > release once one has been done on the 2.10 branch without an ugly v0.
> 
> Thanks, agreed that's better.

Oops, should have been s/2.10/1.10 :)  Anyway that is released now.

> > I still see the problem I was getting with the previous version
> > though:
> > 
> > 2016-11-19T16:36:32.261Z symphytum rmilter[19335]: ; mlfi_data: 
> > cannot get queue id, set to 'NOQUEUE'
> > (Plus no evidence of it feeding the mail through rspamd..)
> 
> I don't see that on my end:

And I've figured this out at last; I was trying to run it alongside my
existing "before-accept" amavis setup. rmilter was on the external-
facing instance of the MTA, then feeding it via smtpd_proxy_filter
to amavis, then to an internal instance of the MTA.

Fixing that up (I just removed amavis for now, though I could probably
move it from smtpd_proxy_filter to content_filter) gets queue IDs into
rmilter, and now that rmilter has queue IDs it is successfully passing
mail to rspamd.

Example Postfix setup:

smtpd_milters = unix:/var/run/rmilter/rmilter.sock
milter_protocol = 6
milter_mail_macros = i {mail_addr} {client_addr} {client_name} {auth_authen}
milter_default_action = accept

And in my case Postfix is chrooted, so I have this in rmilter.conf

bind_socket = unix:/var/spool/postfix/var/run/rmilter/rmilter.sock;

I've attached an updated tgz. I'm happy with this now, so I'm looking
for an OK to import.


rmilter.tgz
Description: application/tar-gz

Re: [new] mail/rmilter

[Johan Huldtgren]

On 11/19/16 11:37, Stuart Henderson wrote:
> On 2016/11/19 11:13, Johan Huldtgren wrote:
>> hello,
>>
>> rmilter is used to integrate rspamd with any milter-compatible
>> MTA, this is mostly taken from openbsd-wip with all the heavy
>> lifting by @sthen. I made some minor changes:
>>
>> - Update from 1.9.2 to a later working version
>> - bumped the uid/gid as 780 which was originally in there is now taken
>> - changed the configuration files to live in /etc/rmilter/* as the
>>   recommendation is that you make changes to a .local file and/or create
>>   /etc/rmilter/local.d/ for further user/local customization
>>
>> # cat pkg/DESCR
>> Rmilter is used to integrate rspamd with any milter-compatible
>> MTA, for example postfix or sendmail. Features include:
>>
>> - ClamAV scanning (via unix or tcp socket).
>> - Rspamd scanning
>> - Greylisting and rate-limiting with redis upstream
>> - Auto-whitelisting (internal and via redis upstream)
>> - Replies check (whitelisting replies to sent messages)
>> - Passing messages and/or their headers to beanstalk servers
>>
>> I've been running this on my mailserver (amd64) for a few weeks
>> now and it works for me.
>>
>> thanks,
>>
>> .jh
> 
> Updated one - fix deps and use a PKGNAME that can move to a normal
> release once one has been done on the 2.10 branch without an ugly v0.

Thanks, agreed that's better.

> I still see the problem I was getting with the previous version
> though:
> 
> 2016-11-19T16:36:32.261Z symphytum rmilter[19335]: ; mlfi_data: 
> cannot get queue id, set to 'NOQUEUE'
> (Plus no evidence of it feeding the mail through rspamd..)

I don't see that on my end:

from /var/log/maillog

Nov 19 09:29:41 www rmilter[2887]: <22672d052d>; mlfi_data: queue id: 
<829A4CB7F>
Nov 19 09:29:41 www rmilter[2887]: <22672d052d>; mlfi_eom: 
tempfile=/tmp/msg.n6FTWSkH, size=2828
Nov 19 09:29:41 www rmilter[2887]: <22672d052d>; spamdscan: start scanning 
message on localhost
Nov 19 09:29:41 www rmilter[2887]: <22672d052d>; spamdscan: finish scanning 
message on localhost
Nov 19 09:29:41 www rmilter[2887]: <22672d052d>; spamdscan: scan, time: 0.188, 
server: localhost, metric: default: [-0.750 / 15.000], symbols: 
R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36], MIME_GOOD(-0.10)[text/plain], 
DMARC_POLICY_ALLOW(-0.25)[gmail.com], RCVD_IN_DNSWL_LOW(0.00)[], 
R_DKIM_ALLOW(-0.20)[gmail.com]
Nov 19 09:29:41 www rmilter[2887]: <22672d052d>; clamscan: start scanning 
message on /tmp/clamd.sock
Nov 19 09:29:41 www rmilter[2887]: <22672d052d>; clamscan: finish scanning 
message on /tmp/clamd.sock
Nov 19 09:29:41 www rmilter[2887]: <22672d052d>; clamscan: scan 0.013669, 
/tmp/clamd.sock, /tmp/msg.n6FTWSkH
Nov 19 09:29:41 www rmilter[2887]: <22672d052d>; msg done: queue_id: 
<829A4CB7F>; message id: <3d72cff0-2837-9dea-50f2-af04a1014...@gmail.com>; ip: 
2607:f8b0:400d:c09::22a; from: ; rcpt: 
 (1 total); user: unauthorized; spam scan: no spam; virus 
scan: clean; dkim: not signed, ignored

>From /var/log/rspamd/rspamd.log

2016-11-19 09:29:41 #93200(normal) <2c578b>; task; accept_socket: accepted 
connection from 127.0.0.1 port 38870, task ptr: 04A231124790
2016-11-19 09:29:41 #93200(normal) <2c578b>; task; rspamd_url_text_extract: got 
empty text part
2016-11-19 09:29:41 #93200(normal) <2c578b>; task; rspamd_message_parse: loaded 
message; id: <3d72cff0-2837-9dea-50f2-af04a1014...@gmail.com>; queue-id: 
<829A4CB7F>; size: 2828; checksum: <80cf205c674f4682ffc4789fdcbcae65>
2016-11-19 09:29:41 #93200(normal) <2c578b>; task; fuzzy_generate_commands: 
<3d72cff0-2837-9dea-50f2-af04a1014...@gmail.com>, part is shorter than 100 
bytes (2 bytes), skip fuzzy check
2016-11-19 09:29:41 #93200(normal) <2c578b>; task; bayes_classify: skip 
classification as ham class has not enough learns: 0, 200 required
2016-11-19 09:29:41 #93200(normal) <2c578b>; task; rspamd_stat_check_autolearn: 
<3d72cff0-2837-9dea-50f2-af04a1014...@gmail.com>: autolearn ham for classifier 
'bayes' as message's score is negative: -0.75
2016-11-19 09:29:41 #93200(normal) <2c578b>; task; 
rspamd_stat_classifiers_learn: <3d72cff0-2837-9dea-50f2-af04a1014...@gmail.com> 
contains less tokens than required for bayes classifier: 4 < 11
2016-11-19 09:29:41 #93200(normal) <2c578b>; task; rspamd_task_process: learn 
error: <3d72cff0-2837-9dea-50f2-af04a1014...@gmail.com> contains less tokens 
than required for bayes classifier: 4 < 11
2016-11-19 09:29:41 #93200(normal) <2c578b>; task; rspamd_task_write_log: id: 
<3d72cff0-2837-9dea-50f2-af04a1014...@gmail.com>, qid: <829A4CB7F>, ip: 
2607:f8b0:400d:c09::22a, from: , (default: F (no action): 
[-0.75/15.00] 
[DMARC_POLICY_ALLOW(-0.25),R_DKIM_ALLOW(-0.20),R_SPF_ALLOW(-0.20),MIME_GOOD(-0.10),RCVD_IN_DNSWL_LOW(0.00)]),
 len: 2828, time: 187.683116ms real, 4.045855ms virtual, dns req: 21


.jh

Re: [new] mail/rmilter

[Stuart Henderson]

On 2016/11/19 16:37, Stuart Henderson wrote:
> On 2016/11/19 11:13, Johan Huldtgren wrote:
> > hello,
> > 
> > rmilter is used to integrate rspamd with any milter-compatible
> > MTA, this is mostly taken from openbsd-wip with all the heavy
> > lifting by @sthen. I made some minor changes:
> > 
> > - Update from 1.9.2 to a later working version
> > - bumped the uid/gid as 780 which was originally in there is now taken
> > - changed the configuration files to live in /etc/rmilter/* as the
> >   recommendation is that you make changes to a .local file and/or create
> >   /etc/rmilter/local.d/ for further user/local customization
> > 
> > # cat pkg/DESCR
> > Rmilter is used to integrate rspamd with any milter-compatible
> > MTA, for example postfix or sendmail. Features include:
> > 
> > - ClamAV scanning (via unix or tcp socket).
> > - Rspamd scanning
> > - Greylisting and rate-limiting with redis upstream
> > - Auto-whitelisting (internal and via redis upstream)
> > - Replies check (whitelisting replies to sent messages)
> > - Passing messages and/or their headers to beanstalk servers
> > 
> > I've been running this on my mailserver (amd64) for a few weeks
> > now and it works for me.
> > 
> > thanks,
> > 
> > .jh
> 
> Updated one - fix deps and use a PKGNAME that can move to a normal
> release once one has been done on the 2.10 branch without an ugly v0.
> 
> I still see the problem I was getting with the previous version
> though:
> 
> 2016-11-19T16:36:32.261Z symphytum rmilter[19335]: ; mlfi_data: 
> cannot get queue id, set to 'NOQUEUE'
> 
> 

(Plus no evidence of it feeding the mail through rspamd..)

Re: [new] mail/rmilter

[Stuart Henderson]

On 2016/11/19 11:13, Johan Huldtgren wrote:
> hello,
> 
> rmilter is used to integrate rspamd with any milter-compatible
> MTA, this is mostly taken from openbsd-wip with all the heavy
> lifting by @sthen. I made some minor changes:
> 
> - Update from 1.9.2 to a later working version
> - bumped the uid/gid as 780 which was originally in there is now taken
> - changed the configuration files to live in /etc/rmilter/* as the
>   recommendation is that you make changes to a .local file and/or create
>   /etc/rmilter/local.d/ for further user/local customization
> 
> # cat pkg/DESCR
> Rmilter is used to integrate rspamd with any milter-compatible
> MTA, for example postfix or sendmail. Features include:
> 
> - ClamAV scanning (via unix or tcp socket).
> - Rspamd scanning
> - Greylisting and rate-limiting with redis upstream
> - Auto-whitelisting (internal and via redis upstream)
> - Replies check (whitelisting replies to sent messages)
> - Passing messages and/or their headers to beanstalk servers
> 
> I've been running this on my mailserver (amd64) for a few weeks
> now and it works for me.
> 
> thanks,
> 
> .jh

Updated one - fix deps and use a PKGNAME that can move to a normal
release once one has been done on the 2.10 branch without an ugly v0.

I still see the problem I was getting with the previous version
though:

2016-11-19T16:36:32.261Z symphytum rmilter[19335]: ; mlfi_data: 
cannot get queue id, set to 'NOQUEUE'




rmilter.tgz
Description: application/tar-gz

[new] mail/rmilter

[Johan Huldtgren]

hello,

rmilter is used to integrate rspamd with any milter-compatible
MTA, this is mostly taken from openbsd-wip with all the heavy
lifting by @sthen. I made some minor changes:

- Update from 1.9.2 to a later working version
- bumped the uid/gid as 780 which was originally in there is now taken
- changed the configuration files to live in /etc/rmilter/* as the
  recommendation is that you make changes to a .local file and/or create
  /etc/rmilter/local.d/ for further user/local customization

# cat pkg/DESCR
Rmilter is used to integrate rspamd with any milter-compatible
MTA, for example postfix or sendmail. Features include:

- ClamAV scanning (via unix or tcp socket).
- Rspamd scanning
- Greylisting and rate-limiting with redis upstream
- Auto-whitelisting (internal and via redis upstream)
- Replies check (whitelisting replies to sent messages)
- Passing messages and/or their headers to beanstalk servers

I've been running this on my mailserver (amd64) for a few weeks
now and it works for me.

thanks,

.jh

rmilter.tgz
Description: GNU Zip compressed data